“I’m not particularly interested in typing prompts to make stuff, I’m interested in breaking them and dissecting them,” writes Jasper Shuoyang Zheng as he introduces Latent Terrain. Now using an intuitive, elegantly designed open-source Max external and UI, you can play the weird world of neural audio codecs like an instrument, transforming your own sounds into new textures.
There’s really a lot to say about this general area of activity. Unlike the hyped-up AI in music technologies stealing some headlines, the data source is your own sounds. And the processing is running locally, not in a water-sucking data center. And … well, it’s just beautiful and weird, and you can play using instrumental techniques.
Naturally, that is attracting musicians and experimenters, because it brings neural nets back to discovering new sounds instead of making everything sound the same.
Latent Terrain is a beautiful place to start — not least because it’ll be accessible to Max users, especially those already working with FluCoMa or Data Knot. It produces a visual map, a kind of warped texture, which you can move through however you like, from a mouse to a controller. You can even train your own small neural network directly in Max, and watch timbres shatter, fracture, and meld into new materials. You can carefully construct sound libraries and chart paths through them; it’s deeply personal.
Here’s a great example by Keigo Yoshida, just to get a sense of the sound — this one using the tech to sonify EEG readings.
There are also more than just demos, but artistic demos that push the technology as a way to unravel sonic meaning. I love Jiatong Liu’s work here, not least because my first impression of Beijing was being struck by the Hutongs, really wanting to walk forever through them.
Would love to hear more, Jasper, so consider this a first look! (The project will also be at NIME – The International Conference on New Interfaces for Musical Expression – in London later this month.)
Jiatong Liu‘s nn/mémoire is a virtual gallery soundscape built from archival recordings of Beijing’s Hutong neighbourhoods — a rapidly disappearing urban soundworld. The terrain becomes an ambient archive you move through spatially. Liu described “learning to deal with the unpredictability” as a central design question, not a problem to eliminate.
It’s also all exquisitely well documented, so dive in. There’s this article by Jasper, which is a good guide through the work and its possibilities:
Latent Terrain: Dissecting Neural Audio Codecs
And a project page, including the research:
https://jasper-zheng.github.io/nn_terrain/
Example Max for Live devices are inbound, and you can use different audio autoencoders for different results, with a few popular choices already supported. There’s been a bunch of action on the Stable Audio front, too, more generally, so I’ll try to catch up.
Download for Max (and Max for Live), on macOS and Windows. (And yeah, I’m sure someone is thinking about Pure Data, so stay tuned…)
https://jasper-zheng.github.io/nn_terrain/installation
Previously, and very relevant: