It was inspired by Nikolas Tesla’s radical ideas about energy in air – and site-specific opera. It breaks every notion you have of how to mix, how to set volume, and what “panning” or “stereo” means. It’s, specifically, the forest of metal columns filled with omni-directional speakers we’ve come to know as 4DSOUND. And it’s all coming to Amsterdam Dance Event in October in a big way.
But what’s most important about 4DSOUND isn’t just this particular, not-inexpensive and specific installation. It’s the fact that once you start imagining sound as virtually projected into three-dimensional space, you probably won’t really think about sound in the same way.
Taking something like a site-specific spatial audio system and putting it into an online video is a recipe for failure. But the team at Ableton have done a pretty bang-on job of doing just that in two films, one focused more on the system in general and its significance, and one on specifically how the technique works.
Various composers have worked on 4DSOUND; this film focuses on Stimming. That makes an interesting choice, because his set is so live. In his work, Ableton Live is mostly a control interface for the spatialization; its audio duties are limited to mixing in the system and adding some clips. Everything else is outboard, like the MFB Tanzbär drum machine, a Teenage Engineering OP-1, and an acoustic piano.
Just as important, 4DSOUND’s Paul Oomen, a classical composer, talks about the connections to Tesla and theater. See the deeper meaning introduced at top, then the technical – and thoughts for the future – below.
With that conceptual background, it’s likewise important to understand that this system is neither a surround setup like those in cinemas (most recently Dolby’s Atmos), nor Wave Field Synthesis.
Cinema sound is generally a different animal. Those systems, or crude systems like quad (or even stereo), are capable of spatializing sounds, but they’re dependent on listener position. Wave field synthesis is closer, in that it does produce virtual sonic locations, as if sounds are in specific places beyond the speakers, even as you move around. Wave field is also interesting in that it has been adopted by MPEG. But wave field synthesis, while very precise, works on a horizontal plane, and requires very specific settings and speakers. (Dolby Atmos is something of a hybrid, using both conventional multichannel and projected positioning via a proprietary system – but that’s a topic for another article, and because Atmos is still aimed at cinemas, 4DSOUND is worth considering on its own.)
4DSOUND takes a different approach, using something called vertical phantom imaging. By taking advantage of omni-directional speakers, they get the advantages of virtual projection – that illusion that sounds fill specific locations or volumes – without requiring so many speakers or particular environments. That makes a unique sound space in which artists can play, and while this isn’t cheap or yet ready for club environments, it is able to make it to festivals. 4DSOUND came to Berlin’s Atonal Festival last month, for instance, and in a series of events (including a lab co-hosted by CDM), will next head back to Amsterdam Dance Event.
I’ve been working with 4DSOUND now in my own music, in a collaboration with Robert Lippok, and it’s been a unique learning experience. I couldn’t agree more with Stimming that it can change how you listen to music and sonic environments. Stereo is artificial enough that it’s easy to lose sight of sounds in terms of how they exist in space. It’s simply too distant from how we hear. But when you can manipulate sounds in a virtual environment, you really begin to appreciate the spatial as a compositional element.
In our project, we’re working to use those elements to create our own virtual architectures. It’s a first opportunity to see how you might perceive architecture purely as sonic, non-physical form. We’re working with Berlin’s Arno Brandlhuber, who constructed a form in a proposal for housing that perfectly fits the grid of the 4DSOUND – real and virtual.
Above: Translating architecture into sound, in process on 4DSOUND. Photos by Robert Lippok.
As seen in the video, you’re not only positioning sounds: you can produce volumes, paths with motion, and create effects that are calculated around the space (for reflections, delays, and more). You can add Doppler effect and other filtering to enhance the illusion that sound sources are moving around you. You can create sonic perceptions that seem real, and others that would normally be impossible.
To implement this system, you’re granted per-voice controls of each sonic object. Ableton Live is a bit ill-equipped to work in this way; music software in general is built around mixers that assume stereo recordings are the end result. But those voices are represented by graphical controls added to an Ableton session, built in Max for Live. It in turn is a front-end, alongside a Lemur remote control communicating over OSC, for a back-end system that does the processing necessary to pipe 57 channels of audio out the RME audio interfaces to the amps. (The back end is built in Max/MSP, with apparently heavy use of gen~ DSP objects for performance.)
So many of our sonic habits have been constructed by the stereo mixdown and its crude virtual space that we may be unaware how much it impacts our composition and sound design. So it’s interesting to listen to a binaural recording of Stimming. You’ll want to not only listen to headphones, but be patient as the work builds up. Obviously, even binaural recordings don’t really capture the impact. But you will begin to hear panning that’s vertical, with a great deal of distance in the mix rather than the packed recordings common in dance music. This will be less evident if you haven’t heard the 4D in person, but a lot of the timbres you hear, the sense of these sonic objects in some real space and the way they reverberate, is also a feature of working in this way. It will no doubt transform habits producing and mixing even in stereo – once you’ve done this, you can’t ever go back to even mono and stereo in the same way.
Stimming explains:
Equipment used: MFB Tanzbär, Clavia Nordrack2, TeenageEngineering OP-1, Arturia Microbrute and AbletonLive as master clock, sampler and midi sequencer.
Everything on the 4D sound was tweaked by hand in real-time, as well as the whole arrangement. I preprogrammed some chords and grooves on my machines though.
The 4D System is an advanced spatial sound system and the set is binaural (also called dummy head) recorded – in order to get an idea of how it sounded you need to use your headphones.
For the full binaural experience, I made the lossless AIFF file available for download. Please note that the download is over 1 GB in size.
Imagine being INSIDE the music, and the sounds move around you in all three dimensions.
It really is thinking in four dimensions – the three spatial dimensions, plus time (and adding that fourth element truly feels like a fourth dimension).
And the 4DSOUND setup is complex enough to feel like an instrument, the combination of its spatial capabilities and various effects and live controls.
So, it’s significant that in Amsterdam, we’ll have a full program of new music for the 4DSOUND (including Stimming, a Raster Noton showcase including Robert and myself along Grischa Lichtenberger, Frank Bretschneider, and Senking), Max Cooper, and Vladislav Delay.
It’s just as important that we’ll have developers from Ableton joining a select lineup of artists and researchers of lots of backgrounds on Spatial Audio Hack Lab we’re co-hosting. We have everyone from doctoral experts in spatialization to singers.
This isn’t a gimmick or a fad or some cool new toy. There is a lot of work remaining to be done, on 4DSOUND and spatial audio in general. The 4DSOUND itself is a canvas for all kinds of work; it’s not obvious how to work with it or what it should do. Imagining how interfaces should look is a wide-open question. And on 4D and spatial audio in general, there’s a huge opening for people to suggest new ideas for sound, composition, performance, and control. That can relate to architecture, to data sonification, to simulation. In Paul’s case, sensors on singers can produce a new way of enhancing theatre with amplified and electronic sound, as audio follows performers.
And the whole field is about to blow wide open. New microphone and headphone technology could make 4DSOUND’s specific system still more relevant – a playground for challenging ideas that will become increasingly commonplace.
So, if you’re in Amsterdam, I hope you’ll join us. If not, we’ll keep piping these spatial possibilities to you.
Thinking in 3D – or 4D – will be a new challenge. Above, photos from our recent working sessions.
Amsterdam events:
http://www.facebook.com/4dsoundonline
More of the latest from 4DSOUND:
http://4dsound.net/news/