Spatial Sound, in Play: Watch What Hackers Did in One Weekend with 4DSOUND

The impressive, futuristic physical form of the 4DSOUND system. Photo: George Schroll.

You can’t really hear the results of the Spatial Audio Hacklab sitting at your computer – by definition, you had to be there to take in the experience of sounds projected in space. But you’ll probably feel the enthusiasm and imagination of its participants.

And that’s why it’s a pleasure to share the video documentation, produced for 4DSOUND by a team from FIBER – the Dutch audiovisual events and art platform – at Amsterdam Dance Event last month. In unleashing a diverse group of artist-experimenters on 4DSOUND’s unique speaker installation, we got a chance to create a sonic playground, a laboratory experiment in what people could do. It’s tough to overstate just how much those participants brought to the table – or just how little time they had. Actually working on the system was measured in minutes, forcing artists to improvise quickly with reality television levels of pressure. (Only, unlike TV show challenges, everyone kept their nerves and wits.)

4DSOUND Spatial Sound Hack Lab at ADE 2014 from FIBER on Vimeo.

To get through it, these artists focused on collaboration, finding ways of connecting essential skills. In the days and weeks leading up to Amsterdam, many of them fired missives back and forth wondering how best to exploit the spatial sound system. They then worked intensively to devise something they could try quickly, forming spontaneous teams to combine resources. They did in minutes what resident artists had done in days. With input from Nicholas Bougaïeff from Liine and a whole lot of guidance and assistance from the entire 4DSOUND team, in particular founder Paul Oomen, gathered hacekers managed to get a whole lot up and running. No project went silent; with tweaks, everything worked.

This wasn’t merely a show of coding prowess or engineering. Each project found some way to involve musical practice and sound, each was a “jam” as well as “hack.” That’s something different from the typical shape of hack days; these projects weren’t just demos. They were given a voice — sometimes literally singing, rather beautifully.

It was what we hoped for, and more. The Spatial Audio Hacklab was made in cooperation between myself and CDM, FIBER, the 4DSOUND team who have built the system and its software, and Liine (makers of the Lemur app), with support from Ableton (and their talented developer relations liason, Céline Dedaj). It followed a week of intensive artist projects on the 4D from Max Cooper, Vladislav Delay, Stimming, and even myself with Robert Lippok (on a bill with raster-noton labelmates Grischa Lichtenberger, Senking, and Frank Bretschneider). But it was also a kind of contrast to those performances and their accompanying master classes, one where any “what if?” question was game.

If days of programming hadn’t already convinced you, by the rapid-fire hacklab it was clear: a spatial sound system can be more than just a clever effect. It can feel like a venue, as unlimited in possibilities as the stage of a concert hall. It’s an empty box to be filled, in the best possible way.

Photo (during the raster-noton showcase) by Fanni Fazakas.

Hearing fully-formed improvised music was especially gratifying. But for me, perhaps the most promising result was the Processing-powered game of Pong whipped up by a coder team, because it validated the accuracy of listeners’ perception. Using sensors that had previously tracked singers, it involved players scurrying around to bang a sonic ball back and forth. (That, too, makes it a nice three-dimensional counterpart of InvisiBall, a similar sonic game by Hakan Lidbo – and, tellingly, that game has been played even by blind people. Thinking out of the box can mean inventing things that aren’t limited to the usual population and audience.)

Developers from Ableton have hacked a game of pong using spatial sound. #4dsound #ade14

A video posted by Peter Kirn (@pkirn) on Oct 10, 2014 at 8:29am PDT

Participant Will Copps also shared some of his thoughts after the experience, alongside lots of positive feedback we got from the hacklabbers:

The documentary does a good job of capturing the event, but is also incredibly impressive in capturing the various hack lab ideas and distilling them into a thirteen-minute piece… I’m very impressed. I know we could have spent more than thirteen minutes talking about each individual project.

As for additional thoughts, I’d stress that there was a clear benefit to just being there and hearing the system. While many of the ideas we had were specifically for the 4D system, hearing sounds through it challenges us to incorporate as many of the benefits of spatialization as we can into our current practices. The most obvious takeaway for me, at least to implement imminently, is exploring the possibilities of binaural recording. I may not be able to easily create immersion by placing firework recordings all around the listener like we did at 4D, but now it just seems foolish not to explore ways I can try.

It’s incredible when exposure to a technological development like this alters your perception of your practice in such a specific way. We’ve been fortunate to take lifetimes of those developments for granted in our art: the ability to record, to mix in stereo, to capture colors in video. I recognize that this distinct spatialization of sound is still a ways away from being as widely implemented as the developments I just mentioned. But to be one of the first to experience something like it is unreal. Comparing 4D to those developments may sound like (and may be) hyperbole… but I don’t think any of us knows for sure. And that’s perhaps what is most exciting.

Ana Laura Rincon, aka the DJ Hyperaktivist, echoed similar sentiments:

Some experiences in life must be lived so you really understand; in this case, the 4DSOUND is an experience that must be heard so you can have a real idea of the possibilities the system offers. And so we did in the Hacklab. The 4DSOUND is a very forward-thinking idea that allows you to control sound and its dimensions in the space, giving you the possibility of creating natural environments and also of making music with the space. The system gives you the possibility of listening to sound without being confined to a speaker or a stereo system; you hear sound as it is generated outside in everyday life. Having had the opportunity to participate in the HackLab, sharing ideas with amazing musicians, developers, and just great people, and being able to use the system and really understand how it works gave me a really good insight of the vast possibilities it has, and how the exploration of these is just starting. There is a new window a very big door just open for the future exploration of music, sound and how it can be experience and perceived — look out.

My feeling was, looking through the applications, that something would happen just by getting people together in the same room. Spatial audio is an nascent but evolving field, with a scattering of different systems. I’ll talk more about that soon, but the short version: they’re all rather different, from wave field synthesis to Dolby Atmos to the unique sound and physical presence of 4DSOUND. Amsterdam was a chance to reach human critical mass thinking about the problems, accelerating progress by connect people. It brought into one arena some of the most passionate researchers and artists, from those who have done doctoral dissertations on the topic to those curious to explore spatialization for the first time.

And that alone was transformative – as if everyone began dreaming in color for the first time, and then we all got to share the dream.

It’ll be terrific to see what’s next.

In the meanwhile, we can reveal the next playground: we’ll be in Berlin at CTM Festival making Tuning Machines in our third hacklab collaboration with that event in as many years, no doubt with a new group of collaborative spirits.

You can also have a listen to some audio impressions of the event, recorded in binaural sound.