Inside a new immersive AV system, as Brian Eno premieres it in Berlin

“Hexadome,” a new platform for audiovisual performance and installation began a world-hopping tour with its debut today – with Brian Eno and Peter Chilvers the opening act.

I got the chance to go behind the scenes in discussion with the team organizing, as well as some of the artists, to try to understand both how the system works technically and what the artistic intention of launching a new delivery platform.

It’s not that immersive projection and sound is anything new in itself. Even limiting ourselves to the mechanical/electronic age, there’s of course been a long succession of ideas in panoramic projection, spatialized audio, and temporary and permanent architectural constructions. You’ve got your local cineplex, too. But as enhanced 3D sound and image is more accessible from virtual and augmented reality on personal devices, the same enhanced computational horsepower is also scaling to larger contexts. And that means if you fancy a nice date night instead of strapping some ugly helmet on your head, there’s hope.

But if 3D media tech is as ubiquitous and your phone, cultural venues haven’t kept up. Here in Germany, there are a number of big multichannel arrays. But they’ve tended to be limited to institutions – planetariums, academies, and a couple of media centers. So art has remained somewhat frozen in time, to single cinematic projections and stereo sound. The projection can get brighter, the sound can get louder, but very often those parameters stay the same. And that keeps artists from using space in their compositions.

A handful of spaces are beginning to change that around the world. An exhaustive survey I’ll leave for another time, but here in Germany, we’ve already got the Zentrum für Kunst und Medien Karlsruhe (ZKM) in Karlsruhe, and the 4DSOUND installation Monom in Berlin, each running public programs. (In 2014, I got to organize an open lab on the 4DSOUND while it was in Amsterdam at ADE, while also making a live performance on the system.)

The Hexadome is the new entry, launching this week. What makes it unique is that it couples visuals and sound in a single installation that will tour. Then it will make a round trip back to Berlin where the long-term plan is to establish a permanent home for this kind of work. It’s the first project of an organization dubbing itself the Institute for Sound and Music, Berlin – with the hope that name will someday grace a permanent museum dedicated to “sound, immersive arts, and electronic music culture.”

For now, ISM just has the Hexadome, so it’s parked in the large atrium of the Martin Gropius Bau, a respected museum in the heart of town.

And it’s launching with a packed program – a selection of installation-style pieces, plus a series of live audiovisual performances. On the live program:

Holly Herndon & Mathew Dryhurst
Tarik Barri
Lara Sarkissian & Jemma Woolmore
Frank Bretschneider & Pierce Warnecke
Ben Frost & MFO
Peter van Hoesen & Heleen Blanken
CAO & Michael Tan
René Löwe & Pfadfinderei

Brian Eno’s installation launches a series of works that simply play back on the system, though the experience is still similar – you wander in and soak in projected images and spatial sound. The other artists all contributed installation versions of their work, plus a collaboration between Tarik Barri and Thom Yorke.

But before we get to the content, let’s consider the system and how it works.

Hexadome technology

The two halves of the Hexadome describe what this is – it’s a hexagonal projection arrangement, plus a dome-shaped sound array.

I spoke to Holger Stenschke, Lead Support Technician, from ZKM Karlsruhe, as well as Toby Götz, director of the Pfadfinderei collective. (Toby doubles up here, as the designer of the visual installation, and as one of the visual artists.) So they filled me in both on technical details and the intention of the whole thing.

Projection. The visuals are the simpler part to describe. You get six square projection screens, arranged in a hexagon, with large gaps in between. These are driven by two new iMacs Pro – that’s the current top-of-range from Apple as we launch – supplemented by still more external GPUs connected via Thunderbolt. MadMapper runs on the iMacs, and then the artists are free to fill all those pixels as needed. (Each screen is a little less than 4K resolution – so multiply that by six. Some shows will actually require both iMacs Pro.)

Sound. In the hemispherical sound array, there are 52 Meyer Sound speakers, arranged on a frame that looks a bit like a playground jungle gym. Why 52? Well, they’re arranged into a triangular tesselation around the dome. That’s not just to make this look impressive – that means that the sound dispersal from the speakers line up in such a way that you cover the space with sound.

The speakers also vary in size. There are three subwoofers, spaced around the hexagonal periphery, bigger speakers with more bass toward the bottom, and smaller, lighter speakers overhead. In Karlsruhe, where ZKM has a permanent installation, more of the individual speakers are bigger. But the Hexadome is meant to be portable, so weight counts. I can also attest from long hours experimenting on 4DSOUND that for whatever reason, lower frequency sounds seem to make more sense to the ears closer to the ground, and higher frequency sounds overhead. There’s actually no obvious reason for this – researchers I’ve heard who investigated how we localize sound find there’s no significant difference in how well we can localize across frequency range. (Ever heard people claim it doesn’t matter where you put a subwoofer? They’re flat out wrong.) So it’s more an expectation of experience than anything else, presumably. (Any psychoacoustics researchers wanting to chime in on comments, feel free.)

For comparison: there are other installations of this type. Dolby Atmos uses a similar configuration for movie theaters – 64 separately-addressable speakers – but with Dolby’s proprietary processing software controlling it, and only in full Atmos cinemas. (And as with past Dolby tech, this gets bundled together with other tech, like mobile, that lacks that full speaker configuration.) See Dolby’s own explanation of what that’s about. The principle is the same, though: by using more channels and sending them more precisely to individual speakers, with some maths done behind the scene to pan the sound across 3D space and process it accordingly, your ears are able to perceive more details in space. (And your ears, by the way, are very, very good at this, whether trained or not.)

Audio interfaces. MOTU are all over this rig, because of AVB. AVB is the standard (IEEE, no less) for pro audio networking, letting you run sound over Ethernet connections. AVB audio interfaces from MOTU are there to connect to an AVB network that drives all those individual speakers. There are some six MOTU 24Ao interfaces in play connecting that huge array.

Sound spatialization software. Visualists here are pretty much on their own – your job is to fill up those screens. But on the auditory side, there’s actually some powerful and reasonably easy to understand software to guide the process of positioning sound in space.

It’s actually significant that the Hexadome isn’t proprietary. Whereas the 4DSOUND system uses its own bespoke software and various Max patches, the Hexadome is built on some standard tools.

Artists have a choice between IRCAM’s Panoramix and Spat, and ZKM’s Zirkonium.

On the IRCAM side, there’s not so much one toolchain as a bunch of smaller tools that work in concert. Panoramix is the standalone mixing tool an artist is likely to use, and it works with, for example, JACK (so you can pipe in sound from your software of choice). Then Spat comprises Max/MSP implementation of IRCAM spatialization, perception, and reverb tools. Panoramix is deep software – you can choose per sound source to use various spatialization techniques, and the reverb and other effects are capable of some terrific sonic effects.

Zirkonium is what the artists on the Hexadome seemed to gravitate toward. (Residencies at ZKM offered mentorship on both tools.) It’s got a friendly, single UI, and it’s free and open source. (Its sound engine is actually built in Pure Data.)

Then it’s a matter of whether the works are made for an installation, in which case they’re typically rendered (“freezing” the spatialization information) and played back in Reaper, or if they’re played live. For live performance, artists might choose to control the spatialization engine by sending OSC data, and using some kind of tool as a remote control (an iPad, for example).

I’ve so far only heard Brian Eno’s piece (both the sound check the other day and the installation), but the spatialization is already convincing. Spatialization will always work best when there are limited reflections from the physical space. The more reflected sound reaches your ear, the harder it is to localize the sound source. (The inverse is true, as well: the reason adding reverberation to a part of a mix seems to make it distance in the stereo field is, you already recognize that you hear more direct sound from sources that are close and more reflected sound from sources that are far away.)

Holger tells CDM that the team worked to mediate this effect by precisely positioning speakers in such a way that, once you’re inside the “dome” area, you hear mainly direct sound. In addition, a multichannel reverb like the IRCAM plug-in can be used to tune virtualized early reflections, making reverberation seem to emanate from beyond the dome.

In Eno’s work, at least, you have a sense of being enveloped in gong-like tones that emerge from all directions, across distances. You hear the big reverb tail of the building mixed in with that sound, but there’s a blend of virtual and real space – and there’s still a sense of precise distance between sounds in that hemispherical field.

That’s hard to describe in words, but think about the leap from mono to stereo. While mono music can be satisfying to hear, stereo creates a sense of space and makes individual elements more distinct. There’s a similar leap when you go to these larger immersive systems, and more so than the cartoonish effects you tend to get from cinematic multichannel – or even the wrap-around effects of four-channel.

What does it all mean?

Okay, so that’s all well and good. But everything I’ve described – multi-4K projection, spatial audio across lots of speakers – is readily available, with or without the Hexadome per se. You can actually go download Zirkonium and Panoramix right now. (You’ll need a few hundred euros if you want plug-in versions of all the fancy IRCAM stuff, but the rest is a free downloads, and ZKM’s software is even open source.) You don’t even necessarily need 50 speakers to try it out – Panoramix, for instance, lets you choose a binaural simulation for trying stuff out in headphones, even if it’ll sound a bit primitive by comparison.

The Hexadome for now has two advantages: one, this program, and two, the fact that it’s going mobile. Plus, it is a particular configuration.

The six square screens may at first seem unimpressive, at least in theory. You don’t get the full visual effect that you do from even conventional 180-degree panoramic projection, let alone the ability to fill your visual field as full domes or a big IMAX screen can do. Speaking to the team, though, I understood that part of the vision of the Hexadome was to project what Toby calls “windows.” And because of the brightness and contrast of each, they’re still stunning when you’re there with them in person.

This fits Eno’s work nicely, in that his collaboration with Peter Chilvers projects gallery-style images into the space, like slowly transforming paintings in light.

The gaps between the screens and above mean that you’re also aware of the space you’re in. So this is immersive in a different sense, which fits ISM’s goal of inserting these works in museum environments.

How that’s used in the other works, we’ll get to see. Projection it seems is a game of tradeoffs – domes give you greater coverage and real immersion, but distort images and also create reflections in both sound and light. (On the other hand, domes have also been in architectural use for centuries, as have rectangles, so don’t expect either to go away!)

The question “why” is actually harder to answer. There wasn’t a clear statement of mission from ISM and the Hexadome and its backers – this thing is what it is because they wanted it to be what it is, essentially. There’s no particular curatorial theme to the works. They touted some diversity of established and emerging artists. Though just about anyone may seem like they’re emerging next to Eno, that includes both local Berlin and international artists from Peru and the USA, among others, and a mix of ages and backgrounds.

The overall statement launching the Hexadome was therefore of a blank canvas, which will be approached by a range of people. And Eno/Chilvers made it literally seem a canvas, with brilliantly colored, color field-style images, filling the rectangles with oversized circles and strips of shifting color. Chilvers uses a slightly esoteric C++ raytracing engine, generating those images in realtime, in a digital, generative, modern take on the kind of effects found in Georges Seurat. Eno’s sounds were precisely what you’d expect – neutral chimes and occasional glitchy tune fragments, floating on their own or atop gentle waves of noise that arrive and recede like a tide. Organic if abstract tones resonated across the harmonic series, in groupings of fifths. Both image and sound are, in keeping with Eno’s adherence to stochastic ideas, produced in real-time according to a set of parameters, so that nothing ever exactly repeats. These are not ideas originated by Eno – stochastic processes and chance operations obviously have a long history – but as always, his rendition is tranquil and distinctively his own.

In a press conference this morning, Eno said he’d adjusted that piece until the one we heard was the fifth iteration – presumably an advantage of a system that’s controlled by a set of parameters. (Eno works with a set of software that allows this. You can try similar software, and read up on the history of the developers’ collaboration with the artist, at the Intermorphic site.)

What strikes me about beginning with Eno is that it sets a controlled tone for the installation. Eno/Chilvers’ aesthetic was at home on this system; the arrangement of screens fit the set of pictures, and Eno’s music is organic enough that when it’s projected into space, it seems almost naturally occurring.

And Eno in a press conference found a nice metaphor for justifying the connection of the Hexadome’s “windows” to the gallery experience. He noted that Chilvers’ subtly-shifting, ephemeral color compositions exploded the notion of painting or still image as something that could be consumed as a snapshot. Effectively, their work suggests the raison d’etre that the ISM curators seemed unable to articulate. The Hexadome is a canvas that incorporates time.

But that also raises a question. If spatial audio and immersive visuals have often been confined to institutions, this doesn’t so much liberate them as make a statement as to how museums can capitalize on deploying them. An inward-looking set of square images also seems firmly rooted in the museum frame (literally).

And the very fact that Eno’s work is so comfortable sets the stage for some interesting weeks.

Now we’ll see whether the coming lineup can find any subversive threads with the same setup, and in the longer run, what comes of this latest batch of installations. Will these sorts of setups incubate new ideas – especially as there’s a mix of artists and engineers engaged in the underlying tech? Or will spend-y installations like the Hexadome simply be a showy way to exhibit the tastes of big institutional partners? With some new names on that lineup for the coming weeks, I think we’ll at least get some different answers to where this could go. And looking beyond the Hexadome, the power of today’s devices to drive spatialization and visuals more easily means there’s more to come. Stay tuned.

Institute for Sound and Music, Berlin

Martin Gropius Bau: Program – ISM Hexadome

In coming installments, I’ll look deeper at some of these tools, and talk to some of the up-and-coming artists doing new things with the Hexadome.