AudioGL, a project teased in videos first in April and then again last week, is a new concept in designing a user interface for real-time music creation. Visuals and sound alike are generative, with the rotating, 3D-wireframe graphics and symbolic icons representing a kind of score for live synthesized music. The tracks in the video may sound like they’ve been pre-synthesized, polished, and sampled from elsewhere, but according to the creator, they’re all produced in the graphical interface you see – what you see is what you hear.

The newest video, released this week, reveals in detail the project’s notions of how to make a 3D, live music interface work. The UI itself is similar to other graphical patching metaphors, but here, like exploding a circuit diagram in space, routings and parameter envelopes are seen and edited in a freely-rotating view in three dimensions rather than on a flat plane.

There’s a reason interfaces like this have been few. Computer displays and pointing methods tend to be heavily biased to two-dimensional use, modeled as flat planes like pieces of paper. Working in two dimension is simply easier; there’s no reason you can’t take another layer of parameters and represent it on a two-dimensional interface. And rotating around in 3D space can make it difficult to keep your bearings.

Those challenges, though, don’t make this less interesting – they make it juicier and more delicious as design problem and stunning, futuristic musical model. Freed in three dimensions, a complex set of envelopes and parameters has room to spread out visually, making a kind of spatial score. This particular project strikes an interesting balance between traditional, iconic UI – operators are represented with graphic symbols – and more free-flowing geometry representing the sequencing and envelopes. To me, the latter is more compelling, but putting the two together may make the program more flexible and familiar to users of other music software.

What could knock you out of your chair, though, is the sheer depth of the software teased in the video. This is no simple tech demo: it’s an attempt to build an entirely new, live-synthesizing music tool from scratch in 3D. It’s like the International Space Station of music software, assembled in some void. I got a couple of tips on this today, and some are even wondering if it’s real.

It appears to be very real. Whether this particular tool is usable or not to me almost isn’t important: a spectacular failure in this arena would even be useful. Anyone waiting for some sort of “singularity” in music tech, I think it’s coming: it’s just going to be a singularity of human software ingenuity, explosive creativity and invention from independent developers. I can’t wait.

Stay tuned to find out more about this particular project.

See also the earlier video (not able to grab the embed code for some reason).

Thanks, Bodo Peeters, among others, for the tip.

  • B.

    Well, except for the 3D-Interface (which is the main point of the news) this looks a whole lotta like the Jeskola Buzz environment.

  • It looks like a lot of modular/patching environments, really. The neatest thing is the 3D view, which actually seems useful in presenting an overview of a project, but I could also imagine shoehorning that view into an existing DAW (like Ableton Live or whatever). I'm iffy about actually working in 3D though. Notice how all the patching and envelope editing (read: music production) is done in 2D in the video. The last thing you'd want to do is make music production as unwieldy and cumbersome as making 3D graphics.

    This demo evades all the hard questions of how to make a 3D interface for doing actual music production work (not just visualizing a preexisting project you've made in a 2D interface). Since our traditional computer tools and our displays are 2D, you might have to start by requiring a 3D monitor, a Space Navigator and a Novint Falcon. I dunno

  • Looks like it should be controlled with 30" multi-touch monitor.

  • DDDD

    Ridiculously cool! Totally seems like an interface that would work pretty well with a high resolution (and large touch screen).
    I am curious to try it and see if the workflow and visualization system is actually useful or is just visually stunning.

  • You're very welcome, Peter 🙂
    I think this looks great, the most 'immersive' app I've seen in a while, or should I state ever.
    Would be nice if you could colour-code the different signals, because it seems rather tricky to remember what goes where. 
    Can't wait to try this out!

  • salamanderanagram

    so cool i'm tempted to believe it's fake!

  • Holy Smoke ! It looks like the stuff i see in my dreams ..

  • Random Chance

    I don't get it. Where's the 3D in this? The basic structure you're working with still seems to be the good old directed graph which in most cases is even planar. At this point (after watching the video) I cannot imagine how I would derive any extra value from this quirky interface. 

  • Max

    Impressive. Regardless of the 3d stuff, it seems capable, fast-patched and sounds very nice. I particularly like the focus (and take) on automation; for all the live stuff out there, automation is still an art form in itself (I tend to do slower fadey stuff myself, but this is epic).
    One big question: will it be possible to view the automation side by side (as in a traditional daw) or only by skipping from window to window? Without that syncing stuff could be a real pain.

  • Max

    Also: joysticks for automation recording. Time to bring out the ole sidewinder.

  • shim

    -looks like fun. i like the intent most of all.
    -bro, seek funding in europe.
    -i don't do windows and likely the mac version a long time coming. so…
    -as long as it doesn't end up dance dance revolution, it's worth a look.

  • Tim

    Interesting concept … you'd really want to pair it up with a 3d screen and a kinekt interface too! I agree it's probably more of a cool way to ride chillwaves and buzz out on the track you wrote rather than trying to construct things in the 3d view?

  • "Anyone waiting for some sort of “singularity” in music tech, I think it’s coming: it’s just going to be a singularity of human software ingenuity, explosive creativity and invention from independent developers" 

    Hell yeah! 

  • Stij

    Holy crap, I don't even care if this is practical or easy to use, it's amazingly cool. And the idea of composing a song entirely with automation intrigues me (though I'm sure it's hardly new).

  • inspiral

    amazing work and exciting initiative, definitely worths a few bucks to support it!

  • moala

    Well, this looks like a 3D interface for classic (not especially 3D) audio. For 3D audio, maybe look for something like AudioStage:

  • Impressive 😀 I like where this is going! Hope it gets a start.

  • Sinus

    This seems to be too good to be true.


  • Imagine you are in a room and everything you see on that screen is actually holographic and is all around you, and you need only your hands to tweek stuff !!! How cool is that !!! like 2050 cool 😀

  • Maf

    Well Done

  • I hear you on the input limitations we're going up against. I'm stating the obvious, and you mention the pointer, but I wanted to draw this out a bit more: to this day, most of us have only one focus on the screen at a time: the cursor. With a track pad, you'd think we'd over come this mono way of interacting with things (that's where controllers come in.)

    Also, in terms of ergonomics, the consensus is that one is much more comfortable on a keyboard for long bouts of time than our hands touching a vertical screen. A word of caution, if you replace this with a table, you get no tactile feedback, no topography, as you would a mixing board, for example (which encourages muscle memory: partly why you can actually get better at playing a keyboard, for example. It's not a new experience every time to sit down to one, and you can feel where your hands are).

    Also, I would consider establishing some theoretical foundation for spacing objects-its one of these conceptual hurdles for me with other patching environments that this nebulous work area will end up looking organized and not arbitrary to other technicians (say like a flow chart on a presentation slide). A protools session, for example, needs no further translating to the next person down the chain. Now if you're going to navigate in that area, it might make sense to adhere to standard spacing, work in orbits, or shapes that are something analogous to an ideal controller (yes, even strings on a harp are color coded.)

  • Or a grid system, I might add. You need not base this size of the grid on the objects (or perimeter block spacing), and the blocks need not be uniform (they could be logarithmic, even, if it made sense to do so) you could do it more like the theoretics in Urban Planning. You could start somehow with the scope of the project, much in the way sim city starts with the constraints of the landscape, or you might want to look into Superblocks and taxicab geometry.

  • mat

    Cool stuff. And I love the music!
    But I ask myself where is the advantage of 3D here….just a visual effect waving around in playback mode? Can I set the sound generators in 3D space? (It shows just 2D) and wouldn´t it make sense to to give the distance between objects some meaning? (I guess reactable did so… like more away= less loudness)… hm, anyway – great work and keep it coming!

  • Have we any word yet on if there is a Kickstarter for this yet? I have never given $ for such campaigns but this would get me handing over funds with the quickness. I really need to get back to trying to make use of Jeskola Buzz.

  • Don't get me wrong, it looks good, it sounds good. But anyone who thinks it is practical, hasn't tried making 3D visuals in 3DSMax or similar software. Navigation and holistic mind  control of all parameters is painful. It's good that this option exists too, alongside the classic linear sequencers or 2D patching programs, but I really doubt this paradigm is going to substitute the present ones any day. What I do grant this idea is the ability to make you produce music using other parts of your brain. The outcome may not be pre-planned, and that's not a bad thing. I would like to see someone develop a stripped off, simpler version of this, just to create synth patches and sound design, and with bolder graphics (those nodes and patchcords look hard to grab).

  • rv

    I'm reminded of that (in?)famous "it's a unix system" from jurassic park. If there's ever an electronic musician character in a movie, I think we know what software they're going to be using 🙂

    Beauty aside, it's going to be hard to beat the functionality of 2D. Maybe innovative use of multitouch gesture interfaces can help solve the the control/input problem of navigation…

  • I agree that it does look like fun, but the 3d seems like it is not really integral. What initially got me excited about the idea was the possibility of patching in 3d, rather than displaying waveforms and automation data..

  • For other fun music-making in 3D, check out Q3osc, which uses OSC extensions in the Quake3 game engine, and the (apparently never released) OpenSoundEdit, which was an immersive 3D audio editor.

  • kkvv

    what really strikes me here is how he builds that track with one track and automation only .. 😮

  • Aaron

    Although the working method is similar those available in Buzz, Audiomulch, etc. the 3D visualization is interesting in showing the actually work being done with the automation. A good overhead view. Plus, I'm always welcome to a new modular host. There cannot be enough and the major companies are still behind in this area. Always leave it to indie devs to do something nice with a modular approach!

  • I wonder how possible is it to use the program for more traditional sounding music?