This is about as affordable and easy as gestural interaction with music can get. The powerful Geco music controller app pairs with the $80 Leap Motion hand tracking hardware – and now the app is free.

But it could be just the beginning.

For its part, the Leap Motion is now sort of yesterday’s news. But the small rectangular box is still a quick-and-easy way to get your computer tracking hand gestures – if you’re into that sort of thing. Geert Bevin’s Geco app provides the glue between the Leap’s sensing capabilities and your music software, allowing the computer to recognize gestures and then convey them as MIDI or OSC messages (among other tricks).

And if for some reason you had a Leap and waited to pick up the app – or if you needed an excuse to give this a play – now the app is free. (Since its release, it’s also had some major updates, so it’s worth another go even if you tried it before.)

I’ve played with Geert’s app before, and it’s fairly impressive. You’re always going to be a tough critic of any sort of gestural interaction, because the link between hands and perception is so finely tuned. But the Leap opens up some possibilities – even if you don’t really want to wave your hands around for a whole performance, it could add the ability to perform quick shortcuts or control a single parameter. And it’s a huge advance in comparison to things like Roland’s IR-tracking technology, for instance.

But it’s what’s coming round the bend that may be most interesting. The reason Geert had to make Geco free at this particular moment is that Leap is killing its app store. (See their blog post on the topic. It’s not the most elegant “sunsetting,” but then it seems the whole industry had to get over this idea that everyone should create an app store as Apple had.)

Leap are moving on to take the software and hardware smarts of the Leap Motion and start to build it into two new (overlapping) arenas – mobile and VR.


Right away, in fact, you can use the Leap Motion with Windows and Android VR headsets. (The, erm, sophisticated integration technology there is a “universal adapter” that involves just mounting the Leap Motion to the headset itself – plastic and 3M adhesive.)

The thing is, the Leap Motion is kind of cool when tethered to a computer, but way more interesting when it’s set loose. And that’s the next step, with something upcoming that Leap is calling the Leap Mobile Platform.

Think virtual reality and augmented reality – battery powered, untethered from a computer, and totally mobile.

For music, this is especially compelling as it opens up the possibility of new experimentation with interfaces. VR and AR have given us the visuals of what that could look like, but that’s meaningless without the ability to interact with those worlds.

Geert tells CDM he’s working in this direction: “I’ll be getting an early version in order to be able to take what I’ve learned from my GECO and GameWAVE Leap Motion apps and apply this to Mobile Leap Motion with VR and AR,” he says. “I’m really interested in the AR part for live performance.”

AR is augmented reality – that is, a visualization that you see atop the real world, instead of replacing your vision entirely. AR beats VR onstage, unless you want to shut yourself off from your audience with enormous goggles.

In the meantime, there’s no need to wait – you can use Geco right now, provided you can get your hands on a Leap Motion. And with Apple having unveiled its augmented reality solution last month, and a bunch of parties jumping on VR and AR on Windows, Android, and beyond for gaming and other experiences, we’ll be watching to see whether musicians find a way to use these technologies in coming months.

  • Max

    Augmented reality?
    Are we really talking about this?
    The thing where you run around a table or something like an idiot, watching virtual stuff through the display of the iphone or iPad?
    (Apple hired a guy from Zeiss so it looks like they are interested in glasses too)
    Ok the tec is there,
    but where are the use cases?
    It’s a solution in search for a problem to solve.
    Ok, maps & location stuff, but what else?
    Im open minded, show/tell me something useful that solves an actual problem.
    I guess that’s the question everybody is asking now.
    What to do with this?

    • Polite Society

      Did you see the augmented reality thing that they prototyped with the deepmind 12? It was pretty clunky looking, but it had promise.

      • Max

        No, I haven’t. Thx.
        I’m not a fan of waving your arm in the air/ in a void.
        It’s the very early days of this and I am interested to see what ppl will come up with.
        Big clunky and sweaty helmets/glasses are definitely not my taste.
        MPE is the future I think, unsure about VR/AR.

        • Polite Society

          I don’t think it has to be exclusively coupled with waving your hands in the air. I can imagine some glasses which let you see information you would normally have to squint at a screen to see. Being able to see envelope shapes above knobs, active modulation moving through a synth. Floating Oscilloscopes while you work on your sound. Sequences and sequencers visible like the best DAW ever. I still feel like physical controls are the only sensible interface for instruments, however, as a performance modulation parameter, waving your hand in the air is very performative, and quite powerful when done intelligently.

          • Max

            I think we are in the same boat.
            I feel like controlling modulation is less weird if you don’t wave into the void. I like to modulate stuff with the accelerometer/gyroscope in iPad/iPhones …
            Than I have an object in my hand that gives me some kind of feedback about position in 3D space …

          • Max

            It doesn’t look as fancy as waving your hands in the air though , if you are after that.

  • Polite Society

    Interested for live control of parameters, but will wait until it’s possible to use it without a laptop.

    • Carlo Cattano

      It is possible but you need to use it in combination with a microcontroller