This is about as affordable and easy as gestural interaction with music can get. The powerful Geco music controller app pairs with the $80 Leap Motion hand tracking hardware – and now the app is free.
But it could be just the beginning.
For its part, the Leap Motion is now sort of yesterday’s news. But the small rectangular box is still a quick-and-easy way to get your computer tracking hand gestures – if you’re into that sort of thing. Geert Bevin’s Geco app provides the glue between the Leap’s sensing capabilities and your music software, allowing the computer to recognize gestures and then convey them as MIDI or OSC messages (among other tricks).
And if for some reason you had a Leap and waited to pick up the app – or if you needed an excuse to give this a play – now the app is free. (Since its release, it’s also had some major updates, so it’s worth another go even if you tried it before.)
I’ve played with Geert’s app before, and it’s fairly impressive. You’re always going to be a tough critic of any sort of gestural interaction, because the link between hands and perception is so finely tuned. But the Leap opens up some possibilities – even if you don’t really want to wave your hands around for a whole performance, it could add the ability to perform quick shortcuts or control a single parameter. And it’s a huge advance in comparison to things like Roland’s IR-tracking technology, for instance.
But it’s what’s coming round the bend that may be most interesting. The reason Geert had to make Geco free at this particular moment is that Leap is killing its app store. (See their blog post on the topic. It’s not the most elegant “sunsetting,” but then it seems the whole industry had to get over this idea that everyone should create an app store as Apple had.)
Leap are moving on to take the software and hardware smarts of the Leap Motion and start to build it into two new (overlapping) arenas – mobile and VR.
Right away, in fact, you can use the Leap Motion with Windows and Android VR headsets. (The, erm, sophisticated integration technology there is a “universal adapter” that involves just mounting the Leap Motion to the headset itself – plastic and 3M adhesive.)
The thing is, the Leap Motion is kind of cool when tethered to a computer, but way more interesting when it’s set loose. And that’s the next step, with something upcoming that Leap is calling the Leap Mobile Platform.
Think virtual reality and augmented reality – battery powered, untethered from a computer, and totally mobile.
For music, this is especially compelling as it opens up the possibility of new experimentation with interfaces. VR and AR have given us the visuals of what that could look like, but that’s meaningless without the ability to interact with those worlds.
Geert tells CDM he’s working in this direction: “I’ll be getting an early version in order to be able to take what I’ve learned from my GECO and GameWAVE Leap Motion apps and apply this to Mobile Leap Motion with VR and AR,” he says. “I’m really interested in the AR part for live performance.”
AR is augmented reality – that is, a visualization that you see atop the real world, instead of replacing your vision entirely. AR beats VR onstage, unless you want to shut yourself off from your audience with enormous goggles.
In the meantime, there’s no need to wait – you can use Geco right now, provided you can get your hands on a Leap Motion. And with Apple having unveiled its augmented reality solution last month, and a bunch of parties jumping on VR and AR on Windows, Android, and beyond for gaming and other experiences, we’ll be watching to see whether musicians find a way to use these technologies in coming months.
https://developer.leapmotion.com/