There was a time when skeptics thought mice would never catch on. “People will never give up their QWERTY keyboards,” they said. They were half right: now we take both for granted.
Now, more experiments in multi-touch interfaces are appearing by the day. Aside from mysterious Apple patents, we have, via We Make Money Not Art, new research in multi-touch interactions from a team led by Jefferson Han. (Demos pictured.) This isn’t just any touchscreen: not only does it recognize multiple fingers as inputs, but it projects whatever imagery you want in response, enabling new, fluid interfaces, and even responds to force feedback.
Internal tracking allows you to use up to ten fingers at once — that could be both your hands, you and a friend, you and four friends using two fingers each, etc. That much we’ve seen before, in the form of the one shipping product in this category, Cycling ’74’s Lemur.
But this research project features key capabilities the Lemur lacks: instead of fixed, pre-defined, static interface objects, Han’s research features back-projected imagery. That blows open the possibilities of this kind of interface, because it allows the team to experiment with fluid, game-like interfaces that provide the illusion of the tangible world. You’ll see demos of modular sound synthesis, lava lamps, and various other ideas. In other words, the touchscreen is finally as flexible as the computer in the graphics department, only now no longer limited to a single point of interaction (the mouse cursor). Han again applies his Frustrated Total Internal Reflection technique to tracking, the magic behind the multiple touch points. But there’s another twist: force sensing, granting this interface the potential for more realistic expression.
I’m not convinced these interfaces will ever replace the tactile sensation mechanical devices provide, as found on knobs, faders, and musical keyboards. But as computer interfaces, they could fundamentally change the way we relate to software.
Now the question is, who will get to market first with an affordable, mass-market device? The beauty of this research project is that the back projection can be whatever you want. Musicians could benefit from the mass market purchasing such hardware for other applications. Then it’ll be up to us to figure out smart musical applications — sure, the technology is cool in itself, but think how many possibilities there could be for each of those demos?
What does a digital lava lamp sound like?
Multi-touch Interaction Research [Description, QuickTime movie]