With multi-touch fully exploited and the basics of camera vision largely understood, interaction moves to the realm of free space, “augmenting” your world with gestures that find some physical connection. They surprise by working in some way that seems intuitive and natural, somewhere away from what seems to be the realm of the computer. And early in the month of May, we see a flurry of new research in just this area. Not one but two projects from Microsoft hold potential, and one from MIT Media Lab is just … absurdly cool.
A summary:
- “Levitated Interaction Element,” out of the MIT Media Lab, uses a magnetically-floating ball as an interface. Advantage: free-form 3D interaction with physical, tangible feedback.
- “SoundWave,” from Microsoft Research and the University of Washington, uses an ordinary mic and speaker, with the aid of the Doppler Effect, to add natural interaction without additional sensors. Advantage: no special hardware needed for gestures – not even the use of a camera. Another win for the power of sound. (You can even listen to music at the same time.
- “MirageTable” from Microsoft Research builds on the growing power of Kinect by adding natural perspective and the illusion of real-world manipulation. Advantage: a more complete illusion. (though why aren’t they using back projection in the demo?)
More details. From top, “ZeroN” / “Levitated Interaction Element:
What if materials could defy gravity, so that we could leave them suspended in mid-air? ZeroN is a physical and digital interaction element that floats and moves in space by computer-controlled magnetic levitation.
http://media.mit.edu/~jinhalee/zeron
Title: Levitated Interaction Element
Researchers: Jinha Lee, MIT Media Lab Tangible Media Group,
in collaboration with
Rehmi Post, MIT Center for Bits and Atoms
Advisor: Hiroshi Ishii, MIT Media Lab Tangible Media Group
Institution: MIT
Gestures are becoming an increasingly popular means of interacting with computers. However, it is still relatively costly to deploy robust gesture-recognition sensors in existing mobile platforms. SoundWave is a real-time sensing technique that leverages a speaker and a microphone to robustly sense in-air gestures and motion around a device. It is capable of detecting a variety of gestures, and can directly control existing applications without requiring a user to wear any special sensors.
Researchers: Sidhant Gupta, Dan Morris, Shwetak Patel, Desney Tan
Institutions: Microsoft Research; Ubicomp Lab, University of Washington
In MirageTable, a 3-D stereoscopic projector projects content directly on top of the curved screen. The information is captured by the Kinect camera, which also tracks the user’s gaze. This enables presentation of correct perspective use to a single user on top of the dynamic changing geometry of the real world.
Researchers: Hrvoje Benko, Ricardo Jota, Andrew D. Wilson
Institution: Microsoft Research
Presented at ACM SIGCHI 2012
By the way, speaking as an immigrant in Germany, as an American citizen, and as a journalist covering international technological innovation, I’m always struck by reading the names in projects like this. Innovation is an international affair, and part of the strength of academic institutions (MIT, University of Washington) and corporate research (Microsoft) alike is its ability to pull talent from around the world to work together. Those benefits can produce wide-reaching opportunities for everyone – and everywhere – involved. Without knowing the specific background of these particular researchers, but based on what I see with the people I do know, I’d point to this:
Why America Needs More Immigrants [Wired Science; Jonah Lehrer writing last year for Wired and The Wall Street Journal]
It includes numerous factoids like this one: “In recent years, immigrant inventors have contributed to more than a quarter of all U.S. global patent applications.”
Worth considering as governments around the world debate immigration policy, freelancer policies, and other key decisions.
In particular, it seems that the many pieces for improving interaction design and natural interaction technology lie scattered around the planet, a bit like a Triforce of futuristic interaction. Now, of course, “patents” are one potential obstacle – the involvement of both corporations and academia in the patent system threatens just this sort of international exchange. But the progress of the ideas underlying those specific patents remains something that flows globally across a range of institutions.
And that pace of innovation can really make the Next Big Thing seem to emerge out of thin air.