ActionScript programmer Peter Kaptein has done some brilliantly creative work to mimic the infamous gestural interface in the film Minority Report using only Flash, a webcam, a printer, and your fingers. (Okay, you may want to pick up some Scotch tape, too.)
The tricks to make it all work (as I see it, anyway):
1. Treat distance as “pressure” for gestures.
2. Use two markers, allowing for multi-finger manipulation of the interface.
3. Create combined actions – and provide lots of visual feedback.
4. Don’t work as much with transformations perpendicular to the screen – by avoiding some of the nuances of these, he navigates around the problem of losing marker tracking.
You’ll also notice he’s got a lot of light.
But the results are simply stunning. And a lot of it isn’t technical trickery, but smart interface design, controlled use of the gestures, and the use of extensive visual feedback to keep the user connected to the experience.
In other words, in a world that may seem all about engineering prowess, good design is still essential. (And that, to me, was part of what made the movie’s vision of the future so compelling in the first place.)
He has code excerpts for each of the individual tracking techniques. It should be possible to port those code snippets to your language of choice – or simply to be influenced by some of the ideas – and go on to do something very different from this project. In other words, augmented reality is finally evolving beyond the Hello, World stage.
“Minority Report” interface using Flash and FLAR toolkit
If you are interested in these techniques, I taught a short workshop on doing this stuff in Processing last week; once I’ve refined my code a bit, I’ll share here.
Thanks to Miguel Isaza via Twitter for the tip!