Anyone for a game of Harmonix Mime Hero, with the Marcel Marceau expansion pack?
We’ve seen simple computer vision applications, “augmented reality” systems and object tracking schemes that use specially-printed tags, 3D tracking using IR emitters, and specialized motion detection sensors (most notably Nintendo’s Wii). But the holy grail, of course, is getting tracking without any of that stuff. That’s the idea behind the widely-anticipated release today of Microsoft’s Project Natal for Xbox 360.
What’s different about the new tracking systems that makes them work better? In short, a z axis. By detecting depth from the camera, you can track motion in three dimensions, which in turn makes detecting specific gestures far easier.
Microsoft had acquired 3D motion detection system maker 3DV Systems, as confirmed earlier this year on VentureBeat. Today’s news: that technology will see commercial distribution. Project Natal for Xbox 360 uses a three-camera device that interprets z-axis depth. Already, this leads to some impressive game demos. Of course, a big challenge of the Nintendo Wii has been that its sensors work poorly, but another challenge has been that developers often don’t use the sensors well, either. So it remains to be seen if developers figure out just what to do with this stuff.
There’s more, too:
- 3D motion detection and tracking
- Facial recognition (which could in turn lead to multi-person control experiences with this sort of technology, because you can tell the difference between different people)
- “Object scanning” – no mention of object detection, but this could mean tangible interfaces that don’t require special tags
Project Natal gets a brief announcement on the Xbox site with a sign-up for updates. As they put it:
No Strings (or Controllers) Attached.
Introducing Project Natal, a revolutionary new way to play: no controller required. See a ball? Kick it, hit it, trap it or catch it. If you know how to move your hands, shake your hips or speak you and your friends can jump into the fun — the only experience needed is life experience.
Xbox.com | Project Natal [with sign-up]
Great coverage of the announcement from our friends at Boing Boing Offworld:
E309: the 7 things you need to know about Microsoft’s press conference
I had expected this announcement today. But what I hoped was that we’d also get Windows compatibility, as with many other Xbox 360 peripherals. That was not announced – and I’m concerned that with the sophisticated tracking required, the analysis may happen in proprietary libraries for the Xbox and its (ironically) PowerPC-based processor. But I’m still holding out hope that we’ll see something like this for the PC. If we do, experimental interfaces from artists and adventurous independent game designers could become a possibility, instead of just EA on the Xbox. Ironically, we’ve seen cooler applications for the Nintendo Wii remote by independent musicians and digital artists than we have from major game makers. It’d be a shame if Project Natal didn’t find away to be open to similar innovation. If I hear anything, I’ll share it.
There’s still reason to believe that other technologies will endure in the near future for people doing interaction design, tangible interfaces, and alternative controllers for things like music and visuals – the sort we’re working on this week at our first-ever global hackday.
- Deployment and compatibility: Sure, this works on the Xbox 360 – a closed, proprietary gaming platform with pre-defined hardware specs. But people are working on interfaces for things like mobile phones, which have different processing requirements, and may need more compatibility than just the Xbox.
- Precision: The reason augmented reality and tangible tracking systems like the Reactable use printed tags is partly because these systems are very precise, particularly when it comes to tracking multiple objects. That precision may be necessary for certain applications.
- Low latency: Musicians using a system like the Reactable, for instance, may have a much more stringent sense of responsiveness. (As it happens, this impacts the feeling of other things, too.)
Of course, this is all really, really cool – it’s fantastic that these technologies are progressing so fast.