I’ve been blogging effectively every day for nearly five years, so it’s hard to avoid novelty – that’s kind of what we publish. The question is, is it the novelty that’s important, or do you see these as little steps toward something greater that hasn’t happened yet? I tend to favor the latter.

Here’s the question: can augmented reality – using computer vision tools to mix computer graphics with stuff in the real world – become a real medium and not just a gimmick?

If it works in a game, you begin to think there’s really something to an interactive design. I’m not certain yet about the upcoming PSP game from giant Sony. Georgia Tech researchers, on the other hand, are coming up with stuff I really want to play.

Oh, yeah – and it makes me crave some Skittles. (Product integration, anyone?)

About the game:

ARhrrrr is an augmented reality shooter for mobile camera-phones, created at Georgia Tech Augmented Environments Lab and the Savannah College of Art and Design (SCAD-Atlanta). The phone provides a window into a 3d town overrun with zombies. Point the camera at our special game map to mix virtual and real world content. Civilians are trapped in the town, and must escape before the zombies eat them! From your vantage point in a helicopter overhead, you must shoot the zombies to clear the path for the civilians to get out. Watch out though as the zombies will fight back, throwing bloody organs to bring down your copter. Move the phone quickly to dodge them. You can also use Skittles as tangible inputs to the game, placing one on the board and shooting it to trigger an explosion.

Skittles could be the computer interface of the future. See also a music sequencer made with Skittles. (The only problem, as your inner Cookie Monster might quickly point out, is when you eat the interface.)

It’s still just an experiment, but the interaction itself starts to really require the technique – and, in turn, it becomes something that actually gets you closer to the game.

For visualist artists and performers, there’s something really powerful and expressive here, too, which is the opportunity to remix “reality” itself (or at least what you can see through a live video feed) with computer-imagined graphics, all live.

It’s even more impressive to see this running on mobile devices. And we’re not far from this kind of performance on current mobiles, either.

You’ll find lots of other projects on the Augmented Environments Lab site, and plenty of other ideas. There are technical solutions, like building an augmented reality scenegraph for easier development. There are concepts for integration, like bringing augmented reality into the virtual reality world of Second Life. There are virtual reality pets and mixed reality games, too – and even a concept working with the MIT Media Lab’s Scratch that makes augmented reality serve teaching programming in a more tangible way for kids. (Somewhere, game developers are looking at it thinking it could be a great way for grown-ups to prototype games.)


AR Scratch: A Tangible Programming Environment for Children

Developers, there really are some great concepts there in terms of how to develop both programming concepts and mapping of imagery to the AR-tagged objects. Not only does this have educational potential, but it could help develop notions of how interactive programming in general could work, and how our closed digital boxes relate to the world around them. Figuring out just how that works is not a small problem – it’s the kind of problem you could spend a lifetime thinking about and developing, on a timescale we can easily miss in the blogosphere.

But I’ll close with a project called Joe Warpin, from Georgia Tech’s handheld augmented reality class, the inspiration for the zombie game. It reminds me of the days as a young kid when I spent hours roaming the universe inside a room in my house with Matchbox cars and toy planes. That brings up an important question that often gets left out: the question of imagination. On one hand, you could say that getting overly focused on augmented reality takes out the ingredient of imagination, by working too hard to substitute technology for the imagined virtual reality in a child’s mind.

I’d say just the opposite: I think imagination is what gives augmented reality real potential. If you can meet the user halfway – actually stimulate their imagination – and leave just enough to their head, I think AR could be not only successful, but addictive.

  • neb

    Ah, I can see giant AR playmats for kids to build their toy towns and lego metropolises over, animating the scenes through handheld magic windows. Imagine legions of lego armies, pew-pewing starwars style as dust clouds billow among the low-flying ships glinting on the horizon…

    On a more practical level, network the handhelds and you've got the platform to add useful interaction to maps and blueprints. Maybe this approach will have success with architects and community planners where projected information systems haven't (yet).

  • Joost

    maybe we can leave the handheld magic window and wear glasses, nice to have both hands free

  • Koen

    @Joost: exactly!

    See-through glasses with added video and sound is when true immersion can really happen. Hopefully, that won't be too scary 😉
    Anyone know what's the current state of these things? Still limited to cockpit display systems?

  • jack

    I like the idea of a sim city style game for kids, playing on a placemat and positioning elements to see changes in the system. It doesn't have to be a shoot em up zombie game, and from the looks of this video, maybe it shouldn't?

  • four words:
    augmented reality tabletop wargaming.

  • I was just reading a blog post on generally speaking about this and had to search for more stuff about it. I am really excited about the possibilities this brings up, it seems like it could be a winner for non gamers much like the Wii was, I hate shoot em up games on normal consoles but the the AR version looked fun!