Teaching has its moments of reward and frustration in equal measure, but I’m of the belief that there are no limits to the new ideas that can grow out of groups of engaged students. In fact, it’s often in the first draft, the first experiment, the first time a student has tried something that I feel like I see the greatest spark of new ideas. Freed of concerns about whether something has been done before (looking at you, jaded artists and comment trolls alike), liberated by the bone-crunching stress of deadlines (ahem, yes, they do have a motivating impact), students on their best days have tremendous energy. Smart, motivated students plus smart, eloquent teacher plus enthusiasm?
All of that is to say, I think as we go from one or two classrooms messing around with Kinect to more, some really extraordinary things will happen. So let’s look first at one of those handful of early-adopter classrooms to get a peek of what that might be.
Golan Levin, both a great teacher and a media artist with some serious contributions to the field, has unleashed his Carnegie Mellon classroom on the problem of thinking up what to do with Kinect. Assisted by friend of the site Dan Wilcox, they’ve been hacking hard. The results are playful, sometimes off the wall, and inventive. Here’s what the Interactive Art & Computational Design class has come up with so far — hell of a mid-term.
Project 3 is just a single computer vision assignment, but there are some terrific results. A few of my personal favorites (biased as I am toward certain interests):
We be Monsters by Caitlin Boyle and Asa Foster, top, turns participants into a two-person puppet, “inspired by multi-person Chinese dragon costumes and Snuffleupagus.” It’s actually a bit reminiscent of a work we saw in 2008 with the terrific open-source Animata, but naturally benefits from skeletal tracking in Kinect.
Magrathea by Timothy Sherman and Paul Miller demonstrates some of the potential for cameras in 3D sculpting, mapping depth to polygon meshes that are shaped into realistic terrains. The fundamental idea here reminds me of the tangible, sculptural analog in the pinscreens built for the Exploratorium. And yes, do they work in both a reference to The Hitchhiker’s Guide to the Galaxy and Close Encounters? You bet. Nerd up.
Alex Wolfe and Honray Lin’s Kinect Flock uses familiar particle techniques, but deserves a call-out here for being simply visually lovely. Also, I love Alex’s blog – understanding her pen-and-paper drawing aesthetic makes this more meaningful.
Maya Irvine, Emily Schwartzman, Ward Penney, and Mark Shuster deserve some Internet fame for their Batman-style word animation, irresistible to comic book lovers.
DMX lighting fans will love, and unions will hate, the Kinect-powered Automatic Spotlight by Samia Ahmed. Specs: “openFrameworks, Kinect, UsbDmxPro, Martin Mac 2k, Marek Bereza’s version of Eric Sjodin’s cppGlue/DmxPro addon.”
And, of course, there’s the obligatory projection mapping, which comes to us via Marynel Vázquez and Madeline Gannon. Technically, it’s working beautifully; with some great content, this could be stunning.
Essential tools
Various tools are in use here, but the most essential and complete place to begin is probably the killer combination of OpenNI, the open source implementation of natural interaction (including but not limited to Kinect) with elegant, artist-friendly, Processing-style C++ tool openFrameworks. The glue is the ofxOpenNI add-on:
https://github.com/danomatika/ofxOpenNI/tree/experimental (Dan Wilcox branch – updated with that URL)
— and Theo Watson’s ofxKinect: https://github.com/ofTheo/ofxKinect
(Processing is coming along, too … and new OpenCV libraries could help, too. Stay tuned.)
So, what do you think of this? And obviously, I don’t want this site to become Create Kinect Motion, but that’s all the more reason to ask, what would be most useful for us to cover?
What makes this all possible is the spirit of sharing, the fact that open source tools are not only accessible, but modifiable. I think that makes it all the more regrettable that Microsoft, while funding this class, is also banning any use of the GPL or any license like it on any documentation or file anywhere in any software for Xbox 360 or even Windows Phone 7. You just dream that whoever works in the legal and policy end of the Microsoft campus in Redmond could spend a day in one of the Research offices – or auditing this class. (I’m guessing, I should add, that the anti-GPL stuff will remain on the distribution side but not the upcoming official SDK – hopefully.)
Lots more information on these projects if you click through their Vimeo sites, and be sure to see:
http://golancourses.net/2011spring/projects/project-3-interaction/