In a whimsical proof of concept, artist and inventor Dan Wilcox harnesses the depth-sensing powers of the Kinect camera to turn a room full of drifting balloons into music. It occurs to me that the basic spatial model can be seen as descended directly from the Theremin – way to go, Leon, still relevant today. The sounds are simple, but it seems something you could continue to develop musically – to say nothing of what it could do for the ball pit at Chuck E. Cheese’s. (Slogan: Where a Kid Can Be a Kid Who Gets Obsessed With Skeeball Prizes / Get Scared Out of a Kid’s Mind By the Other Kids in the Ball Pit. Sorry, it’s an American suburban thing of a certain age, for the more than half of you who have no idea what I’m talking about.)

Full description from Dan:

Experiments in balloon motion and sound using an MS Kinect depth sensing camera.

Created for the Carnegie Mellon 1st & 2nd year MFA Graduate show entitled “Fresh Baked Goods” at Bakery Square, April 2011.

A machine stands in a room surrounded by balloons. Circulating fans blow the balloons over the machine which creates sound based on their movements.

Mode 1: Tones

Balloon height and x/y position control the pitch and panning of a treble and bass voice. The tones can be quantized into a certain key or a glisssando can be employed for a theremin-style effect.

Mode 2: 99 Luftballons

The playback speed of Nena’s 99 Luftballons is controlled by balloon height. The balloons must be kept in the air for the song to play. Feed the machine.

Built using Open Frameworks, ofxKinect, and Open CV for balloon tracking and Pure Data for sound generation/playback.

See​blog/​balloon-project for more info.

Dan has a master plan with a robotic music-playing suit and other ideas, so I can’t wait to see where this goes.

  • Michael Coelho

    Bakery Square is close to me, I wish I'd gotten a chance to check this out. As far as Chuck E Cheese's goes, I hated taking my kids there, it was totally scary. There is a CDM angle to this, Chuck E. Cheese's Pizza Time Theater was founded by Nolan Bushnell, the founder of Atari. 8-bit music and pizza anyone?

  • It's great to see people doing these sorts of music concrete installations.  At some point it will either evolve beyond what is conceivably possible for most, or it will become the internet version of an old parlor trick.  

    I realize I sound like a troll….I mean I love to see people doing these things, but I keep seeing projects mutations whereas people are controlling audio with electronic actuators.   I mean it's cool and all, but oftentimes not terribly useful.  In the end, not to sound traditionalist, but nothing beats striking a single note, on a well tuned, physical piano.

    Again, I'm a Max user.  I love this sort of thing……..