Like an animated equivalent of the Transporter on Star Trek, depth-sensing cameras and custom code can transform the human body into 22,000 points, then back again into something our mind again imagines as flesh and alive, even in abstraction. In “unnamed soundsculpture,” by Cedric Kiefer (onformative) and Daniel Franke (wearechopchop), the body takes on a new meaning in movement, a cloud of points, choreographed.
The making-of video itself tells a kind of story: first seeing choreographer/dancer Laura Keil and her ability to animate her own, human body live, then seeing that dance as data in three dimensions, rendered via a variety of tests before settling on the final form. The music, Kreukeltape by Machinenfabriek, produces its own space, crackling and hollow.
Cedric tells CDM:
We had a system of 3 [Microsoft] Kinects that recorded the point cloud data that was later combined in another custom-written application that was able to sync all three point clouds into one large file and export it into a file format that is readable by Krakatoa, a particle simulation plugin for 3d Studio Max.
From there, we added the particles, the camera movement that was also controlled by the audio and did a lot of post in [Adobe] After Effects in the end.
The dancer was Laura Keil, a fantastic dancer from Berlin; you’ll find some of her other performances on Vimeo, as well.
[Laura’s videos: https://vimeo.com/keili20]
Working with a dancer was an important part of the concept for us. Speaking for myself, I thought it was pretty important not always to work with digital data, turning sound directly into visuals, but to bring back the human
influence, her interpretation of what the sound feels like for her. She didn’t rehearse the performance but started to move and perform the very first moment she heard the sound. So everything that was happening, all the little mistakes, or delays of her movement — all are intended, and are part of the overall look and feel, [just as] the little noise that is created by the Kinect data that we recorded. To emphasize that and transport the idea visually, we finally chosen the rough, and noisy, grainy look that you see in the end.
I was curious about live performance, too; while this system is incapable of that, says Cedric, the idea holds some interest. I hope we’ll examine live possibilities with Kinect (and OpenNI, generally) soon.
The video has gone on to earn a nomination for Germany’s MuVi Award for music videos.
More:
Krakatoa plug-in for 3D Studio Max
http://www.onformative.com/work/unnamed-soundsculpture/