One reason to use code, and the constellation of open source creative coding libraries, is the ability to express ideas across media. HoloDecks is a beautiful experiment in doing just that. The work translates invisible sound to three-dimensional form, and combines visualization in the physical and virtual domains.
HoloDecks goes through multiple phases of expression, built entirely in the multi-platform, free and open source OpenFrameworks.
Analysis of the song. First, there is a spectral analysis of the selected tune – in this example, it’s “Zebra,” by Oneohtrix Point Never.
Virtual three-dimensional visualization. Next, the spectral data is plotted in a disc – the creators say they were inspired by CDs.
Physical object. The 3D visualization is then 3D printed, using yet another library for OpenFrameworks. The result: a tiny sculpture of the song.
Augmented object with app. Finally, virtual and physical, momentary and real-time, are merged into one. Point the custom OF-developed app at the 3D object, and it is un-frozen, colored images dancing along its skyline of imagined sonic towers.
Watch:
HoloDecks – Augmented Sound Sculpture from lukasz on Vimeo.
The augmented reality portion is thanks to a library wrapping Qualcomm’s Vuforia (QCAR).
https://github.com/julapy/ofxQCAR
More on the project:
HoloDecks – Augmented Sound Sculpture [Realareal]
