Okay, first imagine that you can control drums with sound. Not a new idea; audio-driven software has been around for a while. Now imagine that the software is intelligent enough to learn from the sound input it hears. Bang a desk, clap your hands, hit your head against the wall, slap someone you don’t like repeatedly with a fish — it’ll adjust itself to the input. That’s the vision of a new project called BillaBoop.
The creator writes CDM to tell us more about it:
Hi, My name is Amaury Hazan. I’d like to introduce a software I have developed.
BillaBoop is a real-time audio driven drum controller which allows the user to control up to 3 drum instruments. The user can control any drum synth with the voice (beat box), or any object or musical instrument. Unlike other audio-driven systems wich require a lot of parameter tuning to be able to discriminate the sounds you are playing, BillaBoop incorporates an efficient Machine Learning component which enables the system to learn by demonstration.
Right now, this is just a technology demonstration, but it works on Mac, PC, and Linux, and could eventually be built into all sorts of other tech. It works in real-time, and recognizes up to three distinct kinds of sounds at a time. (Think, bang your friend with the fish, then kick your desk, then knock your head into a wall. You could assign one to a hat, one to a snare — well, you get the idea.) As Amaury suggests, drum controllers are only the beginning; other possibilities include music sequencing, games, interactive installations, sound-sensitive lighting, and so on.
Amaury had some talented folks in his thank you list, including plug-in developer Bram de Jong and music technologist Xavier Serra. He himself is a PhD student in Machine Learning, Music Cognition and Audio Signal Processing at Music Technology Group, a department of Pompeu Fabra University.
BillaBoop [Project Page]
Long-time readers will recognize a resonance between this project and Georgia Tech’s robot drummer Haile we featured some time ago, which analyzed rhythms so it could play “duets” with human musicians:
Robot Drummer Responds to Human Playing; How They Did It