Music lovers are hacking Apple’s Siri voice recognition technology. By connecting to some of the “intelligence” of the cloud, these tools can make your phone rap or send music files to a player piano for instant musical playback.
First up: Yamaha’s piano taking requests, thanks to music grabbed online. A Yamaha rep explains:
Yamaha consultant Craig Knudsen demonstrates a unique implementation of Apple’s incredible Airplay technology in an exciting new way.
Here’s how it works:
Take a standard MIDI songfile and convert it to an audio file (while maintaining the MIDI data). The songfile is then sent wirelessly via WiFi to an Apple Airport Express (which is mounted underneath a Yamaha Disklavier reproducing piano. The audio output of the Airport Express is then connected to the analog MIDI inputs of the Disklavier, using a standard audio cable.
Then, you simply ask Siri to play your favorite song from your iTunes library, and Siri responds immediately, by making the Disklavier’s keys and pedal move up and down, recreating the performance, including full orchestration.
The result is nothing short of magical.
Of course, the actual “playing” is thanks to the capabilities of the Yamaha Disklavier. I’m actually a bit puzzled as to how the online conversion works, exactly, and I was curious for any Disklavier-owning CDM readers whether this is something publicly available. I’m waiting to hear back from Yamaha.
And now, for something completely different: Siri rapping. (Somewhat … erm … badly, if amusingly. It is a hack.)
My friend Robert “Robb” Böhnke had a lot of fun combining Siri’s voice synthesis and the lyrics of Notorious B.I.G.
My hack for the Music Hack Day 2011 in London, a 24h Hackathon for all things music.
SiriProxy is used to intercept the communication with Apple’s servers. Notorious Siri then sends Notorious B.I.G.’s Hypnotize to the device.
Siri’s speech synthesis is synced to the beat using the timestamps obtained from the Echonest API which were then manually tweaked, to smooth out delays in the text-to-speech engine.
Thanks to Universal Music for awarding me a nice pair of Dr. Dre headphones
The Echonest API, by the way, is an amazing do-everything “API for music,” one that analyzes musical files and connects to a vast storehouse of musical intelligence.
Warning: this video is most definitely Not Safe For Work. (Heck, even the thumbnail isn’t, exactly.) If that concerns you, just go listen to the Yamaha video again, okay?
Siri FAQ [Apple.com]
Previously: Google Translate Beatboxing
…and, of course, that means: Android, your move.