Adding hardware to tablets, as it has with decades of computing technology, can open up new worlds for software and music. It can animate a conventional piano, or provide new physical interfaces for touching music. But let’s not wait for it to happen; let’s get hacking.
Following on today’s line of thinking about hardware-augmented touch, I’d like to look a bit at the recently-transformed landscape on Android. iOS users can connect to external hardware via the Core MIDI protocol or, via official channels, through the Apple Dock Connector. That’s not a perfect situation, however. Hardware developers have to be approved through the Made for iPod program in order to make accessories, there’s no standard class support apart from MIDI, and while Bluetooth can connect you to other iPads, you can’t connect arbitrarily with hardware via Bluetooth. (At least, that’s what I’ve found speaking with iOS developers, though some Bluetooth gamepads appear to work, possibly because they simply act as keyboards). WiFi wireless connection is also a possibility, one I expect will continue to be exploited.
But Android does have some interesting options here – a reminder of why getting familiar with more than one platform can be enlightening. A lot of those possibilities could open up new ideas in music hardware, ideas that could work not only on Android but in some cases (as with standard USB support) with desktop and tablet Mac, Windows, and Linux machines, too.
Case in point: check out a Motorola XOOM controlling a piano above, for a kind of hacker’s Disklavier. (Apologies to Yamaha.) Yes, it’s a great deal messier than existing USB MIDI devices (more on that in a moment), but it’s a visual reminder of why we do this in the first place – and it doesn’t have to be quite this hacky on Android, either.
Simplified, your options are:
Android Open Accessory
Currently the best wired common denominator, the Open Accessory program allows you to use a third-party accessory as the USB host, even on devices that lack USB host support. You just need a device with Android 3.1, 2.3.4, or greater. In a much-ballyhooed feature, there are prototyping possibilities with the open Arduino platform. Google’s own hardware is obscenely pricey, though, at around $400. Instead, DIYers will want to use a standard Arduino. Two early examples:
Android ADK with a standard Arduino Uno and USB Host Shield
Google’s open accessory development kit on standard Arduino hardware
These prototypes could, in turn, be converted into simple kits or even standard off-the-shelf accessories, without asking anyone’s permission.
See a video at top for one of the Arduino Uno examples in action.
Also, for the actual code, look at:
UsbAccessory
More on DIY implementations that don’t require $400 hardware kits (more like … $40, tops):
http://developer.android.com/guide/topics/usb/adk.html
Microchip also promises an open PIC24 library with a driver for ADK
That said, because Google’s implementation is specific to Android, those accessories aren’t useful anywhere else. That’s why it’s worth considering USB host mode.
USB Host Mode
Tablets are the Android devices about which I’m most excited at the moment, because as with the iPad, the tablet devices wind up being more flexible and capable than their phone handset cousins.
One key feature: Honeycomb tablets support actual USB host mode. That means you can connect standard USB HID devices like joysticks. It also means you should be able to make a MIDI interface that isn’t kludged together from a bunch of proto boards and $400 Google Android kits and that works with Android but not your computer, as above.
In fact, the existence of this option made me a bit puzzled when I read Phil Torrone’s* editorial in Make. (See postlog, though, on why listening to Phil is still a good idea.) Phil focuses there on Arduino, and doesn’t mention standard USB host.
Why Google Choosing Arduino Matters and is This the End of “Made for iPod” (TM)?
Now, Phil’s correct that the Arduino is currently the easiest platform for hacking with this stuff. But I actually think even that could change. There are already a number of AVR-based platforms for doing USB host implementations. Like Arduino, you can develop for them with free, open tools and a wide community. Unlike a standard Arduino I/O board, though, it’s easy to create something smaller, cheaper, and more flexible – and to plug your creation into any device that supports standard USB hardware. Now, actually implementing some of these things is nowhere near as simple as Arduino at the moment, but that to me is a testing and documentation problem more than anything else. I’m bullish on the possibilities here; it’s part of the reason we chose a standard AVR platform for our MeeBlip synth and not Arduino. I’ve started messing with its firmware, finally; more on that soon – it’ll also be relevant to playing with DIY hardware for Android.
The Arduino stuff is absolutely cool, don’t get me wrong. I just think you could substitute “open hardware” more generally in that article for “Arduino,” specifically. (The article’s nonetheless well worth a read for some stimulating ideas.)
Executive summary: I betcha we can come up with a really simple DIY MIDI interface, via both wireless and wired connections, for Android that doesn’t touch the Arduino side of things and will work with your computer, too.
See:
android.hardware.usb in the Android documentation
IOIO – a Chat with the Developer
Following CDM’s coverage of the Google announcement of the ADK in the first place, a number of commenters pondered the inexpensive, DIY IOIO board marketed by Sparkfun (pictured).
Developer Ytai Ben-Tsvi shares some thoughts about how IOIO relates to the ADK:
First, you can see some of my (and others’) thoughts on this thread:
https://groups.google.com/d/topic/ioio-users/3JDn6XZJ9aE/discussion
Aside from what’s written there, I have immediate plans for implementing IOIO on top of ADK. This means that when IOIO is connected to an Android device, it will first try to establish and ADK (accessory-mode) connection with it and if that fails, will seamlessly attempt to fallback to ADB. ADB will still be used for firmware upgrades, as I believe this is the most secure way of making sure the user is involved in the decision to update the firmware (via enforcement of a single signed application that IOIO agrees to talk to).In the (hopefully near) future, a similar behavior will be achieved by connecting a Bluetooth dongle to IOIO instead of an Android device, enabling a remote connection between Android (or even the Android emulator!) and IOIO, on expense of reduced bandwidth and increased latency.
Some points to emphasize:
- IOIO is targeted at the hobbyist/educational community. I believe ADK is primarily targeted at serious hardware developers. As a result, in terms of support, design considerations, feature prioritization, ease of use etc., IOIO is more likely to stay loyal to the hobbyist community in the future.
- At the time of writing, IOIO seems to be the cheapest solution for communicating with an Android.
- IOIO currently provides a high-level Java API which hides underneath it a rich protocol for controlling many of the board’s features from Java. “Mainstream” users will never have to touch embedded programming when working with IOIO. My hope is that this API will eventually become a standard, and that people will implement it on top of other boards as well. If that happens, application developers will be able to easily port their applications to different hardwares.
IOIO is available now for US$49.95. Honestly, if you’ve got a phone that it works with, it’s a fantastic choice.
IOIO @ Sparkfun
Bottom line:At first it seemed like there is conflict here, but now it appears to me like there really isn’t! IOIO is an end-to-end solution for physical computing (a-la Arduino) from your Android device, ADK is a protocol for enabling you to connect peripherals to an Android device, and this is Google’s motivation in this game. The boards that came with ADK are just reference implementations, and to me they seem in no competition with IOIO at present. So IOIO and ADK will probably be good friend in the future, when ADK provides the underlying connectivity and IOIO provides the high-level platform for the hobbyists.
Updated: There’s already good news.
Since he wrote the comments above, Ytai got everything working with IOIO and ADK:
IOIO over OpenAccessory (ADK) Available [Ytai’s “Microcontrollers, Electronics & Robotics” Blog]
He makes a strong argument for why IOIO may be your best solution. In fact, for my money, IOIO plus USB host development is just about perfect.
Bluetooth
Enough about all these wires: you do lose some of the beauty of these mobile devices when you attach a lot of spaghetti to them. Enter Bluetooth. It’s reasonably cheap, light on power consumption and well-suited to embedded hardware (key differences with wifi). It’s also gotten steadily more robust. For wireless MIDI and wireless control, it really can work.
On Android, it’s possible to interface directly with hardware over Bluetooth. That opens up the chance to do, for instance, MIDI over Bluetooth without a wire in sight. For lots of thoughts on this, the best reading is Peter Brinkmann’s blog. (Peter is also the primary author of libpd.)
I hope we’ll get to talk more about this soon. Ahem.
http://nettoyeur.createdigitalmusic.com/
I’ll admit: I’m actually kind of a sucker for wires. So I’m glad to have Peter around to encourage me not to create giant, ugly tangles of crap.
Postlog – Phil Torrone and Mobile
I just want to add one background note on Phil Torrone.
Phil deserves loads of credit for seeing this stuff coming long before it was popularized. Here’s what he had to say in 2002:
A simple hiking stick was way too low-tech for Macromedia Flash expert Phillip Torrone.
So Torrone equipped his stick with GPS and Flash applications that let him identify trails and track information on local flora and fauna — and leave messages for other hikers.
He’ll show off the hiking stick and more of his quirky creations, such as a digital belt buckle made from an old Palm V, when he discusses wireless fashion at a Flash conference here Friday.
His inventions are sure to delight his geeky audience of developers and designers at FlashForward. But his audience will start taking notes when he brings out mass-market, Flash-enabled devices such as a Microsoft Pocket PC Phone Edition and an O2 XDA, two Pocket PCs that are also phones.
Flash developers believe mobile devices like these are the wave of the future, Torrone said. “Last year, wireless phones outsold PCs for the first time,” he said.
Lots of Flash, Even More Sizzle [Wired, July 2002]
In music, we have the opportunity to test these same ideas not only for a quick geeky hack, but as a real means of self-expression, in deeply emotional, intimate ways.
And just remember, for anyone caught up on platform debates, at some point even words like “iPad” or “Galaxy Tab” will seem as distant as “Palm” and “Pocket PC.” But the ideas behind actual work for those platforms will remain. (Oh, and… I guess we still have Flash, huh?)
Now … who wants to do some hacking this summer for music? (And science!)