Multitouch Prototype 2 from Randy Jones on Vimeo.
2008 has been an amazing year for music technology. But I can’t bring myself to look back on it on this New Year’s Eve: not when there’s so much to look forward to in 2009. Case in point? An extraordinary, innovative new controller that in a matter of hours was already spreading among connected music technologists around the planet.
At the end of the day, it’s not hard to describe what you might want out of an expressive music controller. Most people would agree on that. The challenge is really an engineering problem. Solve the engineering problem in an artful way, and you can spend the rest of your time just practicing playing your invention. That’s what makes the above video so exciting.
Randall Jones has built a really elegant and wonderful multi-touch hardware controller, as reported by MAKE:blog (and picked up on Hack a Day). With $50 in parts and a lot of clever hardware design and software coding, Jones has built an interface that responds to both touch and pressure and, using some smart sonic mapping, can realistically reproduce instruments like the dumbek and guiro.
Intimate Control for Physical Modeling Synthesis [Project Page / Paper Abstract]
PDF, Randall Jones MSc Research Paper
Who needs a “top 10 technologies of 2008” post for CDM when this particular instrument could pretty easily top the whole list? Let’s just call it done, and uncork the champagne: major congrats, Randy! (This is a master’s thesis!)
Jones’ work does have some precedent, but just to review how much he’s accomplished here: he’s innovated in terms of the sensing, the form factor, the software interpolation, and the way in which the control data is mapped to a synthesis method. (Whew!) That has had a number of specific achievements:
- A clever form factor: The basic design here is elegant and could be adaptable to other form factors. Most importantly, the use of polypropylene and silicone rubber means the interface deforms nicely as you press it, giving you feedback. And that opens lots of other similar design possibilities. Translation: being squishy rocks.
- Multidimensional / force-sensitive input: “Multi-touch” usually only means something that can take multiple touch inputs at one time, as in, from multiple fingers. But as I’ve complained in the past, the problem is that most multi-touch interfaces, like the Lemur and iPhone, don’t respond to the amount of pressure or deform (providing tactile feedback) when you use them. That makes them feel a bit like an ATM screen that happens to take more than one finger at a time. Jones’ prototype responds to how hard you’re pressing or hitting it, and it’s more sensitive – more like a real-world instrument. Jones calls this “multidimensional,” but force-sensitive would be another way to look at it. Translation: it’s more organic than entering your ATM PIN code.
- Audio-rate resolution: By choosing to use audio signal for control rather than … well, anything else, Randy gets an extremely responsive control signal. The controller itself is passive, requiring no power. Everything is done by processing audio created by the sensors. That’s not a new idea, but by returning to it in this context, Randy makes a much more responsive controller than most touch and multi-touch controllers before it, and returns to some of the analog-style control of instruments like the Theremin and acoustic instruments. Translation: it don’t mean a thing if it ain’t got that swing.
- Smart centroid processing: To make touch points accurate, Randy wrote a new object for Jitter that does centroid calculation. Translation: Randy worked through the details.
- Waveguide mesh modeling for deep sonic results: None of this would be meaningful without meaningful sounds coming out of it. Here, Randy builds on previous work in creating a physical model of a drum head (back to the dumbek), but both refines the model and works intelligently through how to match it with the controller. Translation: it makes good noises.
There are some other related touch devices, each with their own strengths and weaknesses, but rather than reinvent his work, I’d suggest reading through Randy’s own research paper. He does a great job talking about what’s great and not-as-great about other research and products, and makes a terrific argument for his design choices. (Basically, see also the Haken Continuum, the Korg WaveDrum, the Tactex MTC Express, the CNMAT multitouch controller, and the Audio-Input Radio Drum.)
Now, if this kind of development had come about a few years ago, the next thing we’d be talking about is how this might be commercialized. Part cost is cheap – that’s good news. I don’t know Randy’s plans for licensing and future development. But we can already compare how this might have gone had this been introduced, say, a few years ago.
The old model:
1. Spend a couple of years more refining the prototype on your own.
2. Show off the work at an academic conference in Italy. People can see it if they’re associated with an academic institution that’s also willing to pick up their airfare and let them leave for a week. (Anyone who’s had to wrestle with their department to get a couple hundred dollars for a thousand dollars in actual expenses knows what I’m talking about here.)
3. Publish in an academic journal a handful of people get, so they can try to figure out what it’s like from diagrams and grainy black-and-white photos (and no sound). Oh yeah – no one can actually comment on the story, either, so advanced researchers lack feedback and newcomers lack context and commentary.
4. Get the prototype ready for NAMM. Blow a bunch of cash on a hotel in Anaheim and a NAMM booth.
5. Work out a complex, expensive production and distributi
on scheme that in turn jacks up the price. Hope someone finds it and can afford it.
6. To justify the cost, make the product as finished as possible. Sell it as a “Digital Dumbek” to eccentric, loaded rock stars.
Now, I don’t want to sound grumpy or naive. The truth is, some really brilliant instruments have been down this road. But I’m not sure the description above really did a whole lot for their brilliance. I’m not necessarily saying there aren’t benefits to some of the above techniques, but clearly they won’t work for everyone.
The new model:
“Where’s my flying car?” That’s the question that often comes up when technology and a New Year coincide. With music, though, I think what we’ve really been waiting for is a convergence of new technology and new communities. Flip through the Computer Music Tutorial’s section on new instruments, and you’ll see some fantastic, exotic, and often familiar new interfaces. So what has 2009 got that the last few decades didn’t? Think cut-rate, faster tech, and a connected Web community to develop, evangelize, and use new technologies.
1. Get 20,000 Vimeo views the moment you post your first demo. No one has to fly anywhere.
2. Connect with people doing research and experiments worldwide and get instant feedback and ideas and potential collaboration – even if they’re not publishing research or flying to conferences or are even academics.
3. Instead of assuming you’ll build a finished, closed product (hello, “Digital Dumbek”), figure people will want to hack the result. (Randy has already posted parts lists, so it’s possible to build this thing right now if you’re so inclined. And he also says in Vimeo contents he’s thinking about doing a version that transmits OpenSoundControl data, so you could use it to control other instruments, music, or even visuals.)
You’ve already skipped a lot of the steps that were previously necessary just to find interested parties. And that network could in turn be used to figure out how to make the thing around the world at a lower cost and get it directly to people who might want to buy it. It also assumes those people might hack the tool for very different applications, instead of having to target one particular kind of person.
Nothing about this is a panacea for innovation: most of the hard engineering problems remain, and this doesn’t mean you can magically create new products. But there’s no question that even changing the hurdles means there’s new potential. If 2009 is about anything, I hope it’s about people finding new solutions to taking that potential and tapping into it. You can bet it’ll be a major focus of this site.
Congrats, Randy – we’ll be watching.
Happy New Year.