One of the problems with touchscreens is that, even as they have become more sophisticated about tracking multiple fingers at once, they still generally don’t respond to pressure. To make touchscreens really useful for music, we need genuine pressure sensitivity.

For that reason, you may be intrigued to see this video of Zen Piano, a demo app for the iPhone and iPod touch. The idea: respond not only to the position of your finger taps, but also to how hard you’re tapping the phone That promises “velocity-sensitive” tapping, which would make touchscreen interfaces more powerful.

Here’s the somewhat overheated description by GreatApps, who say their “patent-pending,” “cutting-edge” technology is the result of “having gone through the research and development phases.”

TapForceTM has been developed from the ground up to provide a completely intuitive way of interaction for users. It can detect more than a hundred different levels of force, and has an accuracy that has to be seen to be believed. And all this can now be done in software, no hardware modifications are necessary. Hundreds of millions of devices currently on the market can make use of the TapForceTM technology today.

A whole new range of games and apps has just been made possible.

Okay, so what is it doing, exactly?

Most likely, it’s simply reading data from the accelerometer. Hit the device harder, and the accelerometer will respond to more force. That’s actually a fairly clever combination of two sensors – it’s just not the sort of stuff you’d necessarily want to trademark or try to get patented, at least, not if you’re a normal person. (TapForce creators, feel free to explain to us that you’re doing something fancier and I’ll eat my words.)

In fact, part of the reason I suspect that’s how they’re doing this is I’ve been tipped off by a developer who’s already implemented just this. He even uses a piano-style keyboard to show it off.

Photo: memotv.

Sadly, that developer and application is Memo and his MSA Remote application, which was inexplicably blocked from the iTunes store – I think because whoever would have understood the app was on a lunch break or something. See, previously:

MSA Remote for iPhone

Apple Rejects Free iPhone Tool For Artists Because of “Minimal User Functionality”

But as it happens, this is something any mobile device can do that has an accelerometer. I may try something like this on the Android app I’m developing. (No one can reject that, because Google allows any application package to be installed on the device should the user chose to do so. Perish the thought.) Accelerometer data alone is usually not very useful, but combined with touch, it could start to make more sense.

It’s another reason to look forward to MSA Remote, and I do still think that the snafu with Apple will get cleared up at some point. (Unfortunately, what we had on CDM were a lot of rants – perhaps even justified rants – but not necessarily the best way to make the argument to Apple’s store.)

  • ArmandoC

    whoa this is going to change everything!

  • Heheh, well unless you're being facetious — no, not necessarily. The problem is, measuring tap pressure in this way isn't terribly accurate, and there's no force feedback as with a physical pad, etc. So potentially useful and interesting, but no substitute for doing it in hardware.

  • Phil

    Could be by surface area too. Sort of like "velocity" is used to represent how "hard" you hit a key, the harder you hit it, the more your flesh compresses up against the screen.

  • Phil

    Actually I wrote that before I viewed the video. Definitely the accelerometer.

  • This is about velocity, not pressure — you can't easily do aftertouch if the system is using the accelerometer.

    Maybe fingertip deformation/area of coverage could be used as a basis for control.

  • Jon Simonton of Paia fame wrote an interesting article on touch switches, with speculation on how to make them touch-sensitive:

    Not all that related to this article, but of interest, nonetheless. I hope.

    I post this in lieu of a rant about the modern patent system which would be full of vitriol and, in the end, solve nothing.

  • You can't do finger deformation on the iPhone, certainly. But you could do that with a camera, and absolutely, do "aftertouch"-style deformation based on pressure. I've seen systems that do that. There are also other sensing methods that would sense pressure. So it's certainly a solvable problem.

    Of course, this is part of why we may see some touch applications that don't come from the mass market. There are plenty of solutions that could be powerful for music but wouldn't work in a commodity device.

  • tobamai

    Makes me wonder if it would just make your velocity erratic if you tried to play while riding a bus on a bumpy road.

  • flonk

    notice the soft surface in the video, might not really work without.
    Am I the only one who's not so excited about touhscreen controls after actually using an ipod?
    Lack of physical feedback being the main reason, but also your fingers covering the ui while operating?
    (You probably imagine me with giant hands now, but I assure you, i'ts all pretty average)

  • I'd like to see/hear a trill! I agree with the accelerometer suggestion and if this is the case, it probably doesn't get us very far in terms of larger touch screens. I wonder whether a larger touch screen could be mounted on some form of suspension (with an accelerometer attached). This might help a little, but I think trills might still be a problem.

  • Why not plug a piezo in the mic in and combine with touch sensor?

    Imagine a transparent piezo in juction with touch position (one piezo = one velocity touch at time), maybe this will be more acurate than accelerometer (and left it free for bendings ^^)

    Just wondering…


  • Russ

    Surely a keyboard like the one in Max/MSP (where velocity is dependent on the position of the click) would have a higher degree of accuracy?

  • I *really* hope TapForce doesn't get its patent granted.

  • Cool idea. Patent is a little far. Are they gonna pull everyone into court using the timed combination of touch input and accellerometer action? It's like patenting the use of combining streaming video and streaming audio.. sort of.

    However it's really great that they're eager to licence this technology for other developers.

  • JPV

    I have a hard enough time composing music on a 15 inch screen. I can't even to begin to comprehend the preoccupation with music apps on such a minute screen.

  • @Russ: Yeah, that makes much more sense, because then you can get predictable results.

    Following on @JPV's line of thinking… I think as controllers go, taking advantage of the positioning sensors on mobile devices winds up being more interesting, because then the fact that it's a handheld device starts to make sense (as with the Wii remote).

  • I just thought of another issue with this approach.

    There is only one accelerometer (yes, in three axes, but that's N/A to this discussion). While the display is multitouch, meaning you can play polyphonically on it, the internal accelerometer will only ever measure one velocity setting, which will be the total for all fingers smacking the display.

    As accelerometers are piezoelectric devices, I'm starting to imagine my own take on a velocity sensitive "touch" device using piezo film and maybe an Arduino. Man, too many project ideas, too little money.

  • Freddy

    let's hope they don't pull the patent thing on this one, synth devs removed mapping pitch scales to accelerometer just because the short sighted dev of (crappy app imho)Cosmovox, quoting dev:"filed complaints to Apple that violates his patent rights"

    wtf? accelerometer is available on every iphone, and for that matter accesible to any app via the SDK to do whatever you see fit with it as long as you stick to the public APIs, if anyone can (wrongfully) pull something like this, should be Apple, not a 3rd party developer!

    please Apple, don't let this kind of stuff happen.

  • Erm, already exists in an app on the appstore. I can't remember if it was a gong or a tambourine, but I ended up deleting it because it wasn't reliable enough. Wait, here it is:

    I'm sure it will get updated from time to time. Would be interesting to see these other guys patent something that already exists…

  • Darren!

    I said it before!


  • @Mudo: So you did. Sorry, I must have missed it.

    My biggest interest lately has been in creative uses of any piezoelectric materials. I've even given some thought to focusing my physics education on the field.

  • Pingback: Zen Piano para iPhone / iPod Touch: Boas perspectivas para os instrumentos musicais nas telas multitouch!! |

  • Don't worry Darren about me!!!
    I'm just kidding!

    It would be great see some of your ideas materialized!

    I'm researching and creating new user interfaces (hardware) prototypes arduino/pd based.


  • Pingback: TekDes » Blog Archive » il tocco non è tutto()