MSAFluid for processing (Controlled by iPhone) from Memo Akten on Vimeo.

TUIO is a simple but powerful emerging protocol for multitouch control for live music and visuals, as used on the powerful live tangible synth reacTable. Apparently no one told Apple, however. While the App Store rubber-stamps useless toys like fake cigarette lighter flames, they bizarrely rejected a powerful application by a leading digital artist that would enable standardized TUIO control – for free. (More back story below; see an example in action above.)

As a blogger, my reaction is usually to whine and pontificate, for better or worse. The engineering approach would be to find some hack away the problem. That’s what Andrew Turley did with the TUIO protocol. So, Apple won’t allow an app that does the trick? Why not go back to what developers did before the SDK, and just use the iPhone browser?

As Andrew explains it:

After reading the story I started thinking about seeing how far one could push Safari as an application platform, using web apps to get around Apple’s tight control of the app store. Since you would be connecting to another computer anyway to use an OSC application, why not just have the app be a web app running on a web server somewhere on the local network? The web server can then take care of things like sending out OSC messages or playing music or doing whatever it is people want to do.

To that end I created a little system that implements the TUIO protocol. You use an iPhone to run a web app, which in turn talks to the web server, which in turn sends OSC messages.



touchy feely [Pillowsopher Blog]

Needless to say, there are some downsides: you have to run the Python server, you’re more limited in input and control than you would be in a real app, and you’re stuck inside the Safari browser, which could be a bit inconvenient. So I’m not backing down from my original complaint – I don’t see Apple doing anyone a service by blocking this kind of app, and the only rational explanation seems to be that the folks doing the review process don’t understand what the app is.

Most importantly, I do know that all our griping was sent to Apple, so I’m hopeful the App Store will reconsider the decision once they get it.

But I love solutions, too, and this can be one for many applications.

It also illustrates an important point: the browser on mobile devices (Apple and otherwise) could be a powerful outlet. It doesn’t always make sense to build an entire application; there will be various cases in which a little browser tool will do a job. Need a quick remote control for a live performance / art installation / club lighting rig? You might try the easy solution with the browser first.

Thanks for the great hack, Andy, and I’m curious to hear if anyone uses or extends this.


Apple Rejects Free iPhone Tool For Artists Because of “Minimal User Functionality”

  • OK OK OK…
    Now, I am OFFICIALLY buying an Iphone!!!

    This APP ROCKS!!!

  • This is just what I needed to see today – -great work, Andy!

  • @tobamai: Apple requested additional documentation, including a demo, which he sent (see the video above). I mean, this is the problem I have — the process isn't documented, there aren't clear policies, and apparently Apple is of the opinion that they know better than the developers. Otherwise they'd fix this.

    The application itself is written to Apple's specifications, it follows policies for the store, and he provided copious documentation of what it was. So to me, it's the process that's borked.

    That said, I suspect if someone submitted something similar it might indeed get in, just because the handful of apps falling through the cracks seem to be doing so arbitrarily.

  • I should note, as well, the video above is something you CAN'T HAVE. That's using the app Memo built that Apple rejected. You can get close with this browser app, but it's still frustrating.

    My point is, I expect that we will see some experiments just using the browser on all platforms. And you can imagine that gets especially interesting on gaming platforms, which have *truly* restrictive app processes. But even on something like the Android, on which is is very easy to load an app, you might still have something that worked fine in the browser — and then that single webpage could also work with other mobile devices with similar browsers.

  • vitamon_k

    Anyone know the name of that song? I can't for the life of me remember.

  • martin brinkmann

    the song is of course "little fluffy clouds" by "the orb".

  • Pingback: Create Digital Motion » TUIO Multitouch Control on the iPhone: Now Via a Browser Hack, Since the App Was Rejected()

  • Andy

    If you happen to be in San Francisco, I will be talking about this app and some of my other work with the iPhone on Thursday night at the PygameSF meeting ( The meeting will be at the San Francisco Public Library, starting at 6PM. As an added bonus for CDM readers, Bret Truchan will also be at the meeting talking about using the Nintendo DS as a musical platform. If Bret’s name sounds familiar, it might be because he’s the creator of the very popular GlitchDS, which has been covered on CDM a few times.

  • tobamai

    My understanding is that apple wouldn’t approve the original app because they didn’t understand it’s merit or purpose for the end-user. If I’m wrong… well, the rest of this post is pretty useless 😉

    Has anyone considered packaging the iphone OSC client with a desktop-side visual demo so that the folks at apple see a “complete package” that has purpose and novelty for the end-user? The real gold would still be sending OSC to whatever you want by run software other than the demo. But to get it into the app store, include a demo so they see what it’s about.

  • Pingback: Sweet Multi touch visualization hack for 4-20 | As Seen Through PeriVisioN()

  • Hey Peter, thanks for the post again 😉

    Just a quick note, I really do think Apple just didn't realize what the app (msaremote) did. When I'd submitted the app there were no videos demonstrating it (though I did send them example clients, maybe they couldn't figure it out). I"m going to resubmit the app sometime this week when I get a chance, with more documentation and links to the videos, and hopefully it will be approved this time – I've also added a bunch of buttons and sliders, so should have a bit more than 'minimal functionality' 😛

    On a slightly different note, I think the "It's Apple's App Store, they can do whatever they like, and have every right to reject apps they don't like" attitude does make perfect sense, and I do agree with it. It is their app store, if they don't wanna host and distribute my app, fair enough. BUT if they don't wanna host my app, then they should bloody well let me host and distribute it through my own site! I'm not pissed off that Apple won't allow my app on the app store, I'm pissed off that Apple won't allow me to legally distribute my app to other people at all.

    And to all those who kept suggesting cydia: for this particular app, it defeats the purpose. MSARemote was partially intended for people visiting installations/concerts/events etc. So they could download a free app, and participate. Limiting the participation to jailbroken phones only sucks.

    P.S. what Andy's done sounds awesome, can't wait to try it out.

  • no.soul

    they are either morons or they already made an aggreament of some kind with adobe thus their rejection of your app.