auextensions

While some Apple watchers fear the transformation of OS X into iOS, it would be more accurate to say that Apple has gradually been bringing its next-generation architectures to both platforms. And in the process, iOS is able to perform feats formerly only possible on the Mac.

What’s next: Audio Unit “extensions points.” And the upshot of this will be plug-ins on your iPad and iPhone (and perhaps someday Apple Watch), all using a new framework that will work on both OS X and iOS.

The feature was buried in a slide deck in the WWDC keynote and appeared shortly after in draft developer documentation in iOS 9.0. Now, keep in mind, that doesn’t mean it’ll ship in this form, or in iOS 9.0 at all – features have been known to slip. But it does provide an indication of where Apple is headed.

“Extension points” are a new way of providing third-party functionality inside an app. (Other examples include VPN functionality and add-ons for Safari and Spotlight.) These are most interesting in iOS for audio,though. Whereas combining audio tools has so far meant connecting apps via Audiobus or Inter App Audio, you’ll be able to install Audio Unit from the App Store and, for instance, add a delay effect to your favorite production app on the iPad.

Here’s the description from the iOS 9 documentation:

The Audio Unit extension point allows your app to provide musical instruments, audio effects, sound generators, and more for use within apps like GarageBand, Logic, and other Audio Unit host apps. The extension point also brings a full audio plug-in model to iOS and lets you sell Audio Units on the App Store.

Architecturally, this is no huge jump. Audio apps already make use of the Audio Unit API to talk to the audio stack in the OS (it’s actually how we implemented libpd). What was missing was a way to distribute Audio Units as add-ons for apps; this provides just that. It seems that’ll make sense for things like simple effects; more complex processing can still use inter-app audio features and dedicated apps — iOS 9 even makes that more convenient with split-screen multitasking.

Think more like 99-cent delay effects than something fancier, at least on iOS. Extensions will have a UI, but a very basic one. Apple hasn’t written audio-specific documentation yet, but they do caution extension developers in general to simplify: “An extension’s UI should be simple, restrained, and focused on facilitating a single task. To improve performance and the user’s experience, avoid including extraneous UI that doesn’t support your extension’s main task.”

Making a basic distortion add-on for other tasks should be nice, though.

So, for iOS, this is good news. But it also almost certainly reflects a change in direction from Apple on plug-ins on OS X – something we’ve long expected. Third-party developers had told me they thought Apple would push sandboxing for audio plug-ins on the desktop. And note that the Audio Unit extension point isn’t just for iOS; it’s for OS X, too. This could change the way plug-ins work on the desktop, though keep in mind that hosts from Steinberg, MOTU, Ableton, and others will likely continue to support some mixture of VST and existing AU formats (plus AAX and the like from Avid).

And if it allows App Store purchases on iOS, it seems desktop OS X App Store purchases of plug-ins can’t be far behind. I’m only speculating, but this seemed strategically inevitable the day GarageBand and Logic moved to the OS X App Store. Extension points would provide a technical facility to do so.

For developers, it means you can create plug-ins once and then sell them to desktop and mobile users at the same time.

What that means for users actually consuming those plug-ins: well, that can only be answered when developers have had some time to work with the new operating systems.

If you are a developer, you can read up on this in the iOS 9.0 release notes and App Extension Programming Guide.

  • shane

    I wonder of this means that someone will create a way to send midi out of an AU, allowing the step sequncers and arp in plug ins to drive other plug ins and/or hardware?

    • Good question, but I think unrelated to this format. Would be great if they resolved that…

      • shane

        This could be exciting for the prospect of using an iPad to act as a plugin host, especially if Apple finally allows us to use an cable to interface with the iPad instead of wireless. I know there are 3rd party solutions out there (and have them) but it would be ever so nice if they gave us serious music type people what we need to make the iPad live up to it’s potential for music creation. Especially live, I just can’t rely on a wireless connection to my iPad on stage and if it doesn’t work 100% of the time, it doesn’t get on stage with me.

        • sak

          sideways: check music io for midi+audio over cable.. just discovered recently by accident, super useful. surprised there’s no mentioning here.. (instead we get 5 posts about apple :P)

          • lala

            this has latency
            the new thing won’t
            as I understand it

          • Zymos

            “The new thing” that no one even has a clue what it actually is? Much less, how much latency it will introduce?

          • lala

            meh, i guess something which has to talk to a sever app before it talks to a host is slower than something that doesn’t have to do that?we’ll see 🙂

          • Zymos

            Oh right, you must be using a Mac. Music IO on Windows doesn’t even need a server, it’s all done through a VST. Hopefully Apple can come up with some solution so it’ll all perform better for you….

          • foljs
          • Zymos

            I know- as per my comment above, it needs a server daemon on OSX, but only requires the VST on Windows.

          • oliver

            apple have built inter device audio into iOS 9 and Elcapitain without the need for any apps, daemons or plugins. skip to 37min 18sec of this video https://developer.apple.com/videos/wwdc/2015/?id=507

          • oliver

            Theres MidiMux and AudioMux too, they are cheaper than Music IO and better looking! Peter wrote about them back in Feb

            http://midimux.com

            https://itunes.apple.com/gb/app-bundle/studiomux-bundle/id976702033?mt=8

          • sak

            Oh thanks! Missed that.. It guess that apple exposed the cable connection to the api..
            Seems to have better looks yes, but price is the same here and audio io is audio/midi together in one app (usability win). Setting up also is super easy, no gui required at all (i’m on osx 10.10)

  • shane

    I wonder of this means that someone will create a way to send midi out of an AU, allowing the step sequncers and arp in plug ins to drive other plug ins and/or hardware?

    • Good question, but I think unrelated to this format. Would be great if they resolved that…

      • shane

        This could be exciting for the prospect of using an iPad to act as a plugin host, especially if Apple finally allows us to use an cable to interface with the iPad instead of wireless. I know there are 3rd party solutions out there (and have them) but it would be ever so nice if they gave us serious music type people what we need to make the iPad live up to it’s potential for music creation. Especially live, I just can’t rely on a wireless connection to my iPad on stage and if it doesn’t work 100% of the time, it doesn’t get on stage with me.

        • sak

          sideways: check music io for midi+audio over cable.. just discovered recently by accident, super useful. surprised there’s no mentioning here.. (instead we get 5 posts about apple :P)

          • lala

            this has latency
            the new thing won’t
            as I understand it

          • Zymos

            “The new thing” that no one even has a clue what it actually is? Much less, how much latency it will introduce?

          • lala

            meh, i guess something which has to talk to a sever app before it talks to a host is slower than something that doesn’t have to do that?we’ll see 🙂

          • Zymos

            Oh right, you must be using a Mac. Music IO on Windows doesn’t even need a server, it’s all done through a VST. Hopefully Apple can come up with some solution so it’ll all perform better for you….

          • foljs
          • Zymos

            I know- as per my comment above, it needs a server daemon on OSX, but only requires the VST on Windows.

          • oliver

            apple have built inter device audio into iOS 9 and Elcapitain without the need for any apps, daemons or plugins. skip to 37min 18sec of this video https://developer.apple.com/videos/wwdc/2015/?id=507

          • oliver

            Theres MidiMux and AudioMux too, they are cheaper than Music IO and better looking! Peter wrote about them back in Feb

            http://midimux.com

            https://itunes.apple.com/gb/app-bundle/studiomux-bundle/id976702033?mt=8

          • sak

            Oh thanks! Missed that.. It guess that apple exposed the cable connection to the api..
            Seems to have better looks yes, but price is the same here and audio io is audio/midi together in one app (usability win). Setting up also is super easy, no gui required at all (i’m on osx 10.10)

  • lala

    drumroll 🙂

  • lala

    drumroll 🙂

  • Somewhat related – inter-app audio on Mac OS X is still a bit tricky, whereas it’s getting better and better on iOS. I am hoping for inter-app audio improvements on Mac OS X soon – JackOSX works pretty well (I’ve had some issues with Soundflower), but I’d love to have Apple actually supporting something. Meanwhile, inter-app MIDI is still super reliable and useful!

  • Somewhat related – inter-app audio on Mac OS X is still a bit tricky, whereas it’s getting better and better on iOS. I am hoping for inter-app audio improvements on Mac OS X soon – JackOSX works pretty well (I’ve had some issues with Soundflower), but I’d love to have Apple actually supporting something. Meanwhile, inter-app MIDI is still super reliable and useful!

  • midiflow

    There is some new information about Audio Units on iOS: http://midiflow.com/blog/audio-unit-extensions.html

  • midiflow

    There is some new information about Audio Units on iOS: http://midiflow.com/blog/audio-unit-extensions.html

  • oliver

    Apple has posted an hour long video about Audio Unit Extensions on their developer site. A lot of it is about the code but we do see some of the Audio units in action. Including how Garageband on iOS will handle them. Looks really killer and a much better implementation than inter-app-audio! check it out…. https://developer.apple.com/videos/wwdc/2015/?id=508

  • oliver

    Apple has posted an hour long video about Audio Unit Extensions on their developer site. A lot of it is about the code but we do see some of the Audio units in action. Including how Garageband on iOS will handle them. Looks really killer and a much better implementation than inter-app-audio! check it out…. https://developer.apple.com/videos/wwdc/2015/?id=508

  • Samip Magar

    Hy !