Hey, look: a controller, right in your browser, with drag-and-drop editing. No app needed.
Tablet or phone or touch-enabled desktop computer – now it doesn’t matter. A free tool called (for the moment) Nexus lets you make any browser a canvas for music. iOS, Android, Windows, Mac – if the browser is there, your creations become omni-platform.
Shown at the NIME (New Interfaces for Musical Expression) conference in London earlier this month, Nexus is the most complete foundation for this idea seen yet. And since it’s free, it’s open to others to build upon. Right out of the box, it includes basic interface widgets obviously inspired by Lemur (and apps like TouchOSC), so you get faders and knobs and multi-touch arrangements of balls and keyboards and so on. But because it’s all built with Web tech, anyone can create any interface they imagine – with a custom look and feel, and complete with animations. And even in the browser, it uses OSC (Open Sound Control), for flexibility with a range of commercial or custom-built software.
Various demos are featured in a video:
There are a few ways you can work with Nexus, depending on whether you’re an end user or a coder, and which tools you prefer:
1. Drag and drop in a browser. Take any modern browser on desktop or tablet, and you can make your interfaces without coding at all using NxDrop. You can design right on the device, or build on a computer and then send to a mobile gadget.
2. Work with the browser to make sound. You can interface directly with the Web Audio API, meaning you can both create your interface in the browser and use it with sound.
3. Work with libraries like libpd. Another way to integrate interface and sound engine is with tools like libpd, the open source Pure Data engine, which you’ll see in some of the examples.
4. Make interfaces with – and for – Max/MSP. Integration features with Max make it really easy to automatically route messages, and even (in one demo) to use Max to help create the interfaces themselves. That should be welcome for both Max and Max for Live users.
5. Code with JavaScript. It’s clear that Nexus has some potential for non-coders right now, and that more ready-to-use applications will come in time if it attracts interest. In the meanwhile, though, if you do Web technologies as your day job, you’ll be happy as a pig in slop. With just a few lines of JavaScript and HTML5, you can code interfaces directly.
If you do want to give it a go – and you don’t necessarily know about code or Web server operation – there are a couple of useful video tutorials complementing the resources on the official site. These walk you Mac users (works reasonably well on Linux, too, or a Windows *nix setup of some kind) through the Web server bits and gets you up and running:
The advantage of all of this is, controllers can be distributed and shared and operate independent of platform. You could build a synth and let someone run it in a browser or a simple cross-platform app. You could have backups of your live rig run if your iPad breaks or is stolen. You can make mobile music ensembles, or give friends controls over a music creation for jamming. And you know your controllers aren’t tied to a single device or app – they’ll run on whatever you have, making them future-proof.
So far, Nexus is a starting point. It isn’t as polished in look and feel or installation as dedicated apps; you’ll need to do a little DIY (like installing a Web server, though there’s a good tutorial below). It receives data, but doesn’t send it yet, though that’s on the agenda. It seems some graphic designers could give the whole thing a better look, and the drag-and-drop app could be cleaner. And the name needs some work, because it’s too close to Google’s line of mobile devices. (Anyone got any ideas?)
But the technology works beautifully. And on a project-by-project basis, there’s no question it can begin to rival those better-known apps. There’s already a tool called Vox that converts existing TouchOSC layouts for use with Nexus.
There’s quite a lot of work done already – perhaps the most so far along these lines – thanks to the terrific labors of the team Ben Taylor, Jesse Allison, and Yemin Oh (of Louisiana State University).
And there are some interesting projects already. In addition to the video above, some of the LSU folks are working with the idea of laser-cut overlays, adding tactile feedback to tablets. I had a go on their iPads, and the results were impressive – it was still nearly as smooth running my fingers along the iPad, but my fingers could find their way to notes, as you would with frets or keys.
Lindsey Hartman has made a patch remixing of Imogen Heap, showing how Max and Nexus combine (photo via suncanon Instagram):
You’ll find all the code in a BSD license, with loads of documentation, on GitHub:
https://github.com/lsu-emdm/nexusUI
No need to dive into the code, though. At NIME, we had people up and running after just a couple of minutes following directions to install a Web server. The main site will take you straight to the getting started tutorials and free downloads:
Academics (or the technically curious), check out the full paper:
Simplified Expressive Mobile Development with NexusUI, NexusUp and NexusDrop
I’m curious to see what you do with this. And I do hope some intrepid Web designers reading this venture in, as well, or let us know if this might interoperate with other browser music projects.
