Splice, a site for remixing songs, can now do something no Web browser app has successfully done before: pass for a dedicated audio app.
It’s no [Pro Tools, Ableton Live, SONAR, insert app here], but Splice’s online editor is a basic, functional audio sequencer with real-time arrangement, instruments, sync, and audio effects, all built in Adobe Flash CS3 / ActionScript 3. Sure, Flash has been able to do basic audio playback and mixing for some time. But Splice actually does things that dedicated audio applications normally do exclusively, such as sophisticated audio effects. There’s still quite a lot it doesn’t do, and since many of those things (live multitrack audio recording, hardware connections, and so on) aren’t currently possible in the browser, Web apps are unlikely to usurp dedicated music creation software any time in the forseeable future. But maybe that’s not the point: musicians can keep using the tools they love for music creation, then throw up a track and let a friend mash-up a new beat, or let fans create remixes. In fact, it’s not hard to imagine dedicated apps having similar online counterparts, or hooking seamlessly into such services.
And it’s what Splice does on the Web that makes it interesting. Not only are the tracks actively remixable, even by a casual listener new to the software who just wants to mess around, but sound sources are shared, too. You can pull up open freely-licensed samples from the Freesound Project, a collaborative database of sounds recorded around the world.
Little wonder then, that the lead developer of the new Splice has already made a name founding the sample project and sharing unusual software plug-ins (the traditional, offline type) for free. Bram de Jong, CTO and lead developer of the new Splice, is known to plug-in devotees for his involvement in Smartelectronix, a collective of developers releasing powerful and sometimes downright bizarre plugs for free. I got to talk to Bram about his new day job. He shares what Splice is about, what people are using it for, and how the heck they got Flash doing real audio.
CDM: What’s new in the relaunched Splice?
Bram: I think it would be easier to answer what isn’t new. The Splice team redid the entire website. The sequencer is now in Flash 9, and has been redesigned from the ground up. The whole website backend has been redone, and a lot of “standard Web 2.0” features have been added. All in all, I think we’re quite confident the whole website is easier to use, and we love its new fresh look!
CDM: How were you able to get real-time synths and audio effects (DSP) working in a Flash application? What was required, technically?
Bram: Andre Michelle and I joined Splice late in 2006. Andre had worked out a way to access the audio buffer in Flash. This meant we could actually write samples directly to Flash’s audio output! Starting with this “audio hack”, we built a complete audio host system on top of this. Think effects, channels, synthesizers. Think something like VST or Audio Unit , but all written from [scratch], in Flash.
I think the hardest part was to start from zero: Flash doesn’t have any audio capabilities except for some simple sound playback with volume and pan changes. So, we had to reinvent the wheel. Soon, our project turned out to be a huge monster with lots of tentacles we didn’t expect to pop up!
Flash 9 is a lot faster than Flash 8, but it was still very difficult to get things running in real time. A lot of time was spent forgetting everything we knew from C++ and doing benchmarks and testing various optimizations in ActionScript.
Taking hints from your bog standard audio host, we tried to stay as close as possible to having the user interface completely separate from the audio engine, but (warning: this is for the even more technical readers): Flash has only one thread!
CDM: How have people been using Splice so far?
Bram: Our audience is pretty diverse. We have a lot of youngsters who want to try their hands at making some cool tracks and share them with the world. For them, the site is perfect: they can create something from nothing, no need for expensive audio software. There are a lot of people that come listen to music on the site, or come search for nicely-licensed music to put on their MySpace, blogs, and even videos. And, then there’s the growing layer of people who are really deeply into Splice, making amazing tunes we didn’t even imagine possible with the older version of Splice tech, so we’re really excited to see how their music grows.
CDM: How are people using the new version, or how might you envision them doing so in future?
Bram: Well, there’s a bit of a difference between both versions: the first version had worse sync, no effects, and no synths. This one has (much) slower loading times for large tracks but perfect sync, as well as effects, various types of automation for all parameters, synthesizers, a note editor, etc.! And, of course, in the future we will be adding even more synths and effects.
So, what we think is that although the sequencer is a bit harder to use for novice users, the possibilities have exploded. People who are used to making music in actual standalone music soft look at the sequencer and say “whoa, that’s not bad at all!”, which is pretty amazing for the first application ever to try this.
It has a lot more creative potential than the previous version, so we hope people will do some amazing things with it.
CDM: Do you envision any kind of open API, or ways in which Splice might be integrated into other sites?
Bram: We do. Perhaps not for the very near future, but somewhere in the mid-term future we want to make this possible. Of course you can already embed your creations into your favorite website, but what we would like to offer is a music creation tool for all to use.
Imagine you’re a musician, you release some tracks and you’re able to say: hey guys, you can use these in your music to remix us, but even better, you can do it RIGHT HERE.
We have also been thinking about a linkup between ccMixter and Freesound and Splice, based on ccMixter’s “samplepool” API. Imagine the power of Splice, combined with a huge sound library like Freesound. Ed.: ccMixter and Freesound are essentially open source repositories for remix-able music and sound samples, respectively, licensed via Creative Commons. -PK
CDM: What was the development process like for Splice? It’s very rare to see DSP and audio developers working on the Web; how did that collaboration work? What sorts of challenges did you have to >tackle?
Bram: Surprisingly enough, the team coped pretty well. We have a very heterogeneous team with very different skills. We have a Flash guru who knows a bit about audio, we have Java programmers used to working with large systems, a great designer, a C++ audio monkey who has some experience with website building Ed.: that’s Bram, folks and more. Most of the challenges concentrated in two parts: the first was the sequencer (what you see was built by one and a half person, over a period of 5 months), and the second was trying to stay compatible and converting all the things from the old website while rewriting everything from zero.
At some point it became apparent that building the new Splice was much like trying to solve a very large, entirely white puzzle, but we managed. 🙂
CDM: Traditionally, the Web and pro audio work have existed independently. Do you expect that they’ll continue to do so, or, as other, more conservative apps (calendar, email, word processing, spreadsheets, and more recently image processing), will Web apps sometimes replace local apps? Or is it more likely that rich clients with online hooks will exist alongside dedicated local apps and Web apps for other tasks (such as doing production in Pro Tools, Logic, Ableton Live, then remixing in Splice)?
Bram: If Flash (or [Microsoft Web platform] Silverlight) becomes a little bit more powerful, we will be seeing more and more heavyweight multimedia web applications. Things like a calendar/email, etc., are all really interfaces for editing text, so they require very few resources. Next is image (now: pretty advanced), then audio (now: getting along nicely), and after that realtime video (now: babysteps). This follows very much the evolution of standalone applications in the 90’s.
In the future we will probably also see more things like what Adobe is trying to do now with Apollo/AIR: they try to bring the web to the desktop even more by having Flash work in the desktop environment like regular applications. Will it work? Who knows, but it will surely be very interesting following the evolution.
CDM: Aside from Splice, are there other sites you view as compelling web applications? Competitors?
Bram: When we launched the first version of Splice, there were no other sites even really attempting what we were doing – incorporating a Creative Commons library into a social network with a built-in sequencer. In the last year, some other sites have started to appear slowly, but none of them offer this kind of collaboration alongside a tool as advanced as the Splice sequencer.
CDM: Have any celebrity users picked this up?
Bram: Soon we’re going to have some remix competitions going on that will involve some rather well-known people. We have a lot of musician friends who have tried out Splice already, but none of them is amazingly famous. We have very good hopes it’s only a matter of time before more celebs join the Splice force.
CDM: Bram, will we see continued development from you on the Smartelectronix side? A standalone of the new SupaTrigger as seen in Splice? Online hooks for future plugs?
Bram: For about 3 years, Smartelectronix and especially my part in it have been suffering gravely. Paid jobs get in the way of making crazy software for free, but the contacts I made in the most active period of Smartelectronix are contacts for life. People have been begging me for updates, especially for Mac, but not only do I not have a Mac, I have very little time.
Right after I started fading out my Smartelectronix career, I started working on the Freesound Project. Managing that HUGE community gave me a lot of experience, so when Splice called upon me, I thought it was just the perfect combination of both audio and web. Who could ever say that one half an hour you could be working on a new version of SupaTrigger in Flash (do you like it, people?) and the next half an hour working on server-side audio file processing in Linux? 🙂
CDM: Can you talk a little bit about your background as a music person and developer? How did you become involved in this project?
Bram: Well, surely enough people kind of know my development background. I know I get confused a lot with Bram Bos (of Hammerhead fame) and just now I read somewhere that my last name was Cohen (of Bittorrent fame!), but no!!!
I started with musicdsp.org where I collected audio DSP algorithms, then moved on to smartelectronix.com where I created some pretty insane VST plugins and gave them away for free (!), then moved to Spain to work in MTG where I created Freesound. Now I’m at Splice after Wendell Davis (our CEO at that time) asked me to join in!
I used to play flute a long time ago, but I stopped about 7 years back (internet vs. time). There was a time when I tried making electronic music, but luckily for the rest of the world population I stopped. Many years ago in the music-DSP mailing list I heard a nice quote of someone saying: “audio developers are mostly frustrated/failed musicians”. I think this quote is dead on, but it doesn’t really make me sad: when a musician I have never met before tells me that he has used SupaTrigger or SupaPhaser on all his records, I tend to get a very wide and euphoric grin on my face 🙂
Thanks, Bram. For those interested in doing this sort of thing with Flash yourself, don’t miss Andre Michelle’s blog; the code is all available, from various useful code snippets to the projects with which Splice was built. Andre Michelle also has a lovely 909 simulation in Flash. On that note, crank up your volume and listen to Splice’s 404 page (the Internet webpage 404, though you’ll hear it start to autoplay), as noticed by Music thing.
Splice is in fact no longer alone on the Web; we’ll cover some other websites soon. Send in any nominees you’ve been using.