Here’s a brief video snippet I discovered someone took at a talk I did at this year’s South by Southwest, with interaction design pioneer Joy Mountford (formerly Yahoo, Apple). We were talking about the idea of “data as art”, which happened to coincide neatly with the Design and the Elastic Mind show at MOMA, featuring several works from Joy’s recently-disbanded Design Innovation Group team at Yahoo.

The audience response to the work Joy showed was really overwhelming, as search activity danced around the globe and photos came to life in three dimensions. And it was nice to be able to show them the tool used to create these projects, Processing, and encourage people to try it out for free, even if they hadn’t tried programming before.

But I was surprised by how people reacted to a quick musical demo I closed with. Using Java, I wrote a simple program that checked my Gmail account using IMAP, then translated the time spam messages arrived into MIDI notes. I’m still developing a more advanced real-time version, so I threw the resulting SMF file into Ableton Live.

I’ll actualy be showing a newer version of this for Internet Week at an event sponsored by Make Magazine; more on that in a few days. (I’ll also use that as an opportunity to post some updated code.)

We spend so much time talking about how visualization can make data more expressive that we sometimes overlook other media. The spam “musicification” made sense to people partly because even the untrained ear is sensitive to musical timing, I think. Sonification of data isn’t always the right choice; the results can be abstract, though perhaps there’s value in that, too. But it’s worth remembering that people are sensitive to sound as they are to visuals. Since it’s not an either/or choice, necessarily, it’s too bad that so often designers neglect aspects of sound and timing while focusing only on what something looks like. It’s a challenge, certainly — there’s a reason most of us mute annoying sound feedback on computer interfaces — but I think it’s an area in which we’ll see a lot more discussion.

Now, data in smell-o-vision — that’s a story for another day.

  • 7oi

    damn, i wanted to translate spam into music! i'm a big spam fan!

  • 7oi: we could form a band. 🙂

    Or, perhaps, just a multi-course meal of spam.

  • Dub

    7oi – Send me your e-mail address, you're welcome to some of mine!

    Peter, it'd be all about /1a9ra and Roxel watches – it'd be more like a "Brand" or a "Bland" than a band

  • gbsr

    now this would actually be a good reason to get rid of my spamfilter.

  • Super cool! Sounds like some neat music is coming out of there. Very John Cage with those long pauses.

  • The most interesting part is how you map the spam (or any other data) to the MIDI notes. Could you elaborate on the rules that govern the pitch and duration of the notes? By varying those, you can create completely different music from the same data. The mechanism itself is almost trivial in comparison. You mentioned something about the receive times of the messages in the video, but that is not the whole story, is it?

  • It's not the whole story, no. 🙂

    The easy part is mapping the timestamps, so that the duration of the piece is mapped to spam arrival times. Beyond that, I think this gets more into the area of the art of it than anything meaningful, but I did toy with look at the length of the messages themselves. Here, because spams did tend to overlap, I mapped pitch as a simple algorithm that would randomly generate some variety while avoiding two notes hitting at the same time and flanging. But I could do something fancier in the future.

    So you're absolutely right, though I haven't done the *most* interesting thing with this just yet!

  • Peter, I want to see more on this in the future. Very amusing 🙂

  • Pingback: » Visualizing Spam()

  • Pingback: AtomoSynth Abyssal, free Mellotron samples, Den-kuri Master(den-kuri ver2), and Musicifying Data? Spam Rendered in MIDI()

  • If you used the FBI keylogger machine "Carnivore" like another processing project, you could get something more dynamic. I for one would like to hear a literal translation of spam with vocal samples of porn, watches, viagra, lonely hart's, singing out to me in a beautiful spam opera.

  • Samuel Van Ransbeeck

    Last semester I wrote a software program that translates stock exchange values into music in realtime and improvises on it. For who is interested in it, here you can get it
    (the second one has examples in mp3 instead of aif files)

  • Pingback: Create Digital Music » Depressing Project of the Day: Stock Market, Set Music with Microsoft Songsmith()

  • If you could personalise this and create your own spam soundtrack every time you received it that would be funny. Voices would be good too.

  • eBrNrR comment2 ,

  • Hi, I found the same brief video clip on how you took spam and translated via java. Just came across this post and wanted to take the opportunity to say thanks, and share how it inspired me. I took the digital data from an image: a pieces of art: monoprint, and "musicfied" it. If you interested:….

  • Incredible. is the shit.

  •, how do you do it?

  • Damn, I wish I could think of somteihng smart like that!

  • BxFBdq gmcomgwhampg

  • 9udxAb kjbbketnocfx