We like to push the outer envelope of music technology geekdom. But what if you’re also an obsessed web geek? Then you start playing the data encoded in a Website design like nytimes.com as a musical instrument.

A new patching environment called Lily, inspired by tools like Max/MSP, works its magic using JavaScript inside a browser. So turning your browser into a music tool becomes more practical. And Lily supports the network-savvy OpenSoundControl (motto: “it’s not MIDI!”), so you can hook up an OSC controller like the Monome and jam with Firefox and the New York Times.

Finally found a use for the NY Times from Bill Orcutt on Vimeo.

How does it work? Get prepared for some Web technospeak, kids!

When the patch starts, the browser enters a DOM inspection mode and mousing over a DOM element highlights the node. Clicking on a node writes the element’s data (its innerHTML value if it’s a text element or the binary data if it’s an image) as a sound file and the file is then loaded in a quicktime player in the patch. The sounds can then be triggered using OSC messages.

Hey, where’d everyone go?

If DOM models don’t exactly get your pulse racing, here’s a strange and elegant physics-based sequencer hooked up to Reaktor. Fans of Processing, that environment is also capable of similar stuff; this is even modeled on the Processing-compatible traer physics library.

SVG Midi Sequencer from Bill Orcutt on Vimeo.

Ready to get going with this yourself? Lily is now in public beta, ready to run for free on Mac, Windows, or Linux. Browser not included.

Lily Public Beta 1

Gobs more examples and documentation on the blog

Previously: Browser Beatboxes and the Rebirth of Max-Like Patching

  • By the way, American readers, how'd that ridiculously annoying ad go for the NYT? "She goes for Arts & Leisure. I grab the business section …" (?)

    "…then she fires up the NY Times' DOM model and starts making sweet IDM using her custom Monome. I turn the Autos classified ads into a 500-piece granular orchestra."

    Yeah, that's more like it. Sundays just wouldn't be the same…

  • That processing-based sequencer example is pretty compelling.

  • I should add — the sequencer example isn't really Processing in any way; I'm just saying you could code it in Processing if you so desired. The physics themselves are modeled on traer.physics, which is also available for Processing. Just want to make sure I'm not being misleading.

  • Really nice and creative thinking. Thanks Bill!

    We need more of you.

  • Kitsunexus

    The NY Times example sounds like a typewriter falling down stairs.

    Sorry I can't join the "pretentious club", but I like actual music, not ciruit-bent noise-esque crap. ­¬_¬

  • Damian Sol

    I got the DOM stuff!! And it is awesome! I am a musician and Web developer, so that makes perfect sense to me. I love the idea of loading the binary data representing images as a sound file. Brilliant.

  • boboobob