Even before we get to the “open source hardware” angle, it’s special just finding something that can function as a self-contained, digital visual instrument. And Milkymist more than qualifies, as a video synth that can produce seemingly-endless generative outputs from raw image and live video inputs.
But it is significant that this is open source hardware. Milkymist in its present form isn’t for everyone, but it’s evolving fast – and it’s a visual coder’s and hacker’s dream. Entirely custom hardware runs elegant scripts for effects like video feedback with razor-thin low latency and blazing framerates and performance. Add the open source spirit to that, and apart from the entire design and software being open to your inspection and modification, you get a box that wants you to add inputs, unusual controllers and sensors, and generally muck about with what you can do.
I’ve been watching this project for some time, but had to fully wrap my head around it. One of the developers, Sébastien Bourdeauducq, had been nagging me for some time. “Yes, yes, it’s open source, it does … things,” I responded. “Great.” But to tell the truth, I hadn’t fully grasped what would make you go down this path rather than a computer. Finally, I gathered Sébastien and some friends in a flat in Berlin-Friedrichshain to have a look.
It’s when you see the speed and low-level hackability of this thing that you get it. Jack in a keyboard, and just a couple of lines of code in the Milkymist’s scripting language, and you can suddenly produce an eye-popping feedback effect or layer. Being this close to the hardware is just terrific: you switch on the device, write very lean scripts, and it just works. And while you could do the same with a computer, there’s no mucking about with drivers and trying to contend with high input-to-output video latencies. Nor do you have to set up a whole environment before you begin scripting – it’s there right when you turn on the device (see screenshots of the interface.) There’s even a camera included in the box so you can get to work right away.
A look at some of the effects that Milkymist can produce – this using included presets:
Diversity of Programmable Effects
Granted, this kind of number-crunching with pixels may not appeal to you non-coders. So the main test of Milkymist in 2012, I think, is whether a combination of flexible presets, controller mappings, and documentation can turn this into something other people can just pick up and play. I was impressed enough that I’ll be loaning a unit to try my hand at that, alongside more conventional computer-based projects.
And yes, of course, a computer GPU has greater capabilities. But at this price, and with this integration, I think it’s best to think of Milkymist as a hacker-friendly, high-performance, standalone instrument. Hopefully soon we’ll also see mixer that make it easier to integrate a computer and something like the Milkymist. (There are options for doing that now, but not the digital mixer of which we’re all dreaming.)
Inside the Hardware
See the unopening and hardware tour above. Just a few of the specs that make this worth a look:
- Connect via Ethernet or (if you connect a router) WiFi
- DMX512 support and hardware connectors
- Onboard MIDI in and out
- Analog video input via composite, S-Video, and component
- VGA output
- Audio line input and integrated mic (for your audio-reactive visuals – and they’ve got some presets ready
- Remote control
- USB ports for MIDI devices, keyboards, mice, joysticks, etc., etc.
- Less than half a kilogram, 172x145x45 mm, and runs on a scant 5 watts.
Complete specs: http://milkymist.org/wp/for-developers/
This is all built on an FPGA, or “Field-programmable gate array.” For those of you are aren’t integrated circuit geeks, what that means is that you have a hardware platform where you can actually customize the function of the hardware after you’ve got it, not simply by changing firmware on a chip, but actually modifying the logic of the circuitry. (The Borg would have loved this.) It’s some hardcore stuff, but it’s about time a creative open source hardware project took advantage of this kind of platform, as it seems uniquely suited to the open source hardware concept. And what this means for you as an end user is that the Milkymist’s capabilities – thanks to all this custom hardware and software – are constantly improving. It’s hardware that gets better rather than worse; it’s the opposite of planned obsolescence.
Deep configurability – getting right at pixels and frame buffers with scripting, from the moment you turn it on, and being able to re-configure the hardware via the FPGA – is really the soul of the tool. You can re-program the very way in which signal is processed inside the box. And, in this case, the primary advantage of the FPGA is democratization of custom hardware, enabling a small band of coders to brew their own device, as with the Milkymist.
You can read more on the theory behind this design, and its high-performance approach to visual processing, in Sébastien’s thesis paper:
http://milkymist.org/thesis/thesis.pdf
In Action
Milkymist hasn’t been seen out in the wild much, though hopefully that will change shortly. Here’s an example with our friend, the tallented visualist NO CARRIER, playing in Philadelphia at 8static in November, with Cheap Dinosaurs. (Skip ahead in the second vid – it gets good – or check out the picture gallery on Flickr.)
And lastly, if you’ve got some time, here’s the complete video of an extended presentation at Berlin’s SCOPE Sessions, which is becoming a valuable hub for the visual scene:
We’ll be following this story more, so if you’ve got more questions (including “wait, really, what is this for anyway?”), we’ll entertain them. Stay tuned for some hands-on details shortly.