Can hacking the innards of a file give you insight into how the underlying data works – and how to unlock the aesthetic of a digital file? Here’s one way to look at that question.
Rosa Menkman, presenting at the Patterns and Pleasure Festival run by Amsterdam’s STEIM, gave a workshop today on glitch aesthetics. Based on her ongoing Vernacular of File Formats work, in particular the document above, she starts with the basics of poking and prodding bytes of data inside a file, then moves toward making images and motion out of those effects. We’ve discussed that direction in general – and “glitch” as aesthetic and technique. But I had the pleasure today of seeing Rosa work with a class full of workshop attendees on the topic.
As Rosa puts it: “Just put a little error in any file, and you see a data structure come to the surface.”
Simple File Exploration
Rosa had some nice examples in mucking about in the raw data of file formats. Simply put, though it’s worth getting the full workshop from her, this involves things like:
1. Saving an image as RAW data, then reopening the file while adjusting the interleaving, size, and offset.
2. Opening 8-bit, 16-bit, and 24-bit BMP files after cutting and pasting blocks of text or making deletions in the raw data. (Here, you can save a BMP and then open it directly in TextEdit on the Mac, for instance. I need to try other text editors – gedit, for one, recognizes that you’re trying to open a binary and stops you.)
3. Replacing index colors in an 8-bit BMP.
4. Changing index in a JPEG.
5. Working with GIF compression export and color indexing, even directly from the Export for Web (Photoshop/GIMP) dialog, and cutting and pasting and deleting as with BMP above (#2).
And then you get into PNG and you get fantastic blocky things, and PSD, and so on and so forth.
What people may not fully appreciate is the range of effects so-called “glitch” can produce – far beyond the best-known effects to a broad gamut of visual artefacts.
These are explained more fully in the above PDF. And of course for those of you messing with code, now you have more to play with — as I will certainly do next time I’m wrangling with image buffers in Java.
Rosa showed her examples in Adobe Photoshop CS, so I was interested to see if there were equivalents in the open source GIMP. To my pleasant surprise, there’s a nice, interactive dialog for opening RAW files:
You’ll find some additional options if you install the Export for Web plug-in, though here I’ve only seen an easy way to install on Windows (as binary) and Linux (via a plug-in package), not Mac.
BMP file glitch works for me, as well, and GIMP opens files even when other applications would (correctly) flag an improper end of file. Results as here:
Video Oddities
Rosa also showed some unusual video hacks, like exporting, via QuickTime Pro 7, Cinepak with 256 grays, which on playback creates some really bizarre effects thanks to inability to set a color map to those grays in most players (like VLC).
Poking around for options other than a deprecated QuickTime, I was pleased to see ffmpeg is now available on the Mac via Homebrew, which I can only describe as a faux package manager that actually is sane. (Sorry, MacPorts.) That should open up other video export/reimport hacks.
As with the text editing above, in video, Rosa works with a hex editor to manipulate video files.
Some example videos, including at top rough footage of a live set at the terrific Trouw club in Amsterdam (yes, outside the tourist district, Amsterdam has some wonderful places):
Interactive Live Glitch
Rosa is unabashedly committed to sharing what she discovers freely. One advantage of that has been that it’s inspired software.
Our friend and sometimes-contributor vade, aka Anton Marini, has produced a series of Quartz Composer plug-in effects partly inspired by this technique. The upside: you get effects you can drop into popular Mac live VJ apps, and instead of producing a handsome still or two, you can create motion effects live and in real-time in a performance. That suite of glitch plug-ins got a nice set of updates just a few months ago, so well worth mentioning again.
Another fruit of this discussion is Johann Larsby’s Monglot app, also for Mac, produced in collaboration with Rosa. Its interface is a bit baffling, but the basic notion is to create a GUI consolidated the various effects Rosa describes. Once you work those out, you can spit out tons of images. You can even take sequences of images and assemble them into video, returning to the realm of motion, as in the video here:
Both apps are open source, but it’s clearly time to see this stuff extended to cross-platform contexts. I think having a few of us try it could be interesting – more happy accidents.
This isn’t just technical novelty. Rosa’s Glitch Festival is coming to Chicago and Amsterdam, and will assemble artists working with these media in performance as well as technical symposium.
The official site:
http://gli.tc/H/ (Abandon hope of legible websites, all ye who click here. You know, in a good, glitchy way.)
Thanks, Rosa, for a great workshop, and STEIM for hosting.
Also, in case you missed it before, this is a must-read article by Kyle McDonald for us at CDM in regards to Anton’s work and some of the larger issues of glitch and shared community – also talking about the collaboration with Rosa:
Interview: Anton Marini on Visual Expression, Exploring Glitch, Open Community