What could audiovisual art made for tablets look like? Some dismiss the results as “toys.” The creators of Thicket embrace that very role. Thicket, the product of electronic A/V artists Morgan Packard and Joshue Ott, is intended as a kind of immersive distraction, filling the screen of an Apple handheld or iPad with clouds of abstract color and ambient sound. Free of controls or widgets, playing with Thicket is a bit like sticking your hand into a mist, more gently responsive than overtly interactive.
It’s also lovely, and – for the moment – free. As an experiment, Josh and Morgan are offering up Thicket free of charge on the store, meaning those of you with appropriate devices can try it out as you read.
http://apps.intervalstudios.com/thicket/
A lot of what you read about iOS development and the new “app” markets in general tends to blur into generalities. You’d also be forgiven for assuming the “app” is a gold rush even for avant-garde digital artists. Instead, the picture from Morgan and Josh is subtler. They concede their work may have niche appeal, and view the iPad and iPhone not just as a hot, new market, but as part of a longer narrative of audiovisual expression.
Along the way, they’ve got some tips for learning to develop software, reflections on how interaction can work and why even a baby can play with their creation, and some insight into how their project worked as musical, visual collaboration and not just software creation.
If you had to classify Thicket, how would you describe it?
Morgan: Thicket is a toy. We can get a little more descriptive and call it an audiovisual toy.
Josh: I’d call it portable moving artwork…
What was the original impetus for the idea behind Thicket? What sparked this particular, somewhat abstract notion of how you might interact with sound and image?
Josh: I had been talking about making something for iOS for over a year, ever since Apple released the first SDK, but had been unable to find the time. The impending release of the iPad and Morgan’s interest provided me with the motivation I needed to actually start something. We decided to choose from favorite moments of our past collaborative efforts and find something we could make pretty quickly. Thicket came from one such moment in the “Unsimulatable” DVD/performance that accompanied Morgan’s first album, Airships Fill the Sky. We had referred to this moment (fondly) as the “ball of string”. That particular moment came from multiple experiments and jam sessions with my superDraw program and Morgan’s Ripple program. That was the seed, and once we had that seed, using the unique features of the iOS devices to alter/enhance/affect what was happening seemed fairly natural.
Morgan: My approach to sound for Thicket comes out of a desire to piggyback on the natural ways people want to touch Josh’s graphics. I didn’t want to have any sort of fiddly controls — knobs, sliders, buttons, menus. I simply wanted to sonically respond to the natural, casual interactions which Josh’s visuals encourage. Typically, music apps for mobile devices have a number of controls — buttons, sliders, knobs. And they use simple XY position as a primary source of musical or synthesis parameters. Neither of those input sources — control widgets, or xy position — is compatible with the natural way one wants to interact with Josh’s visuals, so I had to use some different sources of music input — finger speed and multipe touches. I’m really interested in finding other gestural means of input which play nicely with the control of visuals.
While it began on handhelds, Thicket really seems to benefit from being on the tablet form factor on the iPad. What does that device mean to you as a canvas for work?
Josh: It’s a pretty exciting device to work with. As we’ve improved it over the year, we’ve found that we are more and more focused on the experience on the tablet rather than on the phone.
Morgan: I don’t want to discourage anyone from using Thicket on a phone. We’ve made sure the phone experience is solid since the first version. But I’ll admit that Thicket really shines on the iPad. I was a reluctant iPad buyer. I only bought one because Josh forced me to! But now that I have one, I’ve developed quite a passion for it. It’s a seductively humanizing and ergonomic little machine. It’s very book-like, which is a good sign. There’s a reason books are the size and shape they are. Their form has been under development for hundreds of years. The iPad allows us to reach people when they’re in that curl-up-with-a-book mode: relaxed, comfortable, attentive.
Back when multimedia CD-ROMs were what some artists saw as the future (which wound up being both wrong and right, depending on how you look at it), I remember Morton Subotnik talking about his vision for this stuff. He described these devices as a kind of “chamber” experience – the personal quality of them being a strength. What does it mean to the two of you to be able to distribute work to someone on a mobile computer, and not only in a performance or gallery?
Morgan: The mobile art experience has been around as long as radio, so I’m not sure I consider Thicket revolutionary in that sense. I’ve been making music recordings for years which most people experience in solo settings. However, the community aspect of art-making is very important to me. I love performing, and I love what happens when you get a crowd of people together. If we want to test out the “chamber experience” analogy, we can compare the experience of using Thicket to the experience of listening to live (western classical) chamber music. When you listen to the chamber music, you’re out in some sort of venue, you’re setting next to people, probably strangers, you’re in a public place, physically participating in something collective and cultural. Using an app on your own, no matter how artistic that app is, doesn’t have any of that meatspace cultural thing built in to it. The Smule folks have done a great job of allowing you to interact over the Internet with other people using their apps. But I still place a lot of value on getting people physically together in the same space, experiencing the same sounds, the same sights, the same vibe, and actually being interact with one another in the physical world. I’d love to find ways to create software which encourages more connection between people. But on the other hand, sometimes it’s good to recognize what things software is good for, and when it’s time to be a little more old-fashioned.
Josh: I think the most exciting aspect of these devices for me (besides the general human augmentation that all of them increasingly offer us) is their potential to augment the way we communicate in an artistic way. I’ve been exploring this in my performance based work for a while now (performance that involves the audience in new ways, like the multi-user art show earlier this year), but I’m really excited to approach it even more organically, more like little ad-hoc multi-player games that happen to create something aesthetically compelling.
We’ve also seen Thicket adapted to a gallery installation (at TEDxBrooklyn) and theoretically it could be used in live performance. Is this something you’ve been able to perform with, as well? Do you think it’s possible to build a piece that could be both your performance tool and something an end user can pick up?
Josh: I’ve really enjoyed experimenting with how Thicket can be framed (sometimes literally) as a piece of art on a wall. The iPad itself is just big enough to be framed and hung, and people seem to really enjoy interacting with it this way. Another fun use I’ve discovered for it was projecting it on the ceiling for my 4 year old daughter as a moving night light. Thicket was never really designed to be a performance tool, although it’s arguably started to move in that direction. Building a performance tool that is still fun and accessible for everyone, while giving more focused users the flexibility they need, is one of my goals for the next year. I’m not sure whether that will be Thicket or something new, but will definitely be approaching it as something I will use myself for my own performances.
Morgan: I’m wary of allowing Thicket to become too much of a tool. I’d rather make it a better toy. Performers need tools, not toys. For now, I’ll be sticking with Ripple, my performance/composition software for my own music. And trying to make Thicket an easier, richer, and more delightful world to explore.
Game designers have long thought about challenge, reward, and even failure states. And as a result, some of their creations are truly long-form — it’s not uncommon for someone to complain about a 20- or 30-hour game being too short. One of the criticisms of art on devices like the iPhone has been that thing tend to become quickly-digestible toys – perhaps calibrated to the kind of interaction design artists have done traditionally. How do you respond to that criticism? How did you tackle questions of states of interaction in Thicket?
Josh: Being a fairly avid gamer, and having just read a profile in the New Yorker about Shigeru Miyamoto (Nintendo’s Mario creator) I’m really excited about the intersection between art and games. I know this topic is a bit tired, but I’m approaching it from the art side rather than the “gamer” side, and that’s interesting to me. While Thicket isn’t a 20-30 hour experience (though it could be argued that its open ended-ness makes the experience ultimately unquantifiable), making Thicket a deeper experience has been one of our major goals this year. (I think we’re definitely on the right track with our 2.0 update that hit in November.) Games that allow players to be creative in how they play, from open world games like Rockstar’s Grand Theft Auto series, to games where traditional goals can be accomplished in a variety of ways (Bioshock, etc.), to games where creativity is part of the game itself (World of Goo, Noby Noby Boy, Captain Forever), have offered players deeper experiences and replayability. I’m really excited to see these two worlds collide, and possibly try to experiment with that collision in my own work.
Morgan: I am decidedly _not_ an avid gamer, though Josh diligently continues to try and convert me. But I’m intrigued by the idea of guiding a user experience, creating some sort of progression a user (or player) travels through. On the other hand, I’m not sure that’s necessary. There’s room in the world for idle playthings with no specified goal. Take those little magnetic Buckyballs, for example. They’re simply fun and satisfying to play with. They’re wildly popular despite the fact that they’re completely open-ended, with no challenge/reward/failure built in to them. A difference between that toy and Thicket is the range of what you’re able to create with them. We’ve been getting regular suggestions from users asking for the ability to add their own sounds and pictures. I have no idea how we would do this. But it may be that what people are really asking for is greater control, a greater range of end results that they’re able to produce. I’d like to come up with ways to do that without losing the casual, playful spirit that Thicket has now — and without adding any sliders!
Working on Thicket meant collaborating not only aesthetically, but technically. How did the two of you work together, especially as you were in different cities?
Josh: There was (and is) a weekly video chat where we discuss our status, what we’re doing… and we’re of course using an SVN [Subversion version control system] on the technical side to share code. Surprisingly, collaborating technically across a long distance has been quite painless.
Morgan: It’s pretty easy for sound and visuals code not to step on each others’ toes. We have a shared set of data we can examine for things like touch position and speed, and mode changes. Other than that, we’re pretty much in our own sandboxes code-wise. We did do a bit of coordination to syncronize mode changes to tempo. As Josh mentioned, a shared code repository — the Subversion software in our case — is absolutely indispensable.
I know a bit about your coding background – Josh having worked in Processing/Java and Flash, Morgan in SuperCollider. What was the process of learning iOS development like? How was the learning curve? Any lessons learned you can share?
Josh: It definitely has been harder than I thought it would be. Coming from [Flash/Flex] ActionScript and Java, I feel like I had to learn three new languages at once: Objective-C, Apple’s Cocoa Touch framework, and OpenGL. Learning about pointers and handling memory was a big challenge. I would highly recommend the (FREE) Stanford iPhone Application Programming course available on iTunes: It was a huge help for getting started and I still am kind of amazed that you can audit a very well put-together class like this for free.
Morgan: I’ve been comparing the move from higher-level languages to the lower-level iOS environment to stepping out of a go-cart and in to a helicopter. Not only was I learning new languages (C, Objective-C, C++), I was dealing with raw audio data for the first time. Honestly, I’m still in the steep part of the learning curve, though things are beginning to feel a little easier. It’s pretty difficult to simply figure out how to send raw audio data to the speaker, which is necessary if you want to do any dsp. This example project helped me quite a bit with that:
http://sites.google.com/site/iphonecoreaudiodevelopment/remoteio-playback
Morgan, you ultimately decided to code sound from scratch. This was your first project to do that, correct? What led you to that solution, and how did you approach it?
Morgan: Without rolling a significant amount of my own code, I would have been limited to very basic techniques — file playback and looping, pretty much. Most of the audio in Thicket is based just on manipulating loop lengths and playback start points. But even though the techniques I use are quite simple, I wasn’t able to find a higher-level sound library which could do exactly what I needed. Also, I didn’t want to get locked in to something which wouldn’t be flexible enough for me in the future. Unless there’s a really powerful tool available which I’m confident I can grow in to for a very long time, I’m more working at low level and building more things from scratch.
Josh has talked with me a bit about how he feels the iTunes App Store and the Apple ecosystem on iOS, for all the criticism it has garnered, has really made Thicket possible. What was it that drew you to iOS? What is it that Apple is able to provide for you, in terms of hardware, software, development experience, or user experience?
Josh: Apple really does have excellent documentation and support for their SDK. That’s one of a number of reasons why we chose to develop for iOS. Another reason is actually the small number of devices that iOS runs on: from a development perspective, it makes things much easier to test… Then there’s the solidity of the Apple’s hardware and multitouch implementation in general.
Morgan: I like the way the Apple devices and OSs look and feel. It’s a dumb emotional judgement, but sometimes you’ve got to follow that. I also like the fact that people using Apple devices are really into their apps. I wanted to be part of that party.
Do you think there’s potential on new platforms – ranging from Web app stores to Android or Windows and Linux tablets – to do the same thing? Or is there something Apple was able to do for you that was unique? Is there anything those platforms can learn from Apple?
Morgan: We chose Apple in large part simply because that platform seemed to be where the action was. I think we probably could have built Thicket for another tablet platform and the product would have been fine. But I also think fewer people would have downloaded, and we wouldn’t have felt encouraged in the way that we were. We might not have made the version 2 update, which has been much more successful than version 1. At this point, for us to transfer time and energy away from iOS development toward another platform, we’d need to see someone else having significant success selling artsy apps. It’s too big an investment to try another platform simply as an experiment. When we see another app store really taking off, on a device which we can feel some love for, we may reconsider.
How has this business of selling an “app” worked out for you? And why try going free?
Josh: I wish I could say Thicket’s paying the rent, but it’s really not. I think the (emerging) art market on the App store is relatively new, and harder to find (as Apple doesn’t give us a real category in the App Store). The fact that they have a Generative Art & Sound category in their 2010 “best of” list [iTunes link] (which Thicket happens to be in!) suggests this may change in the future. We never really expected it to do well financially and are pretty thrilled when people seem to like it. Making it free is an experiment of sorts. It allows a lot of people who wouldn’t ordinarily try Thicket to do so.
Morgan: I had very low expectations for how many people would be interested in Thicket, and they’ve been clearly exceeded. I’m excited that (a modest number of) people are actually spending money on our art! Sales are much better for Thicket than they’ve ever been for any recorded music I’ve released. Originally we thought Thicket would be a sort of experimental, niche product, and we’d price it a bit high. It seems to have a bit wider appeal than we expected though, and a lower price allows us to reach more people, while still making some money. Going free for the holidays is a way of testing just how wide the appeal is.
Have you seen other interactive creations – on iOS or elsewhere – that you’ve found inspiring?
Josh: I’ve mentioned a couple of games that I play; here’s more stuff.
For PC: Windosill (really everything by Patrick Smith, AKA Vectorpark, is just amazingly beautiful.)
Superbrothers have also inspired me quite a bit…
Machinarium is also really good.
Interactive artwork I’ve been inspired by this year on iOS: Uzu, Sonic Wire Sculptor.
Games (also iOS): The Incident, Acrobots, Enviro-Bear (also on Android)
Morgan: I think BubbleHarp is great. I love the butterfly effect aspect of it — tiny differences in the way you set it up result in huge differences in what you see. Bloom is classy and beautiful, and was a great example of how to create a sequencer without forcing a bunch of fiddly controls on a user. Using Uzu makes me feel like a god.
What’s next for your collaboration (or personal projects)?
Josh: Both of us want to continue to make Thicket better. It’s becoming an outlet for artistic experimentation for ourselves, and we are seriously considering how to involve others as well. Additionally I have lots of ideas and I’m hoping to also release a bunch more software next year: some artistic, some performance based, some more game-like.
Morgan: The next thing we want to try with Thicket is in-app purchase of new modes. It will be really satisfying to have this platform we can be continually adding to. I’m a bit anxious to start devoting a bit more energy toward live performance. I don’t want to get so stuck inside the iPad that I forget that I was once a performing, travelling musician.
Thanks to Morgan and Josh for the insight. See also a superb interview, with code, at the always-excellent Disquiet, which focuses on the sonic and compositional aspects of the app…
BEING DECIMAL: THE ANTICIPATORY PLEASURES OF THE THICKET APP [Disquiet / Marc Weidenbaum]
For more interactive iOS goodness, also free, try Josh’s Snowdrift – not that certain people associated with the East Coast of the United States, or London, among other places, really need an app for that at the moment.
Snowdrift @ iTunes