Real-time design and live visuals can provide transformative powers. But what if you’re not ready to commit both time and money right away? Here are two industrial-strength tools that get you started for free. Meet Smode and TouchDesigner.
My name is Ted, and I wrangle content for live deployment to screens and sound systems. I also manage very large installs. Recent jobs include the NFL Experience in Times Square, a graphics package for Bounce Chicago, and a soundsystem install for a themed entertainment attraction built around Trolls. Next up, looks like a magic show and a year of touring a heady art music piece. I am living evidence that you can study theater and not computers, and still make a daily living doing Crazy Work with computers.
So, working with real-time visuals matters in the work I do.
The old way of doing things: go back into the content source file, make edits, render out a new video, maybe encode the video in an extra step, move the video to the server… it’s a lot, and it’s rarely a smooth process. The growing real-time toolkit, though, gives me a new option – I can just make the change in the media server, and then say “okay, how does this look?” It’s an incredibly empowering change, but with one major challenge – cost of entry, both for a strong machine to run on and time and money invested getting into the proper tools.
There are open source options. But for an easier entry into software – while still not spending money just to get going – I’m recommending either TouchDesigner or Smode.
You can expect the following features out of both packages:
- Playback of a variety of media, moving or still or audio
- Generate text, from string composition to font formatting to animation
- Respond to a selection of external protocols, such as MIDI or DMX
- Some amount of “projection mappability” – Smode locks their stronger stuff behind a paid license, while TouchDesigner gives you everything you’d need (or everything to build what you’d need.)
- Respond to audio input, which can be shaped within the server
- A timeline for making cues happen specifically, repeatedly and reliably.
- The feature you’d really love access to is one you’ll have to pay for – and it’s NDI input/output
- A user community that understands working on projects with high stakes
Resolution can be a deciding factor. If you want 1080p output and you don’t want to spend money on a license, just go download and start learning Smode. TouchDesigner’s Non-Commercial license (“no money needed if there’s no money made”) is locked to 1280×1280 output. That can be a benefit – it’s easier on system resources – but for applications that need HD (for output to monitors, cameras, or where aliasing is an issue), you might start with Smode, instead.
Beyond those features these two softwares are very different, which means they probably both have a home somewhere in your workflow. Here’s a quick comparison:
Smode
What is it? Smode is a real-time compositing engine initially built to map projections and LED screens on TV sets. It features an Adobe After Effects-style approach to look building – you put everything on a timeline, and set keyframes to do what you need them to do. There are also hooks for audio reactivity, MIDI input, and OSC input. Commercial use with the free license is allowed, it will output 1080p, and when installed on the right machine, it’s very reliable.
Who is it for? Smode is for teams that are building shows to a timeline – or even better, to timecode (hosted by Smode for other devices to follow.) I am not a regular Smode user, but with a week of playing with it, I’d say it’s a strong candidate for teams that already have a strong studio workflow (maybe feeding Resolume with content for projection mapping) and would like to start exploring what it’s like to do more work on-site. The UI in Smode references a number of classic media environments, from the stacked/nested object list to the dropdown menus in the timeline. If you don’t like looking at new UIs, this is the package for you. The army of After Effects power users out there will feel right at home in Smode. Bonus points go to the dev team, who bless you with a fat stack of totally awesome 2002-style 3d generative content. If you agree that the 90s are back this year, you should grab this software for building looks in 2020.
Who is it not for? I wouldn’t use this for busking – making up a show as I go – unless I was doing something very specific, such as deploying and live-tweaking a particle engine. I also wouldn’t use this without having a solid understanding of key frames (though this is certainly an ideal place to learn!)
What are the features that cost money? Mostly you miss out on advanced projection mapping tools, low-latency inputs and resolutions larger than a single HD (1920×1080). You also need to purchase a license to bet access to DMX and Artnet – if you’re using the server on a site with a lighting console, they think you have enough money to pay for a license.
I’d highly recommend purchasing a license – they’re affordable, and supporting software is vital to see it continue to develop.
What does it cost? Licensing Smode is currently in an interesting place – there is currently only distribution in France, Spain and Greater China. If I wanted to buy Smode I’d probably audit the heck out of the free version, make something cool, and then send it to the company so you can be linked up with the right distributor for your region. In general I think it’s always a good idea to involve yourself with companies that provide you critical tools – if you fall in love with Smode (from outside France, Spain or Greater China) it looks like your path to purchasing will involve developing this relationship as a default.
TouchDesigner
What is it? TouchDesigner is a node-based compositing tool originally forked off the grandparent of all node-based compositing tools, the great Houdini FX by Side FX. TouchDesigner is a combination of a graphics processing engine, a 3D rendering engine and a 2D engine that rapidly draws to the screen space. It’s also a Swiss Army knife for control operations – MIDI, DMX, Artnet, OSC, depth cameras, Windows multi-touch and audio are all within easy reach. It’s been used on some of the largest projects in the world, including the Salesforce Wall (this is a personal favorite, so I’m deciding to link it), upfronts and sporting events, concerts and the Times Square New Year’s Eve webcast. Touch has a workflow (and vocabulary) built around operators, or “OPs.” There are “CHOPs,” or “CHannel OPerators” for doing things to numbers, “TOPs” or “Texture OPerators” for manipulating images in 2D space, and so on – there’s special OP types for 2D manipulation, number channel operations, 3D space, materials, and data manipulation operations. The language is a bit unique compared to the VJ world, but that’s ok – it’s worth the time to learn, as you’ll be able to slip more readily into the Houdini FX world.
Made with TouchDesigner: The 33 meter (107 ft) long Salesforce Wall
Salesforce approached Obscura with the opportunity to create imaginative media for the 107’ long LED video wall in the lobby of their flagship San Francisco office. Our creative teams worked closely together to envision and produce work that transforms a passageway into a mesmerizing hyperreal environment. From capturing California’s Redwood National Forest in stunning 12K resolution, to a designing a convincing CG waterwall and more – we held nothing back in striving to impart a sense of wonder to everyone that enters the building.
Who is it for? TouchDesigner is for teams that are focusing on interactivity (or insane levels of flexibility) over complicated timeline/keyframe manipulations. Real-time media servers are exciting because they allow the content to inhabit the space more like a performer and less like a painting – the media can listen, respond and operate with specifiable levels of procedurality. Where TouchDesigner really shines as a part of my process is in how the tool approaches noise – there are ways to build any number of elegant solutions, allowing me to get things wiggling and jiggling without having to be so damn deterministic about it.
Who is it not for? If you don’t want to learn something new, stay away. If you want to look at everything from the point of view of a timeline, stay away. If you hate math, stay away. If you want a UI that serves you everything you need all the time (whether you need it in that instant or not), stay away. If you hate forums posts, wikis and YouTube, stay away. All of the rest of you – dig in.
TouchDesigner has one of my favorite origin stories in all of computers – it was initially a fork of Houdini, the massively powerful VFX and 3D compositing engine used in Hollywood for close to three decades. This is where all the -OP language comes from, and if it makes sense to you, go download and learn Houdini. If you are looking for work in media and you know Houdini, you will find work to do.
What are the features that cost money? This is a bit of a different proposition from what we saw with Smode, in spirit – resolution is similarly locked, and so is NDI. There’s also a whole bunch of highly advanced integration features that you’ll need a commercial license to access. The non-commercial free license does include Artnet and DMX, though, both in and out – a favorite use for TouchDesigner around my studio is as a host for LFOs and LFOs only, being utilized for processing somewhere else.
What does it cost? Touch Designer Pricing: TouchDesigner has two levels of paid license – Commercial and Pro. Commercial costs US$600, and unlocks all the features you’d need to handle a “typical” TouchDesigner deployment for a commercial job site. Pro adds a number of features specific to the needs of sites with a live broadcast/camera recording component, 3rd party projector calibration, simple sync of multiple TouchDesigner computers, custom bug fixes specific to the build of TouchDesigner you are deploying (very relevant for installations) and six hours of top-notch support.
Why do these two tools stand out?
There are, of course, other options – especially VVVV (for “free to use”) and Notch (for paid licenses.) [Ed.: Obviously, CDM will keep talking about other tools. Windows-only vvvv or “v-four” or “v-vier” in German has its own robust community; its user paradigm and licensing and workflow are all very much like TouchDesigner, which admittedly also makes choosing between them tough. Notch Builder is its own animal, and also well worth talking about when it comes to authoring motion graphics. In the long run, either could be just as strong, but I agree with Ted that if you want some short-term results to play around with, these two options here are tough to beat.]
I am recommending these two tools, Smode and TouchDesigner, specifically for artists, designers, VJs or teams that are new to a real-time workflow. They’re a gateway to the world of graphics that you can start with right away – without having to roll your own code (at one extreme) or be limited to low-performance, more creatively restricted VJ applications (at the other).
ho
More advice
Which OS – and what if you have a middle-of-the-road Mac? If you are doing graphics on a daily basis, I advise using Windows. I no longer consider the MacBook Pro a viable choice for doing live media work with modern software packages. If you are looking at a computer from 2016 or earlier and wondering “will these tools run well on this thing?” – I’m going to manage your expectation towards “not really.” If you are on on one of these older Macs, I’d consider focusing energy on VDMX, Lumen, and Photoshop – fake real-time by getting good at using VDMX’s LFOs to beatmatch visual FX parameters to music. A very new MacBook will do a slightly better job, but I still wouldn’t expect stellar performance – to push frames you need a solid graphics card. For real time I wouldn’t spec below a GTX 1080 – it’s pricey, but I honestly find myself topping out my rig all the time.
Ed.: I’ve chatted quite a bit about this with VDMX developers Vidvox. They are indeed happy on the newer Mac hardware, but they’ve also specifically optimized their software for that use case – and VDMX isn’t necessarily built for the use cases Ted is describing here. Since a lot of other software is cross-platform, even on Linux, the PC at the moment is an objectively better value for money with graphics. This isn’t a platform war, because it isn’t even a feat: there is an order of magnitude difference in flexibility and low cost for GPU power for your money, and it’s measurable. In music, it’s another ballgame: music applications tend to be less intensive than visual ones, they don’t use CPU resources in the same way, they don’t make any use of the GPU that would make a difference, and the Mac has some specific advantages that are different than the ones in visuals. You might well still choose Windows, but at least there’s more of a choice involved. And… now I’ll wait for comments on this.
Of course, follow the advice here – this isn’t about us telling you to go buy a new computer. Unless you want us to. -PK
Which framerate? Work at 60 FPS. 30 FPS is going to be less acceptable as time marches on. 29.97 is unnecessary for work heading to a projector, and you’ll want the extra frames to retime content if you are VJing a wide range of genres. A 120BPM 4-to-the-floor at 120 BPM would place a keyframe every 30 frames. Working at 60 FPS lets you get as low as half-time and still keep a good framerate, and speed up with enough resolution to maintain the critical elements of the motion.
What else should you be learning? Of course, general computer science skills are super critical to keeping this process healthy for your studio. You should be able to network stuff up, patch a video system, route audio as-needed and keep a tidy file system. It wouldn’t be bad to be familiar with some basic programming syntax, either – a lot of the time cue lists express themselves “programmatically,” and TouchDesigner goes so far as to let you use Python expressions in literally any text field.
Stay in touch! One last note – I’m looking to meet new producers and directors, if you’re looking to integrate real-time content into your project please feel free to reach out to ted@savag.es and we can talk about what you’re getting up to.
Ed.: Let us know how you’re doing with these tools and what you create! And if you work in this field, and have work you want to share or advice you want to give – including arguing with any here – we’d love to hear from you.
Also – one tip, I can wholeheartedly recommend Stanislav Glazov as a TouchDesigner instructor. He’s touring with Dasha Rush and doing various projects around Europe and Russia. Follow his studio Licht Pfad for more. We’ll be checking in with Stas soon, too, especially as he collaborates with our Establishment label. -PK
TouchDesigner: http://derivative.ca/
TouchDesigner YouTube channel (with a bunch of stuff from the Berlin summit earlier this year)
Smode: http://smode.fr/