Part of the big idea behind Resolume Avenue 3 is putting audio and video in the same real-time performance app, without sacrificing quality or live processing in either. One key ingredient, hope the developers, is a new codec that transfers video decoding to the GPU without taxing the resources the GPU needs for accelerated effects. For lower-resolution footage, you may not notice a difference, but with a modern graphics card, Resolume claim intense multi-layered performance at higher HD resolutions. We’re keen to see just how that stacks up in benchmarks, not only Resolume vs. Resolume, but Resolume’s DXV up against codecs like Photo JPEG in other apps. But before we get there, we first wanted the inside scoop from the developers.
I got to speak to Resolume’s Bart van der Ploeg. I knew Bart had been dying to show us the new codec for months, so here I get to grill him about questions a lot of us have.
The good news: no, this won’t screw up your GPU-based shaders.
How did DXV come about?
While doing our research for a new GPU-based rendering engine for Resolume back in 2006, we discovered that a large bottleneck for a fast GPU video engine was the limited bandwidth to the video card. The rendering of video frames with fancy effects at a high resolution was absolutely no problem for the GPU, but half of the time it was just waiting for all the video frames to arrive to the video card’s VRAM. We just could not get the video frames to the video card fast enough to feed the GPU to its maximum potential.
We reduced the amount of data that has to travel to the video card by feeding it compressed video frames instead of uncompressed frames that come from any other traditional codec. This makes the whole engine run much faster and reduces the CPU usage so we can use that for other things like audio effects.
We choose QuickTime because it’s cross-platform, but QuickTime .mov files only act as a container from which Resolume 3 reads compressed frames. This is why DXV videos do not suffer from the generally lower performance of Quicktime on Windows. Large parts of QuickTime are simply bypassed during playback in Resolume.
You note that it was developed with UnitedVisualArtists; what was the relationship between UVA and Resolume?
We hired Dave Green from The Pixel Addicts to help us with Resolume 3 part-time while he was also freelancing for UVA. UVA liked the GPU video decompression algorithms we had developed with Dave, so we decided to share the development costs to turn the developed algorithms into an easy-to-use codec.
The Pixel Addicts and UVA both support DXV in their high-end media servers, so it’s currently used for large shows in club Matter in London by the Pixel Addicts and on the Kylie Minogue tour by UVA.
One drawback of DXV, I understand, is that it does take longer to encode — thus, adds a little work to prepping your footage — correct?
Yes, encoding unfortunately takes longer than most other codecs; we are researching a solution for this.
In most cases, the file size is also larger than MJPEG (Motion JPEG), but in some cases
it’s smaller, especially when a video has large, simply-colored parts.
But reading larger files is not a problem, hard-drives are more than capable of reading the data fast enough. It’s the CPU that’s the bottle neck; the GPU does it much faster.
Have you found any difference in terms of ATI versus NVIDIA graphics cards?
There is no difference in DXV performance between ATI and NVIDIA. But the faster the GPU, the better the DXV performance is.
How about Windows versus Mac?
Encoding is a bit faster on the PC, but we have not seen much difference in DXV playback performance in Resolume, which is great, because QuickTime video playback is usually slower on the PC. [That’s not the case] for DXV, because we bypass a lot of the QuickTime API.
Any chance we’ll see DXV in use beyond Resolume – what about other VJ apps?
Yes, it can be used in other software; The Pixel Addicts and UVA both use it in their high-end media servers. If any other VJ software is interesed in supporting it then we’d love to hear from them.
Ed.: Note – we didn’t get a solid answer as to how this would be licensed or arranged, so you’re on your own, folks!
We did get readers asking about that specifically, wondering if they would want to add an additional encoding step for video that then wouldn’t be usable with other VJ apps. Obviously not an issue if you’re planning a set with Resolume Avenue 3 exclusively.
Indeed if compatibility with other VJ software is required, then DXV is not a good solution yet. But if you’re looking for the best possible performance, then DXV is great.
Since DXV is decoded on the GPU, won’t that have an impact on the performance of GPU-based video effects and shaders?
Using DXV will not slow down any other things rendering on the GPU (like video effects) because it uses features that are otherwise not used. DXV does not use shaders so you can use any shader-based effects without reducing their performance.
Ed.: I tried to get some more specifics on this, but Resolume is holding some of their cards close to their chest as far as how DXV actually works, as their secret herbs and spices! Suffice to say, they claim if you run the benchmarks, you’ll be a believer. Specifically, Bart responded —
“Just saying that it doesn’t use shaders is already [saying a lot.] Benchmarks will do the rest to convince people effects really are not slower with DXV.”
How well does DXV work when scratching video?
Performance when scratching and jumping in the video is excellent because every frame is a key-frame. DXV uses spatial compression, not temporal compression.
What are you using on the back end? Is this still making significant use of the QuickTime API?
No, it only uses small parts of the QuickTime API.
What machines are you testing on in your rig?
We have a couple of MacBooks Pro and an 8 core Mac Pro with ATI and NVIDIA cards. All our machines run Mac OS X and Windows.
Unibody MacBook Pro 2.4 GHz Intel Core 2 Duo
7200 RPM Hard Drive 2GB RAM
NVIDIA GeForce 9400M + 9600M GT 256MB
MacBook Pro 2.33 GHz Intel Core 2 Duo
7200 RPM Hard Drive 2GB RAM
ATI Radeon X1600 256MB
2x 2.80 GHz Intel Xeon Quad Core
7200 RPM Hard Drive 2GB RAM
ATI Radeon 2600 XT 256MB
Nvidia 8800 GT 512MB
In a world in which 640×480 VJ sets are pretty standard, and 320×240 is a pretty recent memory, multi-layer 1920×1080 is, of course, a pretty big deal. Resolume 3 is still hot off the presses, but can you talk about how these capabilities are being used so far? (And when do we get HD-ready visual clubs? I’ve got a few ideas for a Resolume cocktail to serve at the bar…)
HD mixing is one possibility but also multi-screen performances where you feed two or three screens from a large composition in Avenue. Eyesupply uses the DXV Codec for this with great success on large events with multiple LED screens.
Links:
http://www.resolume.com/avenue/dxv.php
http://www.thepixeladdicts.com and their Media Server
http://www.uva.co.uk