“Games” are, at their heart, high-performance, real-time-optimized, interactive three-dimensional graphics engines. And that means that, by focusing on their live graphical capabilities, they can become incredibly advanced live visual instruments – the stuff of VJs and visualists.

A number of artists have put that to good use. Riley Harmon sends along his work with the Source Engine, the Valve-developed graphics engine behind the classic title Half Life 2 (and Team Fortress 2, Left 4 Dead, and the like). It’s an unusually well-balanced, solid engine that works really nicely, so a good choice. Here, it gets warped to new visual performance applications in a live set in Pittsburgh, Pennsylvania.

Just what’s going on here, you ask? Riley explains:

Half-life 2 game engine, multiple scenes (levels) i pre-created and then showed up with my desktop. I used the mouse and keyboard to manually move things in beat with music and physics engine had an effect. Every once and while you’d see the console of the game (code) come up as I would change gravity and things like that. Next time I will map MIDI to the different things.

There are some Half Life 2 models making appearances there, as well, along with the brilliant Garry’s Mod -itself an ingenious experiment with the engine.

Side thoughts:
1. I’d love to see the consoles embrace this kind of free experimentation and modification. (I know, that may be a lost cause.)

2. I’d also love to see more live visual experiments with open source engines; I’m personally looking into jReality and jMonkeyEngine in Java. jReality isn’t a game engine, but it has its own nice live physics and scenegraph. jMonkeyEngine seems to have gotten a little fragmented development-wise, but has some nice capabilities of its own, based on the Lightweight Java Game Library (lwjgl). And open source here is ideal, given that modification is at the heart of what you want to do for live visuals. Anyone out there working on that sort of project?

I’m sure you’re thinking “yes, but have you seen –.” Indeed. That’s part of why I’m posting this. Let’s get a round-up of people doing live visuals with game engines. Go!

  • ash

    I've dabbled with several engines (HL2, Unreal, Torque) and I would strongly recommend using Unity http://unity3d.com/

    It's intuitive and has an asset-centered workflow. I think it's artist-friendly and reminds me a lot of the early days of Flash (when artists with a little bit of scripting knowledge created awesome experiments in ui and interaction).

  • Yeah, Unity would be my first choice among the proprietary engines, no question. XNA is also nice, though it's more a framework than an engine. Interestingly, Unity is based on the open source Mono implementation of .net.

    Some of the open source engines are fun to play with too, though.

  • or maybe the blender game engine with some python powered scripts ,
    yes i know im a blenderjunky 8P

  • or maybe blender running with OSC :


  • ash

    @peter kirn

    Rumour has it Unity is in talks with Microsoft to have Xbox Live as a supported platform.

  • Orubasarot

    Why did I sit on this for so long? Back in '02 I overclocked the shit out of my Voodoo 5500 and got some wonderful things to happen in UnrealED with polygons getting blasted in every direction. I used the screens for a site back then but never dabbled with the idea again.

    Lately I've been thinking about how comfortable I am in UnrealED and how not very comfortable at all I am in vvvv, if only I could get an FFT input of some kind into that thing.

  • @ash: Ah, interesting. It'd make sense.

    @Orubasarot: No time like the present. Anyway, we all have real-time ray tracing at reasonable framerates (like more than one every few seconds) some time in the not-too-distant future.

  • MFO

    Albert Bertolin made very good live-visuals in the last years using the unreal game engine. His work is especially interesting as he goes further as the vj in the video above – Albert squeezes his own grafix into a game engine: http://vimeo.com/3050432
    He played here:
    (there I saw his work)

  • Pingback: Game VJ | Dama Blog()

  • http://www.vimeo.com/user400400/videos

    one is required to include victor morales when speaking of the potential of video game engines in live performance. with garry's mod and the crysis engine he has continued to push the boundaries in striking and intelligent fashion.

  • @MFO / danny: Yes, definitely some really nice work; thanks! Now I need to do a new round-up, though I'm sure there are other people who should be included, as well.

  • Daniel van Gils has created something called NuPlay.


    It is a Mod for Quake 4. But it seems that nothing is happening on the site for some time now.

    I have also thought about using Unreal Engine 3 for live visuals because it supports DOF and post processing but I am not comfortable in a huge framework like this. I rather use processing …

    @Orubasarot: You can use VVVV (or Processing … or whatever) ta handle your FFT stuff and send Data via UDP to the Unreal Engine. I tested it with UE3 and it works … Friedrich Kirchner has used this function with the old Unreal Engine for his old moviesandbox. Now he has written his own engine which is open source btw.


  • or maybe blender running with OSC :