Running Vista? Running Processing? This is important: you’ll want to apply this fix if working with fullscreen apps (as you would be as a live visualist). The short version: there’s a property to set if you’re working in an IDE (Eclipse / NetBeans. If you’re just working in Processing, the easiest thing to do is to download Processing 0156 with Java included rather than the “expert” version. But if you’re interested in knowing the gory details, here goes:

Background: I had an alarming experience this week in which I couldn’t get new builds of Processing to work in full-screen “present” mode when using OPENGL as the renderer (which is what I do nearly all of the time). I’d see a blank screen with the default Processing background color and “stop” text – and nothing else.

The problem is Java 6 Update 10. The latest release of Java on Windows added Direct3D acceleration support on Windows. From Sun Developer Network’s “Introducing Java SE 6 update 10”:

“The new graphics pipeline will use Direct3D to accelerate most common operations, so the powerful 3D graphics card you probably have in your computer will have a use other than playing games. Translucency, gradients, affine transforms, antialiasing, rectangular fills and copies, and various other graphics operations will all be accelerated by 3D graphics cards. And, unlike previous 3D pipelines which were not robust enough to be enabled by default, the new pipeline is rock-solid and used by default in 6u10.”

(emphasis mine) That’s Direct3D 9, for the record, not Direct3D 10/11, so impacts XP and Vista alike. Now, this doesn’t really do anything positive for you if you’re doing OpenGL rendering – not only the preferred 3D renderer in Processing, but many other 3D Java apps, as well. The problem is, some of the bugs aren’t ironed out with the new support, particularly on Vista:

D3D: accelerate embedded applets on Vista (DWM) [Sun Bug Database]

While they’re not directly related, it seems whatever is causing that bug with applets is also causing my Processing problem.

The fix: Disable the new Direct3D pipeline.

From Sun’s release notes:

To disable the Direct3D Pipeline, pass the following property to the Java VM:

Alternatively, set the J2D_D3D environment variable to ‘false’ prior to starting your application (or set it globally).

Here’s what that looks like in NetBeans. (The other property is necessary for NetBeans to successfully find other files it needs for OPENGL.) Click for the full-sized version:

No idea what setting a property means? That means you should be using Processing’s main download and working from that, to save you just this kind of trouble.

If you download Processing 0156, it already includes Java 6 Update 7, so if you’re just building through Processing’s included IDE, you won’t have a problem. (Unfortunately, the dedicated JDK / JRE downloads are no longer on Sun’s servers that I could find. Then again, if you have Update 7, you can use that in your IDE – but you may not want to, as this property fix will work and you’ll get the other bug fixes in update 7, many of which are targeted at Vista.)

I’ve heard from at least one person (the creator of glgraphics, Andres Colubri) that switching off Vista’s Aero effects improves fullscreen OpenGL performance. I need to test this a bit more, because that shouldn’t be the case: once you’re in a fullscreen OpenGL context, your active Windows theme shouldn’t matter.

Why do these things keep happening? This is probably a topic for an entirely separate discussion, but what this demonstrates is that developing for Vista’s new Desktop Window Manager (DWM) is complex. DWM is what supports Aero, and with good drivers, it is capable of running OpenGL fullscreen with almost identical framerates to Windows XP. But because it supports all these new Direct3D effects and changes how graphics are drawn to the screen, it has proven to be a bit tricky to work with. That’s not to say it can’t work – on the contrary, Windows remains a dominant platform for gaming and pro-grade OpenGL apps, both of which are now generally working fine. (Let’s not talk about how bad things were shortly after Vista came out, prior to SP1 and a lot of driver updates.)

So, you should dump Vista, right? Not necessarily: I’ve got everything working fine now. My sense is that, because DWM isn’t changed in any real way in Windows 7, hardware and software vendors just got a whole additional OS cycle to work out these kinds of bugs. And if you think any of this is easy, try out the 3D-accelerated windowing options for Linux, which as near as I can figure are a complete disaster and should be avoided if you’re doing anything performance-critical. That makes it all the more impressive how well Quartz and the windowing environment on Mac OS X work. They had the advantage of starting from scratch, which is even less true of Linux as it is of Windows, but maybe that’s the whole point.

At least Sun is still actively working on Windows support for Java. Let’s hope that the open-source OpenJDK stuff will help, too – probably not in the immediate future, but down the road.

If you want to follow the bug report, it’s here: [bugzilla.processing]

But the good news is, Processing (especially with recent revisions) can run just fine on Vista, as on XP, Mac OS X, and even Linux (especially with new libraries for video).