As someone relatively new to Processing, I’ve experienced equal parts excitement and frustration. 0115 is a reasonably mature build, and I’m glad I didn’t try this earlier, but more advanced work is likely to get easier as the software develops, improves, and squashes bugs.
Before I dive into the full, technical explanation for those of you using Processing, here’s the Cliff Notes version:
- Processing is great, but think of it as a sketchpad — and “beta” is accurate; prepare to hit up against some limitations that could become a time suck if your project is dependent on them.
- OpenGL rendering works, but has major limitations when mixing pixel processing. Then again, you’ll want to use Processing’s other built-in rendering engines so you can share your work on the Web, so try those, and consider waiting on your OpenGL sketches until the bugs are fixed.
- Video capture on Windows (and Mac, to a lesser extent) can be a pain because of QuickTime for Java. JMyron works around this on Windows with DirectX, but there is an important caveat; again, you may find things feel like a work in progress.
- JMyron is fantastic for doing simple motion tracking without a lot of coding. (Here’s where Processing wins; Jitter and cv.jit, for instance, are much harder to use in this area.) Example image below from JMyron’s site.
Interesting example of using JMyron/Processing: Shadow Monsters [Pixelsumo entry]
3D rendering issues
If you’ve had trouble with OpenGL in Processing, don’t worry: it may not just be you.
Processing has multiple renderers, P2D, P3D (based on Java3D), and OPENGL (based on JOGL, Java’s most popular OpenGL implementation). The OPENGL renderer in its current state has one hugely problematic bug: all the major pixel operations are broken. loadPixels() and updatePixels() are both necessary to pixel display, and blend() for blend operations. It should be possibly to create your own PGraphics object and render to that, but I’m exploring how that works and whether the performance hit is painful as a result. This is important to me, because I like to mix video imagery with 3D Watching some of the OPENGL bug reports, I don’t envy the Processing developers: creating cross-platform Java/OpenGL code that works sounds like it’s a pain in itself. Hopefully they’ll get this sorted; it’s been marked high priority.
Possibly for related reasons, the handy (if a little quirky) Fullscreen API external library also doesn’t work with the OPENGL renderer.
You can read about some other OpenGL issues on the Processing FAQ.
Don’t get me wrong: OpenGL is still very usable for many tasks, and you’ll see you can get direct access for OpenGL calls, which works nicely for general rendering settings. But if you’re mixing pixel processing, for now, stick to P3D. And if like me you’re coming from a Jitter background, don’t expect Processing to be quite on the level of Jitter 1.5; in the words of the FAQ, Processing “is not intended as a nice wrapper on top of OpenGL” (which Jitter is). My guess is things will improve in the next major release based on the priority of bugfixes. I’m personally looking at projects and dividing them by ones that look Processing-friendly and those that seem like they might be better long-term, full-fledged Java + JOGL projects.
JMyron for Capture and Motion Tracking
JMyron is a fantastic library for computer vision, and one of the reasons I’m working in Processing. (Old hats at Director will remember it as the Myron WebCamXtra.) I’ll have a full write-up about using JMyron soon, but first, some good news/bad news.
The good news for Windows users it that JMyron now lets you use the DirectX video capture facility in Windows instead of the kludgy WinVDIG with QuickTime for Java. The latter can cause compatibility problems, but most of all, it’s slow; DirectX seems to run with less latency than QuickTime capture on the Mac. (QT for Java could be a whole separate, painful discussion.) You can turn off the motion analysis features of JMyron and just use it as a capture library.
That’s the good news. The bad news is, JMyron doesn’t capture video input into a PImage object, the main image object in Processing (and the one you’ll need for texturing, blending, displaying, etc.) You can use imageCopy() to copy to the pixels[] display array, but then you run up against the OpenGL bug above. By the time you manually copy out the pixels, you can wind up with worse performance than you started with. For now, I’m just limiting myself to imageCopy(). (Note that this won’t matter if you’re just using JMyron for analysis and not displaying the input video signal.)
There’s a new thread on the Processing boards by dmichaelson that sums up the issues quite nicely:
What he said. (Now, note, this isn’t as big a problem as long as you don’t do what I do, and keep wondering why you can’t copy JMyron image data into a PImage array!)
If anyone is interested in this, I can post some sample code.
If you do decide to play around with JMyron, follow the install instructions carefully and then check out the examples. The new features aren’t specifically documented, so look for the JMyron_fasterSimpleCamera example. It includes example code for both JMyron’s new imageCopy() and getForcedWidth() / getForcedHeight() features. (You’ll want the last two if you’ve been getting display bugs on Windows, like I have!)
Using Processing?
We’d love to hear from you. No question is too stupid (or too smart), believe us. To get us started, here’s a thread on Create Digital Noise:
Calling fans of open-source visual tool Processing