Virtuoso coder and prolific digital artist Kyle McDonald is at it again, here in collaboration with similarly expressive and skilled coder Arturo Castro. Together, working in openFrameworks, they make use of a face tracking library to turn the image of a face into new, terrifying visions once imagined only in science fiction. Here, going beyond a pirate hat or mustache, they transform the appearance of the face. (I hesitate to use the word “avatar” because I start to think of 90s “new media” or James Cameron films, but — damn. Yeah. This is what everyone was imagining.)
Arturo and Kyle provide thoughtful notes and technical details via Vimeo, so I’ll let them speak here. If you have other questions to ask them or want to reflect more on what this all means, we can certainly do a CDM follow-up.
But perhaps the most exciting evidence is visual: the latest two videos from Kyle conceive faces from the Philip K. Dick novel “A Scanner Darkly” and its scramble suit and active face substitution.
Descriptions:
Arturo writes:
This is a technical demo for face substitution technique. The application works in real time and it’s developed using the opensource framework for creative coding openFrameworks: openFrameworks.cc
Most of the “magic” happens thanks to Jason Saragih’s c++ library for face tracking web.mac.com/jsaragih/FaceTracker/FaceTracker.html. The face tracking library returns a mesh that matches the contour of the eyes, nose, mouth and other facial features.
That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code: methodart.blogspot.com/ gives it the blending effect that can be seen in the final footage.
I’m also using Kyle McDonald’s ofxFaceTracker addon for github.com/kylemcdonald/ofxFaceTracker which wraps Jason’s library for easier use.
Kyle has uploaded another video giving a try at a different blending algorithm, which get rids of the artifacts and an even creepier look: vimeo.com/29348533
Arturo’s more impressionistic (though also beautiful) results:
Kyle writes:
Working on an idea with Arturo Castro I feel like “good” blending looks almost too natural to be surprising. It doesn’t leave any interpolation up to your imagination. It’s possible to push this style further, so it’s less of a blend and more of a replacement, but then you get unnatural colors and shadows.
FaceTracker library from Jason Saragih web.mac.com/jsaragih/FaceTracker/FaceTracker.html
ofxFaceTracker addon github.com/kylemcdonald/ofxFaceTracker
openFrameworks openframeworks.cc/
And on the video at top:
Working with Arturo Castro on some ideas surrounding face substitution. The scramble suit is a fictional technology from Philip K. Dick’s 1977 novel, “A Scanner Darkly”. It’s effectively a cloak that hides the identify of the wearer by making it impossible to describe or remember them. There’s a nice excerpt here technovelgy.com/ct/content.asp?Bnum=997