Camera input is at last going from waving your arms around in front of the screen to some genuinely compelling ideas. And the more designers use the camera in fluid ways, the more expressive video may be as a means of interaction.
Croatian designer Marko Dugonjić demonstrates a proof-of-concept implementation of typography that responds to your position. Using facial tracking, the text scales based on the distance of your face to the screen. The transitions are a bit jarring now, but it’s enough to suggest how this might work.
http://webdesign.maratz.com/lab/responsivetypography/
There’s source code available for the head tracking on which this is based, so anyone wanting to experiment with facial tracking – particularly in a browser – will want to head to GitHub.
https://www.github.com/auduno/headtrackr/
It’s already somewhat provocative with typography; it could be even more powerful as a mode of interaction design in general. For instance, I’ve lately been collaborating with another artist in Ableton Live. The UI there is far too rigid; as we move around the screen, we’re constantly squinting and awkwardly shifting places. While this demo is simple, it should set off a spark in UI designers’ minds that their entire approach could be very different.
It may spark other concerns, as well. The ubiquity of this mode of interaction – even though the technology is new – is of course raising new questions about privacy, particularly with Microsoft’s more-aggressive second-generation Kinect. In this case, the camera analysis stays in the client, and you must authorize the camera before it switches on. It seems, though, that we may soon need more granular tracking controls. For instance: “May we use your camera?” “May we submit eye tracking data to our advertisers?” Perhaps someone more versed in Web policy than I am has an idea about the status of this, but “do not track” may shortly take on new meaning.
In the meantime, the position of your face may or may not be data that’s an enormous privacy concern. And machines that understand where you are in space may create experiences in which our bodies are again connected to interaction. At a time when we hunch over various devices and become disembodied, that might prove a good thing.
Thanks to our friends at Letters Are My Friends for picking up on this.