AR scratching from vanderlin on Vimeo.

“Augmented Reality” is a fancy term for describing ways of using computer vision to overlay digital intelligence on images. In other words, you can, for instance, scratch a vinyl record using a camera – plus a tag for identifying the object’s position in 3D space.

Cambridge-based designer Todd Vanderlin put together an elegant demonstration of the possibilities here, and his video has accordingly been making the rounds. (See: Synthtopia – and I actually heard about it this morning from a high school friend. The power of the Internet.)

Todd has more details on his site, which includes all kind of wonderful projects, like laser sound fountains and, always favorite around here, creepy circuit-bent baby dolls.

AR Scratching [Todd Vanderlin]

There’s actually some work to this: you need to figure out how the album is spinning. And of course, because this is augmented reality and not reality, there’s real potential here to imagine a new kind of vinyl DJing in which normal physics don’t apply.

From the video description:

I was playing around with some AR markers the other day and came up with this idea. taking just a plain old vinyl record and attaching an AR marker to the label you can track the record in 3D space. The next question was, can you scratch the record?

So by figuring out the velocity of the records rotation and applying it to the payback of the audio you can scratch. There is some digital noise that needs to bee worked out, but sounds pretty good. Its still really hard to scratch, it takes some practice but is super fun. The next step is to figure out some nice triggers for different modes. I like the idea of not needing a turntable but the actual spinning of the record helps with the scratching and playback. I made a couple modes, one where the record is paused and you can just scratch through the song. The other looks for zero velocity for x time and then continues on with the song. If there is velocity you then are scratching and the audio is affected. I think that this project has some legs can’t wait to play more.

I Want My Augmented Reality TV

So, this has sufficiently inspired you and you want more augmented reality? We’ve got more for you.

Digital artist and magician Marco Tempest has just demonstrated what happens when you do card tricks with augmented reality – and he shares some details of his rig:

Virtual Magic: Augmented Reality Card Tricks with Marco, OpenFrameWorks [Create Digital Motion]

We’ve even seen augmented reality climbing walls.

If you’re ready to do this yourself, we have a number of resources:

CDMotion has Andy Best’s tutorial on OpenCV with Processing (not an AR-specific library, but relevant): Getting Started, Popping Bubbles

We’ve also got an in-progress library for use with mapping projections in space for “spatial augmented reality”

A New Year’s video with a library for Flash, Java

A tutorial on getting started with augmented reality using Flash

openFrameworks, a library for C++ coding (which in turn supports multitouch, augmented reality – you’ll see some projects on that page)

Bryan Chung is working on a library for Processing

  • lematt

    interesting project… but it looks like it will never be 'as cool' as the real scratching…

    could be useful for something else tho

  • lematt

    don't get me wrong, it's really amazing, but just mapping the marker position to audio scratching is maybe not the best way to use it…

  • Enough AR videos already! 🙂 I'm just waiting for AR to find its place… I feel like I haven't seen anything memorable outside of Reactable (which just use the fiducials, really) and that 3D graffiti thing.

  • @Kyle: HA! Great video.

    Well, look, I think from a musical perspective, AR is just potentially a new source of control data. So, yes, if we were to say, "Wow! Look! A knob! And this knob controls – something controlled by a knob!" — that'd be fairly silly. On the other hand, something that simple has resulted in decades of electronic music. So if people can take these AR things and do something interesting with them as controllers, then this could have greater creative / expressive potential.

    Then, of course, with the novelty stripped away we have to decide whether this is a particularly good controller — that's another question.

  • Looks like this might be a way to do A/V scratching using regular (not time-code) vinyl records.

  • real

    this is insane! forget implications of music for a second. this means people are already deep in the process of information overlay. probably by 3-5 years our contact lenses will have info overlay. the whole process is moving at a mind-blowing speed

  • mastervolume

    uhh, virtual dj has been able to do this for…years?

  • @mastervolume: Virtual DJ doesn't use a camera. Now, physical manipulation may actually be preferable for DJing — but this is good fun. 😉

  • Pingback: Argumented Reality – Faça Mágica «

  • mastervolume

    Virtual DJ supports webcam input.

  • a vj

    that's just annoying as hell sorry

  • tobamai

    We see a lot of tech demo projects that let you scratch in some new way (this one, just recently the abuse ableton live effect, etc). Does anyone else think it's hilarious that they usually say something along the lines of "it's not perfect scratching, but it sounds pretty good," when it sounds nothing like actual scratching? I'm not trying to rag on this project, I just think it's really funny that so many projects have said the same thing.

    I think the reactable works really well because it also gives you a heads-up display of what's going on. Instead of having to stare at a screen to see the augmented reality, it's all laid out on the table. Ideally something like this could be used with goggles, but any setup that lets people interact without individual equipment adds to the effect of it being real.

    And yeah… virtual dj will let you control a deck with a webcam pointed at a turntable. It works about as well as you would expect.

  • @tobamai: We just need a new term for pseudo-scratching.



  • donfuan

    so much time and energy wasted, a shame 😉

    I just don't get it, when peple want to be able to scratch so desperatly, why don't they just practice?

  • djdune

    goddamn, this AR stuff is everywhere. it's so cool. I've been researching it lately to try and use it in an art project and to see it here, used in this way seems like a funny coincidence.

  • Pingback: Web News Site » Blog Archive » Augmented Reality DJ()

  • Pingback: Daily DIY Network - Science Projects Plans Guides » Blog Archive » Augmented Reality DJ()

  • Pingback: Click World News » Blog Archive » Augmented Reality DJ()

  • Pingback: Augmented Reality DJ - machine quotidien()

  • ollie

    Have you guys seen this?

    some augmented reality thing to remind people to wear masks if they are sick.

  • Talking about wearing contact lenses, Wait till they perfect graphene. . . and then wearing shades will be even cooler. . . eyeware augmented reality is just around the corner and that's a fact!

  • good idea. amazing dj.