Virtual patching has reached a major milestone: you don’t need a controller. Using just your hands and natural gestures, you can now build up entire interactive worlds, connect virtual wires, tweak knobs and faders, and play virtual instruments in PatchWorld on Meta Quest. Combined with Mixed Reality, virtual physics, and support for OpenSoundControl and Ableton Live and Link integration, it’s a glimpse of how mixed reality and VR can work for music making.
Disclosure statement: I have consulted for PatchWorld developer PatchXR on communications strategy and assisted them with how to message this update. I also served as a host for Patchathon hack events for them.
Most people focus on the display hardware when looking at VR and Mixed Reality (MR) (and xR more broadly), but as I wrote following the Apple Vision Pro launch, it’s also about natural interaction. Those hand gestures may soon extend across platforms – it’s not hard to imagine using controllerless hand gestures on mobile and desktop platforms, too. (Newer Apple Watches support hand gestures, for instance.)
But the first fully immersive virtual patching, dataflow creation experience has arrived on Meta’s Quest platform. PatchWorld was already the first tool that lets you freely build up synths, sequencers, and effects, plus other mixed media (basic color/lighting, textures and video, and 3D objects) in a VR/MR environment. You don’t have to patch things from scratch – there are tons of premade tools to play with and community-contributed worlds. But you can modify those environments and patch wires, just as in a modular system. (For more powerful from-scratch creation, there’s beta access to maker tools via PatchWorld’s vibrant Discord.)
Now you can do all of that without any physical controller, in the Hand Tracking update. For anyone who’s donned a VR headset and found the handheld controllers cumbersome, it’s a revalation. And it makes interacting with knobs and faders and jamming with PatchWorld’s unique imaginative instrumental controllers second nature.
It’s more than just a breakthrough in one VR app. We’ve had generations of software that represented virtual patch cords in two dimensions on-screen and used the mouse – or maybe touch – to connect them. PatchWorld demonstrates how that might work with natural hand gestures in 3D. Someone may have tried that before, but this is I think the first product to ship with that capability.
Here’s a demonstration of the individual gestures enabled in PatchWorld. What strikes me is how many of these are intuitive and obvious – and the Hand Tracking works really well:
Put them together, and you can navigate, tweak, and re-patch devices easily. It’s powerful for first-time users and casual gamers, of course, but it also becomes far easier for seasoned electronic musicians to manage, too. It’s how you probably imagined this would work in your head, only now it’s real:
Friend of the Site Martin Delaney has recorded a Mixed Reality jam – and this is just a first exploration; Martin said he planned to develop these ideas more. (It sounds great already – and it’s nice having a Mixed Reality backdrop.)
Because of the Mixed Reality option, it’s also possible – if you’re ambitious – to mix the virtual rig with your physical rig. You can even use your software interface as you work, as with Ableton Live.
PatchXR has been steadily adding additional features to expand connections. There’s Ableton Link support, for syncing up in jam sessions with other devices, apps, and other users in the same room. You can download an Ableton Live integration (built in Max for Live). And there’s an Open Sound Control (OSC) bridge, too.
Here’s Todd with an example of using TouchOSC via that bridge, plus a more general explanation of OSC:
Mixed Reality gets into other unexpected possibilities, like using the Meta Quest headset to record live sessions with hardware – here’s an example with the SP-404 (and in this case, the hardware controllers):
Another major recent update added physics and constraints. Combined with the gestural support, the immersive and intuitive story is becoming clearer.
I’m sure this won’t be appealing to everyone – just as VR headsets aren’t appealing to everyone. But there are some exceptional competencies here from the Patch team and community. And having seen some of the recent experiments in Unreal/UEFN and Vision Pro, here PatchWorld have gone the whole way in imagining a complete environment with
If this is your thing, PatchWorld is a free update for users, and deeply discounted (over 50% off) for new users right now. Expecting some CDM readers will have Meta Quest around and have a little holiday break, or might get one as a gift, so if that’s you, let us know how it goes. (And don’t forget to join Discord to meet other patchers and learn about events and get in on the builder tools – there’s way more than just the levels and objects you see when you load for the first time.)
PatchWorld at the MetaQuest store