Spectral resynthesis is having a moment now that it’s one of the new oscillator types in the mighty Xfer Serum 2. If you’re looking for other ways to get your spectral resynthesis on – or you’re a Serum user who just can’t get enough – here are three free tools to sink your FFTs into.
First, what is spectral resynthesis? The idea is you perform a full analysis of an audio clip, then resynthesize at the harmonic level without being dependent on time. It’s not granular synthesis; you’re working with the sound’s spectrum rather than just the sample buffer – and the resulting sound is different (even if both let you stretch and squeeze time creatively). You can add “spread” and “blur” functions that sample the harmonic content beyond just the single playback position.
This has been available in other commercial products, like iZotope’s Iris 2 (which could use an update)! But let’s give you some free (as in beer) tools to try out. Here are three highlights that I’m enjoying at the moment for sound design (a selective list):
Steinberg X-Stream
Steinberg X-Stream has been a favorite of mine ever since I wrote about it at launch on July 4 last year. (“We’re going to live on. We’re going to survive. Today we celebrate spectral resynthesis!”) You can grab the download for free just by setting up a Steinberg account (if you don’t have one already), and following the link to the downloader. You then get a copy of HALion Sonic – you’ll be looking for that plug-in name, not “X-Stream.” (HALion Sonic’s other freebies are also cool and worth a look.) But then you get a surprisingly powerful instrument and one fantastic secret weapon for making pads and ambient content – enough so that you might wonder why Steinberg gave it away.
To get started, I recommend loading a preset first – and then maybe experimenting with different samples of your own from there. Note that some presets have multiple layers, which you’ll see indicated by “L1,” “L2,” etc. at the top of the patch.
And just listen to how this sounds – I tried this with a random track sketch I was playing with this week:
HALion Sonic lets you layer patches and has some other advanced features, too. There are some non-obvious details to take into account:
- Click tabs to expand them to full edit views. (That’ll be the answer to, “Wait, where is the amplitude envelope?”)
- Press the ‘E’ key to toggle Edit/Play modes. That collapses the window and hides some of the HALion extras for when you’re ready to just ja.
- If you’re wondering how to add layers and where a simple “Init” preset is, while in Edit mode, select Layer, and right-click a layer and choose Load Layer. (Don’t choose Init Layer in the pop-up menu, or you’ll get a. blank layer without X-Stream!) That’ll then bring up the X-Stream Init in the Media Bay on the right. Image below.
Enjoy:
https://www.steinberg.net/vst-instruments/x-stream
Ewan Bristow EB-Diøne
Spectral resynthesis, like a lot of DSP techniques, not only offers rich and varied possibilities based on source material, but it’s also very dependent on implementation. That means there’s every reason to grab both Steinberg’s outing and Ewan Bristow’s fantastic EB-Diøne.
Because this runs in Plug Data, it also is the winner for compatibility above everything else here, with VST, CLAP, LV2, AU and standalone versions that run on macOS and Windows and Linux, with or without a DAW. The paulstretch-inspired spectral resynthesis tool has extras like a gate, and it acts as just a fantastically powerful sampler.
If X-Stream is butter; EB-Diøne is sriracha sauce. If X-Stream will make your ambient record in the next two hours, EB-Diøne will help you deliver to a director who says “I want a film score that sounds like Autechre and Aphex Twin and I need it in 30 minutes.” Go. Spice it up. I took the very same sample I used above, and got completely different results in this, because of the string harmonic interface, freeze functions, and the way it performs the actual resynthesis.
It’s down to the unique parameters here, opening up still more sound design creations. It’s truly one of my favorites from Ewan’s incredible Pd-powered set. And while it’s wonderful it’s free, do consider pitching in some cash to support the developer if you can.
Don’t believe me? Check the excellent sounds from Japan:
Fine, fine, now I need to make a demo with this (and Ewan’s other stuff).
Go. Get it. Pour on that sriracha:
And download Plug Data, if you haven’t already:
https://plugdata.org/download.html
Synthetech Sound Spectra Additive ReSynthesizer (Reason)
Lastly, here’s one for Reason fans – which seems to come from an abandoned company, but because Reason Rack Extensions never get old and they never die, and because the developer set it to free, you can use it as long as there’s a Reason.
Spectra is the most old-school of the choices here – the resynthesis is essentially wired into an additive synthesizer. But that gives it a character all its own. And it is still technically resynthesis/analysis as a waveform source; analysis as the basis of a sound is at its heart. The reason it sounds so different is that this is additive resynthesis instead of spectral resynthesis. But combine it with the other two options here and you’ve managed a seriously varied sound design palette. Find it on the Reason Studios Store:
Spectra Additive Resynthesizer
It fits under the headline, though – you have a spectral filter response you can edit and it still performs a spectral analysis. The resynthesis method is what’s additive. (Had to think that through before I published with this headline. My inner editor is satisfied.)
Hat tip to the legendary BPB (Bedroom Producers Blog) for noting this went free back in 2021. It’s funny some of the excellent abandonware/obsolete stuff out there, so it’s a gift that Reason does give them eternal life. (I really want to link to Wilford Brimley fishing in Cocoon, but it’s been removed from the interwebz, so you’ll have to just remember the scene.
Bonus round – earlier history
I should also mention Alexander Zolotov’s Virtual ANS, which in turn links the spectral resynthesis technique all the way back to its photoelectronic (pre-digital!) predecessor, the invention of Evgeny Murzin in the 1930s Soviet Union (and further developed in decades after). That’s also free-as-in-beer if you download for desktop platforms, including Linux, and just a few bucks for mobile platforms.
Here’s what it sounds like:
There’s not great documentation of the original synth in action, but here’s a 1960 film I’ve never seen before that you can follow along with easily without any language skills. Even though the ANS is an optical synthesizer first constructed nearly a century ago, watching this video in action will likely help you grasp how all the spectral examples above work, because the basic principle is the same. Imagine that the optical representation of the pitches here is akin to the FFT spectral view above, and you play through it like a score to reproduce the sound. That should help you connect your software and FFT all the way back to a player piano/piano roll/music box mechanical model (which is a lot easier for us to imagine)!
One last non-free choice: U&I’s MetaSynth introduced its own form of spectral resynthesis (complete with visualization) way back in the 1990s. Virtual ANS owes at least some of its painting-as-sound interface to this early innovator – even if everything here in turn owes some visual photoelectronic synthesis origin story to the Murzin contraption.
Benn Jordan has made an extended history of this incredible software, which saw use most famously in the creation of the “you just got sucked into a phone line” sound effect in The Matrix:
This particular element is just one interface paradigm in a deep, extensive DAW:
I’m skipping other esoteric tools here, plus resynthesis capabilities in some other paid synths. But any favorites of yours I missed? (And any more free tools?) See you in comments.
Disclaimer: this is not a history of spectral resynthesis! There’s a lot there. Daniel Miller has done a great job of tracing the history from France to Rossum, SoundHack, and Tom Erbe (including the Make Noise Eurorack module that will take you there).
Synthesizers + Spectralism: Charting the Origins of Spectraphon + Panharmonium [Perfect Circuit]
There was western/Soviet interchange (and through the entire Eastern Bloc) at the time, so I don’t think it’s out-of-bounds to also look at the stuff happening to the east, even back to the optical synths. You could do some resynthesis by the way with Tom Erbe’s original Sound Hack, albeit very much not in real time on the original hardware. But I digress…