Can editing sounds be as easy as editing pixels in a tool like Photoshop? That’s the question asked yet again by an audio editor, in the announcement of a new tool called Spectral Layers, seen in a new teaser.
Visualizing sound is not a simple problem, but you can do worse than the spectral view. Mapping frequency over time rather than just amplitude, the graphic spectrum illuminates components of a sound as we hear it, showing sonic energy of different frequencies in brightness and color. And audio editors have routinely made use of these views, whether as displays in various audio editors (some editable, some non-editable views), or in graphical tools like the ground-breaking MetaSynth. In fact, even Adobe themselves have weighed in on the “Photoshop for sound” notion with their own Soundbooth app, which, naturally, copies the toolset verbatim from the company’s flagship Photoshop image editor. See also: Photosounder, which perhaps comes closest to this tool, and SPEAR, which is available free on Mac and Windows and has some fascinating resynthesis features. (Spectral sound design probably deserves its own post, later on!)
Spectral Layers nonetheless looks to potentially break new ground by focusing entirely on the idea. Whereas many audio editing tools that use spectral views have had modest editing facilities, here, it’s the entire program — and with some nice twists. On-the-fly selection previewing means that you’re constantly listening to your audio, not just looking at it. Advanced selection brushes make honing in on certain parts of your sound more precise, including by essential harmonic editing tools. (We hear harmonic relationships intuitively, so editing wave spectra at the literal frequency, rather than in the logarithmic proportions with which we hear, doesn’t work nearly as well.)
Spectral Layers also works with visualizing spectra in more compelling ways than just the typical, two-dimensional frequency vs. time view. Three-dimensional visualizations make seeing details in the sound easier.
Then you get into the actual editing. The developers are promising some powerful features, from extraction to independent pitch and time transformations, all moving this well beyond eye candy to the realm of deep sound editing. (The UI shows other features as well.)
There’s a new UI tutorial, but some of the features in brief:
- Cross-platform Mac and Windows compatibility
- Non-destructive layers for editing, plus compositing audio either by adding or subtracting a selection from a sound. (The latter sounds fascinating for sound design.)
- A multi-pane UI, similar to tools from Apple and Adobe and familiar to people with a graphic software background.
- 32-bit float spectrum.
- Surround project support.
- Pattern matching algorithms for still more-sophisticated selection and editing.
- An “open project format” (presumably something XML-based or the like).
- SDK for file formats, devices, tools, and filters.
In other words, the whole thing sounds mind-blowing and gives us everything we’d want … on paper. Presently described as “alpha stage 2,” the tool is still in development. But we’ll be watching.
DIVIDE FRAME, the developer, is a Paris-based house led by engineer Robin Lobel. Unrelated to the music side of this site, they also have a GPU-based video decoder, but no trial of the audio software – yet. Stay tuned.
Updated: while this is just a teaser, lead developer Robin responds with some more details for CDM:
There are 4 categories of tools: info (to get extensive info on the spectrum), extract (brush, frequency, harmonics, multichannel, noise (wip), time (wip), and others incoming), modify (so far only erase/amplify, but much more coming to transform the sound, like blur and other graphical modifications), draw (any tool to directly draw sound, as frequencies, harmonics, noise, etc).
Available Q4 2011, no price range yet (expect it to be the high, but there will probably be a light, affordable version too)
3D visualisation can display both amplitude or phase velocity using the GPU (OpenGL), it is seamlessly integrated with the 2D view (right clic+drag to make it 3D as you want, double right clic reset to 2D)
I do independent R&D in audio/video for several years now, have worked in some French [post production] companies as R&D developer and [graphics artist], wanted to start my own business (first with GPU Decoder as a small project, then came Spectral Layers). Spectral Layers came from the need to get clean voice tracks when shooting movies (as I do short movies too), then I thought of extending the concept to a general purpose, Photoshop-like tool. iZotope RX and Adobe Audition were not enough for my needs — I found the spectral editing pretty limited — so I decided to do my own.
Thanks, Fahad, for the tip!