What can a new digital synth be in 2011? How will it work and sound? And given access to so many excellent tools, how can it stand apart? In place of a press release and some marketing-speak, developer Christopher Penrose (Leisuresonic, Cosmovox) sent us an extended essay explaining his thinking behind his just-released SynthTronica synth for the iPad. Aside from getting into the nitty-gritty technical details, it cuts to the crux of the issue: how to make something personal and new that nonetheless can work for other people, and how that idea can be tailored to a tablet.
As the geeks are glued to the latest iPad announcement, let’s look for a moment beyond platforms. Great ideas in synthesis endure long past platforms. The specific medium is wonderful in that it gives designers, engineers, and musicians the opportunity to realize those ideas, while presenting certain conveniences for developer and user alike.
All of this is worth reading in this case as it sounds like Christopher has a synth that isn’t like everything else out there – not at all. With audio files of your choosing transforming the timbres of synthesized sound in a graphical, spectrally-displayed filter, it looks like it could be a brilliant canvas for producing unusual sounds. That might help it find a place wired into your desktop PC or Mac for production.
In fact, it reminds me of the kind of creative synth we’ve seen all too rarely. The design feels heavily reminiscent of the ground-breaking U+I MetaSynth conceived by Eric Wenger (of Bryce fame). I was always disappointed other software didn’t run with some of those ideas; seeing it with some new twists, the take of a different artist, and touch input on the iPad looks terrific.
Christopher’s notes are quite long, but worth including in their entirety, especially knowing we have other developers in the crowd.
And, oh yeah, we could ramble on about this all day, but I think most of you will get the idea from this video below. It sounds great, and since you can input different audio files to get different filter content, you may be able to escape both overt recognizability and the disposability of many mobile and tablet instruments.
Here’s Christopher:
Notes on SynthTronica’s Development
Back in January of last year, I had been following the Apple tablet rumors with great interest. When the iPad was announced, I was surprised by both its name and its operating system. But it took me only a few hours to decide that I was going to design a synth for the new tablet.
Context
I have developed music software, with varying levels of attention, since 1988, and much of that effort has been spent developing idiosyncratic DSP algorithms for sound exploration. In particular, I focussed on spectral techniques for mating sounds — taking the characteristics of two (and sometimes more) sounds to create a new one. These efforts haven’t made it out of the Max/MSP, Pd, and Unix shell software ecosystems largely due to the limitations of audio plug-in hosts. “Side-chain” processing implementations are obscure and clumsy.
I am also a composer, and, until recently, my software was largely designed to aid my personal music-making. I can honestly say without pretense that my music is idiosyncratic; even Illegal Art, a label which has released one of my albums, regularly characterizes my music as being on the “challenging” end of the spectrum of their musical offerings. I think that SynthTronica has been a good project for me. While developing it, I have been challenged to corral and focus my motley DSP technologies into a broadly-accessible musical instrument. I took a music making process that combines synthesis and sampling, which I used often in my music making in the last decade, and put it at the core of a keyboard synthesizer. Hopefully, I have been able to distill an elegant instrument design from my personal composition practices.
Synth Architecture
SynthTronica is a hybrid instrument; its synth engine combines characteristics of virtual analog synthesis and digital sampling. For most “traditional” synthesizers, sound evolution is controlled by parametric filters — combinations of VCFs and LFOs. Instead, SynthTronica uses time-varying filter data to provide spectral evolution; an instance of such data is called a “formant”. Formants can be created in several ways, through the iPad’s microphone, importing audio files, and capturing performances of SynthTronica’s multitouch filter. While formants are currently played in strict loops, the maximum formant duration is fairly large — just over 60 seconds — providing potential for significant, albeit prerecorded, variation. The benefit of formants lies with their generality. A formant can be made from sources as disparate and varied as Nord percussion, cellos, choirs, braying donkeys, or the chorus of Katy Perry’s latest single. The latter example is an interesting consideration: a formant can reflect much of the rhythmic and sometimes vocal characteristics of its source sound, while effectively obliterating its pitch. Pitch is instead provided by SynthTronica’s synthesizer front-end. When readily-discernible formant sources are used, SynthTronica provides a unique musical space that lies between the boundaries of pure sound synthesis and referential sampling. Formant synthesis is not explicit like sampling; you play through the Katy Perry groove as if it was your avatar. With SynthTronica, a performer needs to make pitch choices for any sound to be heard.
Multitouch Filter
The iPad’s large touch surface was a serious attraction for this project. The idea for the Dynamic Multitouch Filter immediately came to mind. SynthTronica’s multitouch filter serves as a live and expressive counterbalance to the static character of formants by providing fluid gesture-controlled filtering of the synthesizer’s output. Given the spectral architecture of the synth engine, adding up to eleven touch-triggered filters (eleven per voice, technically, though they currently are used synchronously) does not overwhelm the processor resources of the iPad.
Hold Mode
Perhaps an ambient musician’s dream, hold mode simply allows notes to sustain by a single touch. They can be released singly by an additional touch, or released en masse via the “all notes off” button. Hold mode is an excellent counterpart for the multitouch filter, as the filter can easily be the focus of both hands when notes are sustained automatically.
Multiple Keyboards and Scale Patterns
While there are several ways keyboard control could be further developed in SynthTronica, I took advantage of several possibilities made available by the tablet design of the iPad. It is clear that a touchscreen does not offer tactile feedback, so I sought to implement interface dynamics that would make up for this lack in several ways. I choose to support multiple keyboard designs providing two piano style formats and a unique grid-based design.
From my iPhone app Cosmovox, I had a large database of musical scales available. I repurposed these in SynthTronica by providing selectable scale mappings for the keyboards. The piano keyboard has a particular design pattern which is accentuated by the contrast of white and black keys. Being a mallet percussionist in a former life, it was clear to me that this color contrast is optional and the key arrangement itself provides enough information to discriminate notes on a keyboard. Thus musical scales can be represented on a keyboard by changing this color contrast pattern. I often desire to escape my habits when creating music, and altering the keyboard scale pattern can be revealing for me. Further, the scale pattern facility allows for the use of a more radical keyboard design: the grid keyboard. The grid keyboard alters note relationships in interesting ways. The keyboard is compact, allowing one hand to access a two-octave range. Large intervals are no longer as physically distant from a given pitch. The keyboard can be bewildering to play if you play by note (which is a positive feature for me particularly as the keyboard is optional), and can reveal fascinating characteristics of scale architecture.
Design
I went with a modernist design aesthetic as I believe that SynthTronica does not have any appropriate analogs in gear. I find that creating interfaces for software that imitate gear introduces problematic usability issues. While I can understand the desire to have every useful performance control accessible on a single screen, there are practical limits to the number of interface elements that can coexist and still remain effective. Virtual knobs use less screen real estate, make a reference to audio gear, but are more difficult to use than sliders. I chose a slider-only interface using color and orientation for contrast. SynthTronica’s multi-screen design may reduce the accessibility of parameters during performance, but I think the architecture of the synth favors pre-performance sound design and emphasizes use of the Dynamic Multitouch Filter for expressive control in live performance.
Christopher also includes some frank thoughts on limitations of the synth for the time being, including some of his concerns about third-party audio interface support generally. This is beyond my area of expertise, so I’ll leave others to reach what conclusions they will – and I suspect we’ll hear some other developer views.
Limitations
While the preset architecture is robust from a database perspective, it can be frustrating for performance in its current state. The design of the reverb processor is one of the culprits. If reverberation time is different between two presets, changing from one to the other while the synth is sounding can produce awkward glitches. It is possible to ignore reverberation settings from presets by adjusting a SynthTronica parameter in the iPad’s settings application. I would like to improve preset change behavior in a future update.
Some goodies that the electronic music literati would desire — MIDI, audio interface support — have yet to be developed. MIDI is actually very high on the list now, as Apple has provided SDK support [Core MIDI] and I have purchased two Akai LPK25s and an Emu XMidi 1×1 for testing. OSC support is minimal at the moment: there are no supported in-bound messages yet, but a few outbound messages are implemented. Full class-compliant audio interface support will not be added until SynthTronica migrates to an iOS 4.x-only architecture, and even then there may be a performance reduction for many interfaces. A rant could be placed here which would be directed at audio interface manufacturers.
What I will say is that SynthTronica is less flexible with respect to audio buffer sizes as it is a spectral synthesizer; it uses power-of-two FFTs. But this is not unheard of in the least for audio processing; there probably isn’t an MP3 player that does not use them. To support audio buffer sizes that are not powers of two would cause a significant reduction in performance for SynthTronica (namely, a 50% reduction in usable polyphony due to CPU spikes). While there may be an audio interface that works out of the box with SynthTronica, I can’t name one at the moment. The class-compliant audio interfaces I have tested refuse to provide a power-of-two buffer when requested. While I am sure their engineers can come up with an excuse, they really should understand that power of two constraints are ubiquitous in computing, particularly for digital audio signal processing; it is bizarre that their hardware forces applications to perform additional buffering to support powers of two.
Anyway, let’s wrap up not with words, but with…
Sound Samples
HypnoticCaressing by Leisuresonic
More
Check out the sites for more. It was a bit unorthodox to include all these thoughts, but I enjoyed reading it and it made me want to spend some time with the synth. Let us know what you think.
http://synthtronica.com/
http://leisuresonic.com/