Item — HN Reader

FM synth programming can feel like a dark art for starters. It's much less straightforward to dial in a specific sound compared to, say, subtractive synths.

Now imagine a synth that reverse-engineers audio sounds you feed it, adjusting its parameters to match. This would be a game-changer for learning. This recently released FM synth is trying to do just that with AI : https://www.youtube.com/watch?v=3gTYlYG6-R8

6 comments

As a synth head that teaches synthesizers at university: While I think that is an interesting idea in itself, it is probably not the best idea for learning how FM synthesis works. Learning synths is both about gaining an intution what sounds emerge when you change a set of parameters, as well as having a mental model of the signal flow and how that shapes the sounds.

Both are things that are not helped by having something that creates presets for you — in fact it might even be more productive to look at human made presets if you need guidence.

Also: Synthesizers are also good for creating never before heard, totally out there sounds, and those are hard to reach when you feed it existing stuff. My own personal learning approach always was to use a simple init preset and then create everything manually from there. I called this my "No-presets rule" (meaning I don't use presets that I did not make myswlf for anything, unless ot is a commercial gig). If you then make your 10th preset from scratch you probably understood something already. And then at some point looking at other people's stuff starts to make sense because now you have a set of eyeballs that can actually peeceive the things they did.

Re: your experience teaching synths in university, do you feel there is certain knowledge/concepts that are difficult for those self-teaching sound design?

I do the same "no presets rule", with most being the result of random experimentation and fiddling of parameters. This does a good job at "creating never before heard, totally out there sounds", but I imagine it's trickier the more specific your end goal is.

I think the hardest part is getting the basics straight, and by that I mean things like understanding oscillation produces sound, amplitude influences the spund level while the speed of the oscillation influences the pitch. Understanding that by modulating pitch and amplitude you can already create a ton.

The biggest challenge for students in my experience is grasping all these obscure words and acronyms, with VCOs, LFOs, keytracking, Envelopes, ADSR, Triggers, Gates, CVs, oscillations, overtones, frequency graphs, filter cutoff points, resonance, ...

It is just a lot of concepts at once. Sometimes it can pay of to take a step back and limit one's arsenal and figure out how far you can get only using one or two of those.

Totally. I'm self-taught in this area and I feel like the biggest obstacle for self-study is that the basic concepts are often passed over too quickly (like simply the idea of using phase/the unit circle to represent oscillations, and how all that relates to frequency and period). I recently sat down and tried to make sure I really understood the mathematical modeling part of the whole business and it brings a lot of clarity when tackling more advanced topics. This is kind of true for self-studying any mathematical domain generally. It's easy to pass over the "entry-level" or foundational stuff quickly because on the surface it seems straightforward, but getting those fundamental ideas crystal clear and burned into your brain is absolutely crucial if you're going to comprehend anything that follows.

This is exactly what I think. The basics need to be understood. That doesn't necessarily mean they need to be understood first tho. People that learn synthesizers are first and foremost interested in creating cool sounds, so the best strategy to teach them the (for some: boring) basics, is IMO to repeatedly overwhelm them a little and then step back and explain things throughly.

Using genetic algorithms or other evolutionary computation methods for this purpose is pretty old. There are two versions. First, there is evolutionary resynthesis, where the system is trying to optimize a synth program to match an existing sound. This can be done automatically: try a bunch of sounds, have some automatic comparator compare them to the target sound, assess them, mix-and-match, try again. Then there is interactive evolution, where the system is trying to come up with sounds interesting to the musician. Here the system proposes and auditions synth patches, and the musician responds with what he thinks are the better ones, and using that information the system mixes and matches and tries again.

Evolutionary resynthesis has been around since at least 1996 (Horner). Aphex Twin was enamored with it for a while. I don't think it's particularly interesting: if you already have a target sound, why are you trying to come up with a program to match it? Why not use the sound itself? But interactive evolution is both useful, more difficult, and interesting, as it makes it possible for the musician to explore the space of synth programs without having to program them (and that's particularly good for FM, since FM is EXTREMELY counterintuitive to program).

I wrote what I think is the best general-purpose interactive evolution style synth programming system right now: Edisyn. It works with a wide range of synthesizers and synthesis types. If you restrict yourself to exactly one synthesizer of your own design, you can get better results still, and there are a number of softsynths out there which do this.

But most interestingly: interactive evolution has a specific inventor: BRIAN ENO! He invented it around 1995 in a letter he wrote to Stewart Brand.

> if you already have a target sound, why are you trying to come up with a program to match it?

Once you have the program matching the sound, you can tweak it and get variations. At least that's what I imagine people would want it for.

>if you already have a target sound, why are you trying to come up with a program to match it

Because then you have it (or something close) in your own synthesizer, as opposed to whatever other synth or natural source you sampled it from, and you can enhance it, adjust it, use filters and others on it, and so on using all its native cabilities.

Sure. But you can manipulate the sample in complex ways with filters, effects, etc. on your sampler or rompler already. I'm not saying there's no reason -- I'm just saying it's a fairly narrow and not hugely compelling one.

>But you can manipulate the sample in complex ways with filters, effects, etc. on your sampler or rompler already.

Much much less.

If you say so. I own an UltraProteus with almost 500 filters and an extensive mod matrix. Pretty sure it can mangle a sample quite a lot.

>If you say so.

Nope, it's objective reality.

>I own an UltraProteus with almost 500 filters and an extensive mod matrix. Pretty sure it can mangle a sample quite a lot.

Which is neither here, nor there. I didn't say you can't change the sample "quite a lot", I said you can change it much less than if it was as a synthesis patch on a synth.

Any fx or filter available to the sample is also available to the synth. The reverse is not true.

As a simple example for an extremely basic thing you can't do: you can't change the sample's inherent attack. The ASDR envelope on the sampler works on top of the raw sample's attack, so you can't, for example, change a slow attack to a fast one.

That's been around for at least 30 years[1] although there is a strange lack of tools using it, I looked into it a while ago. There's a tool for generating OPN2 patches[2] (sound chip used in Sega Mega Drive) which is what I'm most interested in this technology for but I couldn't get it running right. And a modern method that uses neural networks instead of genetic algorithms but I haven't found any applications of it [3].

[1] https://www.jstor.org/stable/3680541

[2] https://github.com/jpcima/fmprog

[3] http://kth.diva-portal.org/smash/get/diva2:1334639/FULLTEXT0...

p.s. does anyone know why do so many papers on new technologies like this never even provide code samples? what's the point?

I disagree, given one of my favorite synths FM8, which in particular I've always found easy given its very thoughtful interface mechanisms of an "easy mode" and a "hard mode" both of which have extremely powerful, precise envelopes which can be dialed in really cleanly to visually carve out any kind of sound one is looking for. It's easy to grab a preset, customize to taste, and experiment with it to gain your footing and intuition around how it works.

I also disagree that a synth which "reverse-engineers" the audio sounds you feed it is the right way to go. By doing this rather than learning how to do it from scratch (albeit with training wheels as I've previously described), you rob yourself of the rich experience of learning how to use FM synthesis as applied physical modeling.

Fiddling around with FM, I've found it easy to dial in interesting sounds but very difficult to find sounds that are "musical" so it's difficult to actually incorporate them into a piece of music. I guess the wild success of the built-in patches of something like the DX7 proves that this is down to my own lack of skill but FM synthesis seems somewhat chaotic - a small adjustment to one parameter can cause a wild swing in the sound.

Try to use modulation frequencies that are in some sort of integer ratio to the carrier wave; the lower the ratio the less tinny/fizzy the sound will be. Non-integer ratios are enharmonic, good for percussion and SFX but also quickly tailing off into noise.

Underrated comment. This is one of the most inportant parts of FM, imo. Each integer ratio of carrier to modulator has a specific flavor. The "simpler" the ratio the less complex and flavorful the sound.

Therefore, fading from irrational to rational ratios can make a crazy, inharmonic sound "stabilize."

After working for years with FM, i have found a somewhat narrow range of sounds and the layering becomes more inportant than the algorithm. So getting to know of the ratios is important, 1:2, 1:3, 1:4, etc.

Another approach, rather than generating randomized offspring, differentiable dsp provides a mechanism to self-correct using a loss function

https://deepai.org/publication/ddx7-differentiable-fm-synthe...