Rust VST part 3: reacting to MIDI
We'll be building off of code built in the previous chapter. I recommend you take a look, or download the project files here.
Upgrading our synthesizer
So far, we've built a simple FM synthesizer. It can change pitch and modulation, but generally we want to react to MIDI when using a synthesizer. In this chapter, we upgrade our monophonic synth to respond to MIDI events 1.
To do this, we save the latest MIDI note-on event in our
Synthy struct, and adjust our pitch to match the current note. We'll also need to include some logic for note-off, as well as an envelope to control our audio graph.
To improve MIDI ergonomics, pull the
wmidi crate into your
Plugin::process_events trait method and see how it converts and destructures MIDI data.
In this example, we iterate through
events with the
.events(). We then get the MIDI data and try to parse it using the
wmidi crate. If successful, we can then respond to
NoteOn events 2. To save the current note, we use an option to store a note after a
NoteOn event, and then remove it on a
All together, the code in our
lib.rs should look like this:
use *; use ; use ; use *; use ; const FREQ_SCALAR: f64 = 1000.; plugin_main!;
1. Storing the note as an option
We use the
Velocity types from the
wmidi crate (see new
use imports) to store an optional note. This code doesn't react to
Velocity yet, but we save it here for later use.
2. Setting the frequency
to_freq_f64 method to easily get the correct pitch to play. If the note is
None, we provide a frequency of
0., which is inaudible. In other words, the synth stops playing when the note is
None. (This is not the proper way to start and stop playing notes, but it will work for this step.) Note that we also moved the parameter-setting part into the more predictably sized buffer used for
3. Processing MIDI events
NoteOn branch is simple to understand: we set the
note to new values (ignoring the channel for now). The
NoteOff branch is slightly more complex. We don't want to remove the note on every
NoteOff event. Imagine playing two notes in succession, without stopping the first note. Now, imagine that we stop the first note after the second note began playing. The second note will stop playing too! This would be very frustrating to musicians 3, so we fix that by checking that the
NoteOff note is the same as the currently playing note. We ignore all other events for now.
Monophonic FM synth
Compile your plugin with
cargo build --release, and open in your host. It took 3 chapters, but we finally figured out how to generate silence! Try playing a note, and see how the synth reacts.
Generating an envelope
You may see a problem with this naive implementation: the
0. Hz signal is not necessarily
0. amplitude. Additionally, we have no way to adjust the attack or release - it's either on or off. This results in a sound riddled with clicks and pops. Luckily,
fundsp also comes with envelope generation functions. For a practical example, consider the following:
let offset = ; let env = ;
Here, we create a new tag that controls the offset of an envelope. This is useful when responding to a note on event, as we can control exactly when to "trigger" the envelope.
downarc is a simple curve function that eases in and out. The output of this function applied as amplitude to our FM signal looks like this:
While not as versatile as a proper ASDR, this solution will get us acquainted with envelope generation and eliminate clicking artifacts created by instantaneously starting or stopping a signal.
Differentiating parameters and tags
You might start to realize that we overloaded the functionality of our
Parameter enum. We use it in both our
fundsp audio graph tags and our
Plugin parameters. Additionally, we no longer use the
Parameter::Freq tag, as MIDI notes now determine pitch. This code rot is a consequence of building on our initial naive design that assumes
Parameters are the same. We need separate enums.
params.rs file with the following to remove all references to
Parameter::Freq. Note that this will break our code for the next few blocks. We'll fix everything later.
use FromPrimitive; use FromPrimitive; use Display; use ;
Now, let's move back to our main
lib.rs file and create a new tagged enum for our
With a new
Tag enum, we now define audio graph labels independently of our
Plugin parameters. One flaw with this design is that integer values represent both tags and plugin parameters. This means we could mistakenly refer to an audio graph tag with a
Parameter enum cast as an integer, or vice versa. To reduce the likelihood of this error, we implement some helper methods on
This way, we leave casting to an integer up to the function and ensure the enum type provided is correct. Instead of:
We can now write:
// We always specify a `Tag` and the function ensures it is used correctly self.set_tag_with_param;
Putting it all together
We now have the building blocks needed to understand how things work in context. Replace your
lib.rs file with the following:
use *; use FromPrimitive; use ; use ; use *; use ; plugin_main!;
1. New fields
time as fields to
Synthy. These are all related to envelope generation.
timefield keeps track of how much time has passed since
fundspbegan processing. It should be identical to the internal
tparameter accessible within the
sample_rateis necessary for timekeeping, as it allows us to calculate time passed based on number of samples. At #7 we implement the
set_sample_ratemethod of the
Plugintrait that controls this field.
enabledis initially set to
false. When playing a note, it is set to
truefor the remainder of the plugin's run time. This is to prevent playing an initial noise.
2. Removal of
Just as in
params.rs, remove references to
Parameters::Freq as MIDI now controls frequency.
3. Envelope generation
This is the fun part. Let's dive in and see what's happening in-depth
// 1. let offset = ; // 2. let env = ; // 3. let audio_graph = freq >> sine * freq * modulation + freq >> env * sine >> declick >> ;
- We create a new
tagwith the ID of a new enum variant called
Tag::NoteOnand the initial value of
0f64. This represents the amount of seconds that have passed since our
AudioUnit64began processing. We will later set this tag to the time that a note was pressed. This time offsets the envelope.
- We use
envelope2, which is like the
envelopefunction, but takes an input. With
envelope2, we can pipe our
offsettag into the function with
downarc((t - offset)... The constant
2.at the end of the line scales the speed of the envelope. Try changing this yourself to see how it affects the output, or add another parameter and tag to control it.
- Finally, we apply the envelope to our carrier
sine(). We also add a
delick()node to help with audio popping.
4. Changing to 1 parameters
Because we eliminated
Parameters::Freq, we should change the amount of parameters advertised in
In this section, we increment our
self.time clock by calculating the
Duration in seconds of a
MAX_BUFFER_SIZE block 5. Note that none of this happens if the synth is not
enabled. See #6 for an explanation.
NoteOn time tag and enable synth
When processing a
wmidi::MidiMessage::NoteOn event, we now set our
Tag::NoteOn tag with the current time. Additionally,
enabled is set to true, indicating our synth has received input and is ready to create sound.
To ensure our time calculations remain accurate, we implement
Plugin ::set_sample_rate. We also invoke the
reset method on our
fundsp graph, which allows us to specify a specific sample rate.
We now have a pleasant sounding monophonic FM synth that is actually usable in music composition.
While simple, opportunity exists to make our synth more expressive. For example, there's no reason
Tag::modulation can't go above
1.0. It is only limited to that range because we directly read a parameter. We will modify our
set_tag_with_param function later to re-map the normalized value to an arbitrary range.
A quick search tells us how to map values to a range, where
x is the input value:
fn = / * + output_range_start
input_range_end is just
1, we can simplify the function to:
fn = * + output_range_start
We use the
RangeInclusive type to create a nice API to remap values. Change the
set_tag_with_params method to the following:
Now, update the usage of this function in our
process method to increase the maximum modulation amount from
// Plugin::process // ... self.set_tag_with_param; // ...
Now, a sample using our new modulation range to automate an FM bass line:
fundsp will help us a ton later when we want to add polyphony. For now its easiest to understand a monophonic implementation, as polyphony requires additional considerations like voice stealing.
wmidi can handle a variety of messages like Sysex. We discard those messages when we match
vst::event::Event::Midi, so keep this in mind if your application requires that functionality. If you're not sure, you likely do not need it!
It could be fun to make a synth that refuses to respond properly and pretend its broken or has its own personality.
If you want to be really slick, you could probably write or import a macro to count the number of enum variants and provide that to the
It might be a better idea to calculate this value once in
set_sample_rate and save it to a new