Rust VST part 3: reacting to MIDI
Project files
We'll be building off of code built in the previous chapter. I recommend you take a look, or download the project files here.
Upgrading our synthesizer
So far, we've built a simple FM synthesizer. It can change pitch and modulation, but generally we want to react to MIDI when using a synthesizer. In this chapter, we upgrade our monophonic synth to respond to MIDI events 1.
To do this, we save the latest MIDI note-on event in our Synthy
struct, and adjust our pitch to match the current note. We'll also need to include some logic for note-off, as well as an envelope to control our audio graph.
To improve MIDI ergonomics, pull the wmidi
crate into your Cargo.toml
dependencies:
[dependencies]
"4"
Implement the Plugin::process_events
trait method and see how it converts and destructures MIDI data.
In this example, we iterate through events
with the vst-rs
method .events()
. We then get the MIDI data and try to parse it using the wmidi
crate. If successful, we can then respond to NoteOff
and NoteOn
events 2. To save the current note, we use an option to store a note after a NoteOn
event, and then remove it on a NoteOff
.
All together, the code in our lib.rs
should look like this:
use *;
use ;
use ;
use *;
use ;
const FREQ_SCALAR: f64 = 1000.;
plugin_main!;
1. Storing the note as an option
We use the Note
and Velocity
types from the wmidi
crate (see new use
imports) to store an optional note. This code doesn't react to Velocity
yet, but we save it here for later use.
2. Setting the frequency
We use wmidi
's lovely to_freq_f64
method to easily get the correct pitch to play. If the note is None
, we provide a frequency of 0.
, which is inaudible. In other words, the synth stops playing when the note is None
. (This is not the proper way to start and stop playing notes, but it will work for this step.) Note that we also moved the parameter-setting part into the more predictably sized buffer used for fundsp
.
3. Processing MIDI events
The NoteOn
branch is simple to understand: we set the note
to new values (ignoring the channel for now). The NoteOff
branch is slightly more complex. We don't want to remove the note on every NoteOff
event. Imagine playing two notes in succession, without stopping the first note. Now, imagine that we stop the first note after the second note began playing. The second note will stop playing too! This would be very frustrating to musicians 3, so we fix that by checking that the NoteOff
note is the same as the currently playing note. We ignore all other events for now.
Monophonic FM synth
Compile your plugin with cargo build --release
, and open in your host. It took 3 chapters, but we finally figured out how to generate silence! Try playing a note, and see how the synth reacts.
Generating an envelope
You may see a problem with this naive implementation: the 0.
Hz signal is not necessarily 0.
amplitude. Additionally, we have no way to adjust the attack or release - it's either on or off. This results in a sound riddled with clicks and pops. Luckily, fundsp
also comes with envelope generation functions. For a practical example, consider the following:
let offset = ;
let env = ;
Here, we create a new tag that controls the offset of an envelope. This is useful when responding to a note on event, as we can control exactly when to "trigger" the envelope. downarc
is a simple curve function that eases in and out. The output of this function applied as amplitude to our FM signal looks like this:
While not as versatile as a proper ASDR, this solution will get us acquainted with envelope generation and eliminate clicking artifacts created by instantaneously starting or stopping a signal.
Differentiating parameters and tags
You might start to realize that we overloaded the functionality of our Parameter
enum. We use it in both our fundsp
audio graph tags and our Plugin
parameters. Additionally, we no longer use the Parameter::Freq
tag, as MIDI notes now determine pitch. This code rot is a consequence of building on our initial naive design that assumes Tags
and Parameters
are the same. We need separate enums.
Replace your params.rs
file with the following to remove all references to Parameter::Freq
. Note that this will break our code for the next few blocks. We'll fix everything later.
use FromPrimitive;
use FromPrimitive;
use Display;
use ;
Now, let's move back to our main lib.rs
file and create a new tagged enum for our fundsp
tag
nodes:
use FromPrimitive;
With a new Tag
enum, we now define audio graph labels independently of our Plugin
parameters. One flaw with this design is that integer values represent both tags and plugin parameters. This means we could mistakenly refer to an audio graph tag with a Parameter
enum cast as an integer, or vice versa. To reduce the likelihood of this error, we implement some helper methods on Synthy
:
This way, we leave casting to an integer up to the function and ensure the enum type provided is correct. Instead of:
self.audio.set;
We can now write:
// We always specify a `Tag` and the function ensures it is used correctly
self.set_tag_with_param;
Putting it all together
We now have the building blocks needed to understand how things work in context. Replace your lib.rs
file with the following:
use *;
use FromPrimitive;
use ;
use ;
use *;
use ;
plugin_main!;
1. New fields
We add enabled
, sample_rate
, and time
as fields to Synthy
. These are all related to envelope generation.
- The
time
field keeps track of how much time has passed sincefundsp
began processing. It should be identical to the internalt
parameter accessible within theenvelope2
function. sample_rate
is necessary for timekeeping, as it allows us to calculate time passed based on number of samples. At #7 we implement theset_sample_rate
method of thePlugin
trait that controls this field.enabled
is initially set tofalse
. When playing a note, it is set totrue
for the remainder of the plugin's run time. This is to prevent playing an initial noise.
2. Removal of Parameters::Freq
Just as in params.rs
, remove references to Parameters::Freq
as MIDI now controls frequency.
3. Envelope generation
This is the fun part. Let's dive in and see what's happening in-depth
// 1.
let offset = ;
// 2.
let env = ;
// 3.
let audio_graph = freq
>> sine * freq * modulation + freq
>> env * sine
>> declick
>> ;
- We create a new
tag
with the ID of a new enum variant calledTag::NoteOn
and the initial value of0f64
. This represents the amount of seconds that have passed since ourAudioUnit64
began processing. We will later set this tag to the time that a note was pressed. This time offsets the envelope. - We use
envelope2
, which is like theenvelope
function, but takes an input. Withenvelope2
, we can pipe ouroffset
tag into the function withdownarc((t - offset)..
. The constant2.
at the end of the line scales the speed of the envelope. Try changing this yourself to see how it affects the output, or add another parameter and tag to control it. - Finally, we apply the envelope to our carrier
sine()
. We also add adelick()
node to help with audio popping.
4. Changing to 1 parameters
Because we eliminated Parameters::Freq
, we should change the amount of parameters advertised in Plugin::get_info
to 1
4.
5. Timekeeping
In this section, we increment our self.time
clock by calculating the Duration
in seconds of a MAX_BUFFER_SIZE
block 5. Note that none of this happens if the synth is not enabled
. See #6 for an explanation.
6. Set NoteOn
time tag and enable synth
When processing a wmidi::MidiMessage::NoteOn
event, we now set our Tag::NoteOn
tag with the current time. Additionally, enabled
is set to true, indicating our synth has received input and is ready to create sound.
7. Implement set_sample_rate
To ensure our time calculations remain accurate, we implement Plugin ::set_sample_rate
. We also invoke the reset
method on our fundsp
graph, which allows us to specify a specific sample rate.
Listening
We now have a pleasant sounding monophonic FM synth that is actually usable in music composition.
Extending range
While simple, opportunity exists to make our synth more expressive. For example, there's no reason Tag::modulation
can't go above 1.0
. It is only limited to that range because we directly read a parameter. We will modify our set_tag_with_param
function later to re-map the normalized value to an arbitrary range.
A quick search tells us how to map values to a range, where x
is the input value:
fn
=
/
*
+ output_range_start
Because our input_range_start
- input_range_end
is just 1
, we can simplify the function to:
fn
=
*
+ output_range_start
We use the RangeInclusive
type to create a nice API to remap values. Change the set_tag_with_params
method to the following:
Now, update the usage of this function in our process
method to increase the maximum modulation amount from 1
to 10
:
// Plugin::process
// ...
self.set_tag_with_param;
// ...
Now, a sample using our new modulation range to automate an FM bass line:
Project files
Download the project files so far here.
Footnotes
fundsp
will help us a ton later when we want to add polyphony. For now its easiest to understand a monophonic implementation, as polyphony requires additional considerations like voice stealing.
wmidi
can handle a variety of messages like Sysex. We discard those messages when we match vst::event::Event::Midi
, so keep this in mind if your application requires that functionality. If you're not sure, you likely do not need it!
It could be fun to make a synth that refuses to respond properly and pretend its broken or has its own personality.
If you want to be really slick, you could probably write or import a macro to count the number of enum variants and provide that to the Info
struct.
It might be a better idea to calculate this value once in set_sample_rate
and save it to a new Synthy
field