Dynamic Soundtracks

Going back to the C++ game project I described a few times in earlier posts, I wanted to add support for dynamic background music, where I could turn on or off instruments according to events that take place in the world. Plenty of media use dynamic soundtracks to create atmosphere. For example, one might want to add a delay effect in some environments or turn off all instruments other than drums and bass when entering a particular room in the game.

I implemented this in the simplest way I thought to be possible:

First, I rendered each instrument in my background music track separately and loaded each track into the game as a separate audio source. Next, I added toggles for each instrument to mute and unmute those instruments. For now, the number keys on the keyboard toggle an instrument on and off. Lastly, I modified the logic in my game’s audio loop. With one track, I simply copied all of the latest audio samples (float values) into the output buffer. With multiple audio sources, I now need to mix samples from all of the audio sources into the same buffer. This is just a matter of adding samples together, elementwise in a loop

Pseudocode:

for each audio source {
    // read data for the particular audio source into an array called input[]
    for each frame_index {
        // copy data from a temporary input array that stores the data for each audio source
        // each frame comprises a left sample and right sample if we're working in stereo
        output[(frame_index * 2)]     = input[(frame_index * 2)]; // left stereo sample
        output[(frame_index * 2) + 1] = input[(frame_index * 2) + 1]; // right stereo sample
    }
}

Demo Video:

I’ve recorded an off-screen video of the result. I have not yet set-up my computer for recording internal audio, so please excuse the quality. You can hear me toggling instruments on-and-off. Also, the system applies a subtle delay/echo effect to the sound when the box character jumps.

Extra Note:

Behind the scenes, there is more complexity, as the audio functionality runs in a separate thread of excution from that of the main game. This lets multiple cores of the CPU to handle different tasks at the same time, which, when handled correctly and with the right use cases, should yield faster results than a single-threaded program. Using multiple threads that communicate and share data requires that we ensure that, for example, one thread doesn’t overwrite data that the other was meant to read first. In other words, the main game thread and the audio thread need to be synchronized. Perhaps I could go more into detail at a later date. In the meantime, feel free to look into multithreading, mutex locks, and lock-free queues.