Welcome to AU/VST Plugins, and all things programming+audio. Colby Leider leads this course through a myriad of research papers, software design hurdles, and of course pants on the ground.

Advanced Audio Signal Processing

Assignment #1 - The AU Plugin

This assignment was served as the perfect primer to Xcode and their AU Plugin Effect template. With this pre-build structure of files, adding DSP to the overall package was one of the easiest parts. Below, you'll find the three plugins that were assigned. I've posted demos of the two more interesting plug-ins for your aural enjoyment.

Assignment #2 - PortAudio & LibSndFile

This assignment broke the barrier of IDE programming and took us through some simple trials with command-line applications for reading, manipulating, and encoding audio files. The screen shots below are from tests run using the PortAudio API, and were run with my late 2008 MacBook Pro.

The following two audio clips are examples highlighting before/after processing using a simple gain-controlling application via the command-line. This file was made using LibSndFile, and was a great eye-opening device to audio programming in C. The command is called sfprocess, and it takes 3 arguements, as shown below in the code segment.

Usage:

$ sfprocess input.wav output.wav 0.5

//_______________________________^------- gain factor

//____________________^------- output file name

//__________^------- input file name

Beep Test

Bonus (inspired by Brant Lammig)

Assignment #3 - Feature Extraction

Taking a strong turn from the padded world of AU development, our sights were set for the wonderful world of C! The assignment: construct a command-line utility which extracted several key acoustic features from an input file. The most difficult part of this assignment was trying to get the libsndfile API to play nicely with FFTW, which, after about 10+ hours of debugging, made development in Xcode much more appealing.

Feature Extraction

Assignment #4 - Tempo Induction

The sheep of Colby's flock travel onward and upward; leaving our humble homes of Xcode, Matlab serves as a flexible scratch-pad for experimentation and implementation of the Scheirer Algorithm very commonly used to induce the tempo of a piece of audio. Clips and plots from Matlab are shown below. Mouse over the plot items to display them within the current page.

Tempo Induction

Clips

Plots

Assignment #5 - Binauralization

This week's helping of Digital Audio manipulation brings us into the domain of spatial audio. Several research papers were discussed on various techniques of encoding, decoding, and synthesizing audio outside the spectrum of a stereo pair. Our primary objective for this assignment focuses on processing audio tracks mixed for stereo playback such that they sound as natural and pleasing as possible in a binaural listening environment.

Algorithm

With no set boundaries on how to accomplish this task, I took a cookbook approach of combining several techniques in various orders to find the best result. The CIPIC Database of HRTFs was used to process the original file, creating a spatialized version of the signal based on HRTF coefficients which seemed to match me the best. After placing the sound source in a specific location in space (based on azimuth and elevation angles), a slight reverb was added to the resultant signal. These two steps aided greatly in removing the sound from 'inside the listeners head' when listened to with earbuds or earphones.

Binauralizer Signal Flow Diagram

Sample Audio Tracks (Unaffected | Affected)

Keep in mind that these tracks were processed for playback using either earbuds or small headphones. The algorithm used aimed to undo the harsh "in-head" sound that music mixed for loudspeakers can have when listened to through small earbuds. The processed tracks (marked by 'After') should sound outside of your head, and attempt to emulate a listening experience with loudspeakers when the listener is wearing headphones. Here are some specs on Apple's iPod headphones, just for fun.

Binaural Audio Conversion

Other Spatial Audio-related Topics

Presidential Composition

This one-minute piece was made using Paul Koonce's PVC and some borrowed words from our old pal Richard Milhouse Nixon. Several passes through the library's phase-vocoder and chord harmonizer led to this final work.

Project Page

Final Project - Vocal Harmonizer

Proposal

Relevant Links

Sam Drazin © 2017

Home | About | Contact | Site Map