Sound in processingDSDN 142 Creative Coding

Introducing Sound

Sound in processing is created and played back using a code library called Minim. In order to use it however, you must first import it. This is done by selecting “sketch > Import Library > Minim Audio” as shown in the image below.

Sound in processingDSDN 142 Creative Coding

Signals vs pre-recorded clips

Minim can handle two types of sound, signals and pre-recorded sound playback. There are advantages and disadvantages to both, however, so I suggest you experiment with both to see what you can achieve.

Pre recorded: Pre-recorded clips are a great (and easy) way to make an application which always sounds good. The disadvantage of using them is that you have very little control over the playback. What I mean by this is that if you want to change things like the pitch or tempo of the sample, you will run into issues.

Signal Sound: Signal sound is sound that is purely generated by Processing. Processing takes basic waveforms like a sine wave or a square wave and adds them together to create a constant sound. This method introduces MUCH more control over the playing of sound in terms of pitch, gain, tempo etc etc..... The downsides, however, are that it is much harder to use and very hard to get sound that doesn't sound awful!

As a general rule I will say that it is better to use pre recorded samples that sound fantastic. This ensures a quality application which won't sound bad when you play it. If you are keen, however, to put in some extra work and experimentation, signals can also turn out fantastic when applied properly.

Adding basic waveforms together can create much more complicated waves. This is the basic theory behind a synthesizer. Theoretically it is possible to create ANY sound this way.

Processing uses signal manipulation to achieve this effect.

Sound in processingDSDN 142 Creative Coding

import ddf.minim.*;

Minim minim;AudioPlayer song;

void setup(){ minim = new Minim(this); song = minim.loadFile("mySong.wav"); song.play();}

void draw(){ background(0);}

void stop(){ song.close(); minim.stop(); super.stop();}

Getting started with pre-recorded samples

So you have a sweet sound which you want to play in processing, The first thing to do is to put the sound where processing can see it. You need to create a new folder in the same file as your processing sketch is saved (press apple + k) called data. Any sound you put in this data folder will be seen by processing.

You can play WAV, AIFF, AU and MP3 files with Minim. For really short sounds (10 seconds or less) use .wav files. For bigger sounds, like songs, use .mp3 files.

When playing back a sound, we need both a Minim variable (this controlls all sound playback) and an AudioPlayer variable (this contains the sound we want to play).

When we say song.play() (song is a variable which holds an AudioPlayer) we play the sound contained by song.This example plays the sound on startup.... why?

This won't work if you don't have a sound called “mySong.wav” in your data directory.

Sound in processingDSDN 142 Creative Coding

We saw the play() method being used in the last slide, but there are a lot of other methods that you can use to control your sound file when it is loaded into an AudioPlayer variable:

pause() and rewind() are fairly self explanatory.

loop() will loop the track indefinitely, whereas

loop(int numLoops) will play the track through once, and then repeat numLoops times. (i.e. loop(1) will play through once, and then one more time)

skip(int milliseconds) will set the “playhead” at the specified number of milliseconds from the current position. (a negative number will skip backwards)

cue(int milliseconds) will set the “playhead” at the specified number of milliseconds from the beginning of the track.

position() will return the current position of the “playhead” in milliseconds.

length() will return how long the sound file is in milliseconds.

getGain() will return the current volume of the audio, and

setGain() will set the volume to a new level.

Manipulating your sound playback

Sound in processingDSDN 142 Creative Coding

Exercise 1

The previous example (showed again here) played the sound when the program started. change this code so it plays the sound when the mouse is pressed and NOT when you start the application.

import ddf.minim.*;

Minim minim;AudioPlayer song;

void setup(){ minim = new Minim(this); song = minim.loadFile("mySong.wav"); song.play();}

void draw(){ background(0);}

void stop(){ song.close(); minim.stop(); super.stop();}

Sound in processingDSDN 142 Creative Coding

Solution 1import ddf.minim.*;

Minim minim;AudioPlayer song;

void setup(){ minim = new Minim(this); song = minim.loadFile("mySong.wav");}

void draw(){ background(0);}

void mousePressed(){song.play();}

void stop(){ song.close(); minim.stop(); super.stop();}

Here is the solution to the last slide. You may notice, however, that the sound only plays once even if you keep clicking! The reason for this is that the sound doesn't rewind itself after it has played. The easiest way to fix this is to replace song.play() with song.loop(0).song.loop(0) tells the song to play once then loop 0 times (basically just resets back to the start) this should fix that problem.

Sound in processingDSDN 142 Creative Coding

Creating your sounds

For the following project, we expect you create your own “samples” (short sounds) using garageband and/or Audacity. The reason for using short sounds (try keep them under 5 seconds) is so you have more options with interactivity. Applications which create unique sound AND pattern compositions depending on the users interaction are much more interesting than ones which take one long song and have interaction only affect the pattern. You should aim to have the sound and pattern mirror each other in terms of their response to user interaction (if the pattern changes so should the sound and vice versa).

LINKS:Here are some links to tutorials for garageband and audacity:

Garageband Audacity