interdisciplinary courses and assignments in digital sound production jennifer burg department of...

InterdisciplinaryInterdisciplinary Courses and Assignments Courses and Assignments in in

Digital Sound ProductionDigital Sound Production

Jennifer BurgDepartment of Computer Science

Wake Forest University

S

eiq

q

Points of View, Levels of AbstractionPoints of View, Levels of Abstraction

science arts

application

theory

S

eiq

q

S

eiq

q

In what ways is it helpful, in the teaching of computer science, to move students from theory to practice – which can be, essentially, moving them through levels of abstraction?

What teaching tools are available to help us do this?

My examples are from the realm of digital sound, presented as a computer science course, sometimes with an interdisciplinary flavor.

We can tell students:

sin() = a/b where a is the length of the side opposite angle and b is the length of the side adjacent to angle .

and

This is a sine wave:

But a more dynamic presentation can make this clearer. [Interactive tutorial “Visualizing Waveforms in MATLAB”*]

0 5 10 15 20 25-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

S

eiq

q

*Exercises and tutorials in this talk are from The Science of Digital Media by Jennifer Burg, to be published by Prentice-Hall in August 2008.

We can show students how to apply a frequency filter to sound in Audition, Music Creator, or Logic:

[Screen capture of applying a high shelf filter in Cakewalk Music Creator]

Allowing the students to implement their own filters gives them understanding at a lower level of abstraction.

[Screen capture of generating a 440 Hz tone in Audition][MATLAB exercise on FIR and IIR filters][MATLAB exercise on writing a program for a transfer function to create a filter][Exercise on creating an FIR filter with the windowing method]

S

eiq

q

We explain the Fourier transform to students with words and equations:

S

eiq

q

S

eiq

q

But an interactive tutorial can help to make the concepts clearer.

Note: The following links are web addresses.

[Interactive tutorial on Fourier transform][Interactive tutorial on the difference between the DCT and the DFT][Interactive tutorial on windowing functions for the Fourier transform]

And an assignment that has you apply the Fourier transform for filtering can be even more revealing, as we’ll see in a moment with an assignment using MAX/MSP.

Or we can have students implement the Fourier transform with their own program:

[Fourier programming assignment]

So what is my point?I’m just trying to reinforce, with examples, something that you already know:

Not all students learn the same way. Some think in terms of mathematical abstractions, some algorithmically, some visually, some verbally.

Even if they think, habitually, a certain way, it’s good for students to see the concepts presented from different points of view and at different levels of abstraction.

S

eiq

q

Computer learning environments can help us to place students at different levels of abstraction.

Collaborative work in these learning environments can provide alternative perspectives from students of different

disciplines.

S

eiq

q

Levels of Abstraction

Digital Audio and MIDI

Audition Music Creator

MATLAB MAX/MSP

C Programs

Levels of abstraction, moving from theory to application, represent one axis.

The other axis moves from science to the arts.

What’s the pedagogic usefulness of moving along this other axis, between disciplines?

artists(musicians or digital sound designers)

digital audio and

MIDI processing

Points of View

computer scientists

When and why compress the dynamic range?

Does it sound thin? Should we add reverb?

How does compression of dynamic range work? Does it affect frequencies?

What filtering tools exist? Should I make my own?

My collaborator, Jason Romney,Digital Sound Designer fromNorth Carolina School of the Arts.

NSF CCLI grant:“Linking Science,Art, and Practice with Digital Sound”Jennifer Burg, PIJason Romney, co-PI

The experiment:

Three graduate students from North Carolina School of the Arts’ School of Design and Production joined seven Digital Media students to try some sound exercises in MATLAB and MAX/MSP.

S

eiq

q

The assignment:

Try to create a single-frequency note in digital audio within MSP. See if you can generate a sine wave at the frequency you want. The sound should be in digital audio form.

Try to apply a filter to the sound to determine if its frequency component or components are what you expect them to be.

If you can get the frequency out of the filter as a number, see if you can then map that number to a MIDI note of the same frequency. That is, send a MIDI output to a MIDI device to see if you can play that same note through the MIDI device.

S

eiq

q

MAX/MSP patch

The experiment:

Music and theatre students join computer science students in a ½ semester course entitled “Digital Sound for Music and Theatre.”

How much will they be able to learn about the science and practice of digital sound production?

Will they be able to produce a song or a theatre scene?

How will they interact and collaborate?

S

eiq

q

The assignment:

A computer science student and theatre student are paired to do the sound for the opening storm scene of Shakespeare’s The Tempest.

S

eiq

q

Storm scene from“The Tempest”

To get to the point, is there something to be gained by placing students of different disciplines in collaborative learning situations? …

for example, computer science students and students of digital sound design for theatre.

S

eiq

q

It makes the work more INTERESTING, RELEVANT, and EXCITING.

The computer science students learn what tools are really useful to practitioners and artists, and how these tools are used.

The music and theatre students learn that they can have greater power over their tools by discovering how to do things at a lower level of abstraction.

S

eiq

q

Ideas generated:

How to solve a sound design problem for the play Floyd Collins.

How to solve a sound design problem for The Lion King.

Things that matter to sound designers and things that don’t matter

Training your ears.

S

eiq

q

This summer:

An 8-week music production workshop with 3 computer science students and 3 music students.

We’ll see what happens!

S

eiq

q