ctp431-music and audio computing, fall...

Graduate School of Culture Technology, KAIST Juhan Nam

1

CTP431- Music and Audio Computing, Fall 2017Introduction

Who We Are

§ Instructor– Juhan Nam (남주한)– Assistant Professor in GSCT, KAIST– Music and Audio Computing Lab: http://mac.kaist.ac.kr

§ TAs: – Jongpil Lee (이종필), Ph.D. Student in GSCT, KAIST– Wonil Kim (김원일), M.S. Student in GSCT, KAIST

2

What Is This Course About?

§ Introduction to Music Technology

3Music Technology?

Music Technology

§ A whole set of different technologies that have changed the way that people are engaged in music as composer, performer and listener

4

MusicComposition

MusicPerformance

MusicListening

Distribution

MusicalInstruments

Notation

Production

Education

Entertainment

Piano

5Cristofori’s FortePiano (1722) The Piano Action

§ Invented by Bartolomeo Cristofori (1720s)

Piano

6Steinway Model D

Piano

§ Characteristics – Rich harmonics– Sustained tone– Polyphonic and wide register

§ Influence on music– Composers in the Romantic era: Chopin, Schumann, Liszt – Musical expressions– Tonal harmony: vertical relation of notes

7

Harmonics of Piano

Sound Recording

§ Phonautograph - Leon Scott (1857)– The first invention of sound recording but recording only– Recent research on image to sound restoration: http://firstsounds.org/

http://www.newyorker.com/tech/elements/new-sounds-old-voices http://edcarter.net/home/phonautogram/

Sound Recording

§ Phonograph: Thomas Edison (1877)

Edison cylinder

Grooves in Edison cylinder

Sound Recording

§ Gramophone: Emile Berliner (1887)

Effect of the Early Music Records

§ Globalization– Worldwide distribution of music– Music becomes industry

Caruso<Vesti LaGiubba>“Thefirstmillion-sellerrecord” 1930ssuperstar,왕수복

Effect of the Early Music Records

§ Musical Form– Changed by the limited duration (3-4 minute long) – “AABA” form (or 32-bar form) – Modern popular music

• Jazz standards

12

Souce:http://www.musicarrangerspage.com/251/what-is-a-bridge-interlude/

Musical Tone

§ A pitched tone has a periodic waveform

50 52 54 56 58 60−0.4

−0.2

0

0.2

0.4

time−milliseconds

amplitude

Fourier Series

§ Any periodic signal can be represented as a sum of harmonically related sine waves

Joseph Fourier

Additive Synthesis

§ Synthesize sounds by adding multiple sine oscillators– Also called Fourier synthesis

15

OSC

OSC

OSC

.

.

.

Amp(Env)

Amp(Env)

Amp(Env)

.

.

.

+

Telharmonium

§ Additive synthesizer using electro-magnetic “tone wheels” (Cahill, 1897)

§ Transmitted through telephone lines – Subscription only– The business failed

Tonewheel

Telharmonium

https://www.youtube.com/watch?v=TunkjJvbrHs

Evolved into Hammond Organ

18

Subtractive Synthesis

§ Synthesize sounds by sharping with filters

19

FilterOscillators Amp

5 10 15 20−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

5 10 15 20−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

0 0.5 1 1.5 2 2.5x 104

−60

−50

−40

−30

−20

−10

0

10

20

Frequency (kHz)

Mag

nitu

de (d

B)

Oscillator Filter Filtered Sound

50 52 54 56 58 60−0.4

−0.2

0

0.2

0.4

time−milliseconds

amplitude

20

MiniMoog (1970)

21

Moog Synthesizers

https://www.youtube.com/watch?v=usl_TvIFtG0

23

“Switched-On-Bach” by Wendy Carlos(1968)

Magnetic Tape Recording

§ High Fidelity– Nearly flat over audible frequency range

§ Malleable– Record audio can be edited

§ Multi-track recording– Record and playback simultaneously– Layer by layer recording

Music Concrete

§ Composition by tape editing– Cut– Splice– Reverse – Speed up/down

https://www.youtube.com/watch?v=c4ea0sBrw6M

PierreSchaeffer-- Études debruits(1948)

Sample-based Synthesis

Mellotron (1963)

28

Les Paul’s Innovation

§ Overdubbing– Ensemble Effects

§ Delay effects – Tape delay or phasing effects

LesPaul

Digital Audio

…1001101… …0100110…

Sample-based Sound Synthesis

…1001101… …0100110…

MusicScore Sound

Samples + Digital Signal Processing

Digital Audio Effects

…1001101… …0100110…

Digital Signal Processing

Sound TransformedSound

§ Save and play musical performance

MIDI and Sequencer

33

Step sequencer(Drum Machine)

MIDI sequencer

Digital Audio Workstation

34

Launch Pad

35

DJ Machines

https://www.youtube.com/watch?v=44F0d2CbjM0

Max Mathews

§ Father of Computer Music

§ Developed the first programming language to synthesize sound, called Music-N (1957)

§ Invented Radio-baton, a baton-type controller for computer orchestra

http://opinionator.blogs.nytimes.com/2011/06/08/the-first-computer-musician/?_r=0

Music-N (Max Mathews)

"DaisyBell”,programmedbyJohnKellyandCarolLockbaumandtheaccompanimentwasprogrammedbyMaxMathews.

Audio Programming Language

MAX / MSP / Jitter SuperCollider

Web Audio

43

Web Audio (HTML5 Standard)

http://www.google.com/doodles/robert-moogs-78th-birthday

Web Audio

https://tonejs.github.io

Course Goals

§ Understanding theoretical backgrounds in music technology today– Basic acoustics, digital audio– Sound synthesis, digital audio effects– Musical control and representation (e.g. MIDI)– Algorithmic composition– Music analysis and information retrieval

§ Hand-on experience with sound examples and code – Programming: web audio API (HTML/CSS/Javascript) – Generate, modifying and controlling sounds– Visualization and graphics– Interaction

45

What is Web Audio API?

§ High-level JavaScript API for processing and synthesizing audio in web applications– Contain a number of sound synthesis, processing, and analysis nodes

§ HTML5 standards supported in Chrome, Firefox and other web browsers– Replacing “Flash” for multimedia capability

46

1995 2000 2005 2010 201519901985

Flash: ActionScript

VS

Why Web Audio API?

§ Based on existing language (i.e. JavaScript)

§ Easy to integrate with multimedia components – MIDI (e.g. webMIDI)– Graphics (e.g. webGL)– Input sensor: camera, microphone, keyboard, mouse

§ Free and no installation

§ Platform-independent (but browser-dependent)

47

Why Web Audio API?

48https://insights.stackoverflow.com/survey/2016#technology

Stackoverflow developer survey results (2016) Most popular languages used on GitHub (2016)https://octoverse.github.com/

Course Information

§ Course webpage– http://mac.kaist.ac.kr/~juhan/ctp431/– Basic course info, schedule and resources

§ KLMS– Announcement– Q&A: discussion board– Homework submission– Grading

49

Lecture Format

§ Monday– Theory part– Read the slides before the class

§ Wednesday– Practice part– Web audio programming – Bring your laptop

50

Grading

§ Attendance: 10% – Attendance, participation in discussion, and so on

§ Assignments: 40%– Javascript programming using web audio

§ Midterm: 20%– Paper exam focusing on theories

§ Final Project: 30%– Proposal / Presentation / Submission (by sharing on the web using Github)

51

Pre-requisites

§ Basic literacy– Programming language: variable, control, loop, function, class– Signal processing: meaning of x, y, t and f, Fourier transform (hopefully…)– Music: basic music theory

§ HTML/CSS/Javascript: desired but not required

52