audible design - diagrammatic appendix

39
AUDIBLE DESIGN TREVOR WISHART APPENDIX 2 A diagrammatic guide to sound compositional processes. Published by Orpheus the Pantomime Ltd: 1994 Copyright: Trevor Wishart: 1994

Upload: angie-jennings

Post on 21-Jul-2016

71 views

Category:

Documents


2 download

DESCRIPTION

: Audible Design -DiagrammaticAppendix

TRANSCRIPT

Page 1: Audible Design - Diagrammatic Appendix

AUDIBLE DESIGN

TREVOR WISHART

APPENDIX 2

A diagrammatic guide to sound compositional processes

Published by Orpheus the Pantomime Ltd 1994

Copyright Trevor Wishart 1994

1

UNIVERSITY OF ILLINOIS UBRARV

AT URBANA - CHAMP~IGr MUSIC

JJL 3~S-

J

WAVEFORH

sound represented in the time domain

When looking at much longer blocks of time waveform would be very compressed in display so a different format is used

SPECTRUM

sound represented in the frequency domain NB Frequency axis evenly spaced with respect to frequency amp NOT to pitch

Spectrum displaying analysis channels

LOUDNESS TRAJECTORY (loudness envelope) of a sound

at a mix

display

ALL REPRESENTATIONS ARE SCHEMATIC

to represent realistic waveforms spectra loudness trajectories but only to illustrate the principles involved in the various

Waveform spectral etc shapes have been chosen to clear as possible in diagrammatic form

JA

KEY

~tlme ~a~mp~~~~~~~~

a d

amp Wf

9

rt If tt time

time ~ IgtP

(Nsec~cent

bTII~

(Nse~

CUgtlParp

Op frq

~~

MIX PLACEMENT represents the times which sounds in begin Does NOT the whole of each sound

NB ~

We do not attempt or mix placements compositional processes described

v

make their transformations as

----

SAHPLING

afgt

1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm

_ AIt Ii I AI shown

SEQUENCE ~rwri~ as

GElflRATION

It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending

data when a key is pressed or released We may also calculate sequences of times automatically

II1I 1IIII1IJ~gmiddot Regular times

t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit

f I I I I I I I I I I I It~ Times growing exponentially

~1 I I ~ btll Times following Fibonacc i series

I I II IIF II II tmpound

Regular times slightly randomised

II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser

~

uaveforrn ~ c l)

~ ~ raquo ~ Vl

~ ~

~

Spectrum

Il IIH Il

~ H ~ Z ~

~ ltV

~ H

~ J 0 It

+

II

f-

+

a a a-

l17me domain iepre senttltiort

IF(~uen~ domain representatlonl

I()

6

-CL

~ I

0 lt) ~ ~

FreqyencJ

~

0 I

c E lts

0 () f()

Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~

re~ s ~~~

Ij

~~III ~ V1

~Ili l ~ ~ pound en( 01

S ~

~ o

~I ~

tt

t ~ Q)

~

I

o o N

o

TItlE DOHAIN amp FREQUENCY DOHAIN

Q~ Igt I II - ~ tjl II

~ ~ ~ ~ c ~ pound ~ pound ~

WAVELENGTH FREQUENCY FITCH

Time do_in Frequency Domain ~p amp

I ~ f ti~ I

~e oat

WI1Fl7Jm~r ~i-~-j~ ~~~ h

JJVV) When wavelength becomes 12 as long there are twice as many wavecycles

in the same time Frequency = wavecycles-per-second

So Halve wavelength to double frequency Double wavelength to halve frequencY4

1 l

t~ - l~

20 4 a 60 V~ IDa

A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)

nultiplying all partial frequencies by 2 (or any number)

preserves this relationship

WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE

Pitch is therefore distributed differently to frequency

Equal frequency steps

rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)

Or viewed on a scale which akes equal pitch steps look the same

Equal frequency steps i i ii WdA

I 11 1 1 1 11~~~ I I l I I Equal pitch steps

4

I ~ ~ l CIl s 0 H Q

CI2 2 ~ H ttJ

~

~~ ~4 sshy~ gtlt ~

-iT It ~ ~V E ~~

() 9shyt T

~

shy ~ 1

lt ~H ttJ

2

~ ~ o ~ 2 ~ H d () ttJ

shy t ~

2 lt l~ I l~0 11I

H ~

~

CIl ttJ

s 0 (II

(1)

luf

lt11111111111 M middotqlu-o

l 1111 I IIIII 111

~ 1lltf lt10

~1If 111 111111 1ut middotcj o

11 rr Ill Ill rll i1 ltj1lJU

-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co

3

~

~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J

S4 ~ ~e -ty~ A 01

-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1

I

(

~ ~~ ~i

_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0

~~ g +-9S 11

~j - )-- ~ i-D ( v--~1 ~ ~ Q

jllb~ f q

r Ii

Jgt

S4

~ QIf ~r f

n ~

~ ~ ~ ~

5

~

0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt

t

3 ro ~ 0 3 Q)

01 ~ C

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 2: Audible Design - Diagrammatic Appendix

1

UNIVERSITY OF ILLINOIS UBRARV

AT URBANA - CHAMP~IGr MUSIC

JJL 3~S-

J

WAVEFORH

sound represented in the time domain

When looking at much longer blocks of time waveform would be very compressed in display so a different format is used

SPECTRUM

sound represented in the frequency domain NB Frequency axis evenly spaced with respect to frequency amp NOT to pitch

Spectrum displaying analysis channels

LOUDNESS TRAJECTORY (loudness envelope) of a sound

at a mix

display

ALL REPRESENTATIONS ARE SCHEMATIC

to represent realistic waveforms spectra loudness trajectories but only to illustrate the principles involved in the various

Waveform spectral etc shapes have been chosen to clear as possible in diagrammatic form

JA

KEY

~tlme ~a~mp~~~~~~~~

a d

amp Wf

9

rt If tt time

time ~ IgtP

(Nsec~cent

bTII~

(Nse~

CUgtlParp

Op frq

~~

MIX PLACEMENT represents the times which sounds in begin Does NOT the whole of each sound

NB ~

We do not attempt or mix placements compositional processes described

v

make their transformations as

----

SAHPLING

afgt

1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm

_ AIt Ii I AI shown

SEQUENCE ~rwri~ as

GElflRATION

It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending

data when a key is pressed or released We may also calculate sequences of times automatically

II1I 1IIII1IJ~gmiddot Regular times

t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit

f I I I I I I I I I I I It~ Times growing exponentially

~1 I I ~ btll Times following Fibonacc i series

I I II IIF II II tmpound

Regular times slightly randomised

II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser

~

uaveforrn ~ c l)

~ ~ raquo ~ Vl

~ ~

~

Spectrum

Il IIH Il

~ H ~ Z ~

~ ltV

~ H

~ J 0 It

+

II

f-

+

a a a-

l17me domain iepre senttltiort

IF(~uen~ domain representatlonl

I()

6

-CL

~ I

0 lt) ~ ~

FreqyencJ

~

0 I

c E lts

0 () f()

Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~

re~ s ~~~

Ij

~~III ~ V1

~Ili l ~ ~ pound en( 01

S ~

~ o

~I ~

tt

t ~ Q)

~

I

o o N

o

TItlE DOHAIN amp FREQUENCY DOHAIN

Q~ Igt I II - ~ tjl II

~ ~ ~ ~ c ~ pound ~ pound ~

WAVELENGTH FREQUENCY FITCH

Time do_in Frequency Domain ~p amp

I ~ f ti~ I

~e oat

WI1Fl7Jm~r ~i-~-j~ ~~~ h

JJVV) When wavelength becomes 12 as long there are twice as many wavecycles

in the same time Frequency = wavecycles-per-second

So Halve wavelength to double frequency Double wavelength to halve frequencY4

1 l

t~ - l~

20 4 a 60 V~ IDa

A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)

nultiplying all partial frequencies by 2 (or any number)

preserves this relationship

WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE

Pitch is therefore distributed differently to frequency

Equal frequency steps

rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)

Or viewed on a scale which akes equal pitch steps look the same

Equal frequency steps i i ii WdA

I 11 1 1 1 11~~~ I I l I I Equal pitch steps

4

I ~ ~ l CIl s 0 H Q

CI2 2 ~ H ttJ

~

~~ ~4 sshy~ gtlt ~

-iT It ~ ~V E ~~

() 9shyt T

~

shy ~ 1

lt ~H ttJ

2

~ ~ o ~ 2 ~ H d () ttJ

shy t ~

2 lt l~ I l~0 11I

H ~

~

CIl ttJ

s 0 (II

(1)

luf

lt11111111111 M middotqlu-o

l 1111 I IIIII 111

~ 1lltf lt10

~1If 111 111111 1ut middotcj o

11 rr Ill Ill rll i1 ltj1lJU

-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co

3

~

~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J

S4 ~ ~e -ty~ A 01

-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1

I

(

~ ~~ ~i

_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0

~~ g +-9S 11

~j - )-- ~ i-D ( v--~1 ~ ~ Q

jllb~ f q

r Ii

Jgt

S4

~ QIf ~r f

n ~

~ ~ ~ ~

5

~

0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt

t

3 ro ~ 0 3 Q)

01 ~ C

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 3: Audible Design - Diagrammatic Appendix

----

SAHPLING

afgt

1JIv~v~v~vti~O~i~~~a~O~~~~~t~dw~e~~~~ is stored as a sequence of O~p individual values or samples A ~ A derived from the wavefo rm

_ AIt Ii I AI shown

SEQUENCE ~rwri~ as

GElflRATION

It is often useful to generate sequences of timing information for musical composition A HIDI instrument does this by sending

data when a key is pressed or released We may also calculate sequences of times automatically

II1I 1IIII1IJ~gmiddot Regular times

t I I I I I I I I j I ItrM~ Times growing by addition of a time-unit

f I I I I I I I I I I I It~ Times growing exponentially

~1 I I ~ btll Times following Fibonacc i series

I I II IIF II II tmpound

Regular times slightly randomised

II I I II II I IIQ~ I IIIIIII~ III ~ 11111 ~ JIIIIIII~ II tQ Times completely random Times completely random but denser

~

uaveforrn ~ c l)

~ ~ raquo ~ Vl

~ ~

~

Spectrum

Il IIH Il

~ H ~ Z ~

~ ltV

~ H

~ J 0 It

+

II

f-

+

a a a-

l17me domain iepre senttltiort

IF(~uen~ domain representatlonl

I()

6

-CL

~ I

0 lt) ~ ~

FreqyencJ

~

0 I

c E lts

0 () f()

Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~

re~ s ~~~

Ij

~~III ~ V1

~Ili l ~ ~ pound en( 01

S ~

~ o

~I ~

tt

t ~ Q)

~

I

o o N

o

TItlE DOHAIN amp FREQUENCY DOHAIN

Q~ Igt I II - ~ tjl II

~ ~ ~ ~ c ~ pound ~ pound ~

WAVELENGTH FREQUENCY FITCH

Time do_in Frequency Domain ~p amp

I ~ f ti~ I

~e oat

WI1Fl7Jm~r ~i-~-j~ ~~~ h

JJVV) When wavelength becomes 12 as long there are twice as many wavecycles

in the same time Frequency = wavecycles-per-second

So Halve wavelength to double frequency Double wavelength to halve frequencY4

1 l

t~ - l~

20 4 a 60 V~ IDa

A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)

nultiplying all partial frequencies by 2 (or any number)

preserves this relationship

WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE

Pitch is therefore distributed differently to frequency

Equal frequency steps

rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)

Or viewed on a scale which akes equal pitch steps look the same

Equal frequency steps i i ii WdA

I 11 1 1 1 11~~~ I I l I I Equal pitch steps

4

I ~ ~ l CIl s 0 H Q

CI2 2 ~ H ttJ

~

~~ ~4 sshy~ gtlt ~

-iT It ~ ~V E ~~

() 9shyt T

~

shy ~ 1

lt ~H ttJ

2

~ ~ o ~ 2 ~ H d () ttJ

shy t ~

2 lt l~ I l~0 11I

H ~

~

CIl ttJ

s 0 (II

(1)

luf

lt11111111111 M middotqlu-o

l 1111 I IIIII 111

~ 1lltf lt10

~1If 111 111111 1ut middotcj o

11 rr Ill Ill rll i1 ltj1lJU

-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co

3

~

~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J

S4 ~ ~e -ty~ A 01

-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1

I

(

~ ~~ ~i

_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0

~~ g +-9S 11

~j - )-- ~ i-D ( v--~1 ~ ~ Q

jllb~ f q

r Ii

Jgt

S4

~ QIf ~r f

n ~

~ ~ ~ ~

5

~

0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt

t

3 ro ~ 0 3 Q)

01 ~ C

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 4: Audible Design - Diagrammatic Appendix

uaveforrn ~ c l)

~ ~ raquo ~ Vl

~ ~

~

Spectrum

Il IIH Il

~ H ~ Z ~

~ ltV

~ H

~ J 0 It

+

II

f-

+

a a a-

l17me domain iepre senttltiort

IF(~uen~ domain representatlonl

I()

6

-CL

~ I

0 lt) ~ ~

FreqyencJ

~

0 I

c E lts

0 () f()

Vi ~ t ~ ~ h~ ~ ~ ~ s 0 ~V)~

re~ s ~~~

Ij

~~III ~ V1

~Ili l ~ ~ pound en( 01

S ~

~ o

~I ~

tt

t ~ Q)

~

I

o o N

o

TItlE DOHAIN amp FREQUENCY DOHAIN

Q~ Igt I II - ~ tjl II

~ ~ ~ ~ c ~ pound ~ pound ~

WAVELENGTH FREQUENCY FITCH

Time do_in Frequency Domain ~p amp

I ~ f ti~ I

~e oat

WI1Fl7Jm~r ~i-~-j~ ~~~ h

JJVV) When wavelength becomes 12 as long there are twice as many wavecycles

in the same time Frequency = wavecycles-per-second

So Halve wavelength to double frequency Double wavelength to halve frequencY4

1 l

t~ - l~

20 4 a 60 V~ IDa

A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)

nultiplying all partial frequencies by 2 (or any number)

preserves this relationship

WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE

Pitch is therefore distributed differently to frequency

Equal frequency steps

rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)

Or viewed on a scale which akes equal pitch steps look the same

Equal frequency steps i i ii WdA

I 11 1 1 1 11~~~ I I l I I Equal pitch steps

4

I ~ ~ l CIl s 0 H Q

CI2 2 ~ H ttJ

~

~~ ~4 sshy~ gtlt ~

-iT It ~ ~V E ~~

() 9shyt T

~

shy ~ 1

lt ~H ttJ

2

~ ~ o ~ 2 ~ H d () ttJ

shy t ~

2 lt l~ I l~0 11I

H ~

~

CIl ttJ

s 0 (II

(1)

luf

lt11111111111 M middotqlu-o

l 1111 I IIIII 111

~ 1lltf lt10

~1If 111 111111 1ut middotcj o

11 rr Ill Ill rll i1 ltj1lJU

-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co

3

~

~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J

S4 ~ ~e -ty~ A 01

-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1

I

(

~ ~~ ~i

_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0

~~ g +-9S 11

~j - )-- ~ i-D ( v--~1 ~ ~ Q

jllb~ f q

r Ii

Jgt

S4

~ QIf ~r f

n ~

~ ~ ~ ~

5

~

0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt

t

3 ro ~ 0 3 Q)

01 ~ C

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 5: Audible Design - Diagrammatic Appendix

WAVELENGTH FREQUENCY FITCH

Time do_in Frequency Domain ~p amp

I ~ f ti~ I

~e oat

WI1Fl7Jm~r ~i-~-j~ ~~~ h

JJVV) When wavelength becomes 12 as long there are twice as many wavecycles

in the same time Frequency = wavecycles-per-second

So Halve wavelength to double frequency Double wavelength to halve frequencY4

1 l

t~ - l~

20 4 a 60 V~ IDa

A harmonic spectrum has part ials which are exact multiples of the 1st partial (or fundaaental)

nultiplying all partial frequencies by 2 (or any number)

preserves this relationship

WHEN THE FRraquoUENCY roUBLES THE PITCH OOES UP AN OCTAVE

Pitch is therefore distributed differently to frequency

Equal frequency steps

rl I I I I I I I I I I I I II I I I I 1- I I I Equal pitch steps (eg octaves)

Or viewed on a scale which akes equal pitch steps look the same

Equal frequency steps i i ii WdA

I 11 1 1 1 11~~~ I I l I I Equal pitch steps

4

I ~ ~ l CIl s 0 H Q

CI2 2 ~ H ttJ

~

~~ ~4 sshy~ gtlt ~

-iT It ~ ~V E ~~

() 9shyt T

~

shy ~ 1

lt ~H ttJ

2

~ ~ o ~ 2 ~ H d () ttJ

shy t ~

2 lt l~ I l~0 11I

H ~

~

CIl ttJ

s 0 (II

(1)

luf

lt11111111111 M middotqlu-o

l 1111 I IIIII 111

~ 1lltf lt10

~1If 111 111111 1ut middotcj o

11 rr Ill Ill rll i1 ltj1lJU

-vlf 5 middot ~~ f l-f ~ e ~ (tbr-lt~ll -CI Sshy~ co

3

~

~-raquo s -9~~~ 4- ~ ~ ~ ~ xmiddot J

S4 ~ ~e -ty~ A 01

-0- 0~n- Fi~ ~ ~~ gt ~ ~ ~ ~1

I

(

~ ~~ ~i

_ ~ -~ ~ ~ r ~I ~ ~ ~ 3Ill 33 - 7(]I s0

~~ g +-9S 11

~j - )-- ~ i-D ( v--~1 ~ ~ Q

jllb~ f q

r Ii

Jgt

S4

~ QIf ~r f

n ~

~ ~ ~ ~

5

~

0- -11)lt 5middot 1gt11) 14nshyTo _ s ~(tgt

t

3 ro ~ 0 3 Q)

01 ~ C

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 6: Audible Design - Diagrammatic Appendix

n lt ~

~1e4-- ~~(__---l bull I i =========t

~ f

W Il4 [Il n1lli1l11111

QRlp

I T9shy rq7

HIGH PASS FILTER

lT111 IIlll II I ptttp

I

II frlt fyq

LOW PASS FILTER

fittTIlllllll) r I 1111 )fnt f~~

BAND PASS FILTER

nil INIIY[I imp

I

I f~~ fr~

BAND REJECT or NOTCH FILTER

(AntP each filter h

~tCOl Iml fof a part of

~~~~ _ 111 1 1111gtft FILTER BANK

7

cent

~ ~

0 cent

~ shy

i-

A

~(

FILTERS

lt-t

~ lt ~~------~

~

I ---1

~

( Cl

lttshy

Jl ( I ~~(-__J

d

t

~ ~

[I1 Ul H

o Z

U H

Z o ~

~ ~ H

U H

Z o

~ z

~ o z o H f ~ t-1 o gt[I1 I

~ H

f

6

---IILl

TIME ) I

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 7: Audible Design - Diagrammatic Appendix

FILTERS cont~nued

Q

The Q of a filter determines steepness of cut-off (see diagrams)

Qfgt~ T

~ fret- Band-pass f11 ter fl-ct

centred at some frequency

with low Q

0

II

I I

I I

~ 111 I I fat- ft~Band-pass filter

centred at same frequency with high Q

Low pass filter with high QLow pass filter with low Q (cut off point at same frequency)

o~ --- 1 L

[h --

f~t f~

B

ALL PASS FILTER PHASING

An all pass filter does not alter the frequency content of the spectrum However all filters change the phase of the filtered sound (see below) and different frequency ranges in the sound change their phase differently from one another Hence if we superimpose an all-pass-filtered sound on the original we will find that phase shifts in some ranges will cause partials t o be cancelled out (see

diagram ) whil st in other r anges partials will be reinfoc~d (see diagram)

-~ 2OmP

lpha~e~hft 1 af I I pha~e ~hft

)1 II ~~ieII ~ I i) 1

The Jay in which an all-pass fil ter changes the phase of different frequency ranges in a sound is an aspect of its design And it can be made to vary with time

If a sound is all-pass-filtered in a time-varying manner and the result mixed with the original sound we will hear frequency bands sweeping up or down across the sound as these bands are reinforced or cancelled This effect is often used in

popular music and is known as PHASING

PHASE

Two sounds may have the same waveform but the waveforms may begin at different points in the wavecycle (see diagram) The point in the cycle at which the waveform begins is referred to as

the phase

~cP)~ 2

When the waveform begins at the start of the sound the phase is zero If it begins later the phase is greater and has a maximum when the sound begins just before the

end of the waveform

Alternatively moving along a given waveform the phase increases (see diagram l) The rate at which the phase increases is equal to the frequency of the wave This

fact enables the Phase Vocoder to track the frequenCies of partials in a sound

9

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 8: Audible Design - Diagrammatic Appendix

FORnANTS

Formants are peaks in the spectral contour

tlfgt Ir-- j il r t-

r I(~r 1 _c~ r 1 I

Pitch may change

~(tlt r ~

without changing the formants e g sing a downward sliding pitch on a fixed vowel a

ITrCI f [h-tJ

[[11[11[ [fhl-)

lmttm am [rnlr)

Formants may change without changing the pitch

e g sing a -gt u on a fixed pitch

ITh-rlT[-

-

~

~

-

~ (i

1

3QnjJl4WV

a VI

s ~

~luIfbi

~ Vl tIl

T lt 0 ()

00 t1 tIl ~

I 1 J

~ Vi)~Jn1d--d

10 11

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 9: Audible Design - Diagrammatic Appendix

LINEAR PREDICTIVE CODING (LPC)pressure

~ r

This is a much harder problem We cannot choose just any set A-H which satisfies~ time equation 1 and expect them also to satisfy equation 2 In fact in general there will be no set A-H which satisfies both these equations

The wavef orm above has been sampled where We are therefore looking for a best fit so the vertical lines occur to give the that there is the minimum of error in the values XOXlX2 XB predicted values of both XB and Xq usingCan we predict the value of X8 from the our chosen coef fi ci ents previous 8 values If we can we c ould write down an equation Let us now define a time-window of ( say )

64 samples Can we choose coefficients A-H ( 1 ) X B Ax deg + Bx 1 + Cx 2 + + Hx 7 such that we can predict every sample in

the window from the previous 8 usingClearly by chosing appropriate values for these same coefficients A-H (the coefficients) we can always make this equation true In fact there are If we can do this we can use them to make many many ways to chose a set of A-H the signal predict itself which will all make the equation true

We now proceed to the next time-window andpressure look for a new set of coefficients (I-P) to predict the samples there in the same fashion

The coeffi cients turn out to be the ~ time numbers we need to define a filter - and

all the sets of coefficients together a time-varying filter These filters define the contour of the spect rum of our input sound fro m moment to momentvalue o f Xq from the

previous 8 values Can we predict the

using the same set of In practice we need about 40 coefficientsIf we ca n we couldcoeefficients A-H (predicting ea c h sample in our window fromwrite down an equation the previous 4 0 samples) to achieve a reasonable resolution of the spectrum ( 2 ) xq AXl + BX l + CX ) + + HX8f-

~

0gt aI r

~ r ~ lt 11 rf

~ rf Q () o Po 3 ~ n l ~i T n () Pgt 1 I III lt III III l lt l l ~O~~ l lt Pgt 0 lrf III ()10 l 0 Hn aI a tr

T ~ Polt shy aI~ lt l lt rttjH~~ co rf aoco l fJoHOTao(t ~ III aI 1 ~ rfaI co rf 0 T lZ~~

co aI aI lt 1 1 rf 3 C) ~GlHco 0 co

co (t or aI ~ Potltl 11 l()rf ~ ~

rfrf0 ~ co n rf

Po ~

1 1 rf~ trOal

J

o 1 tgtl aI 1 rftl IaI

lt IIgt I Porf1II

I C) l C)

n rf v

1 Po 1 aI T aI o ~ 0 lt bull rf3lt aI III

I T ao lt () lt Po C)I 0 ttl QI ttl tAlltl n tl rf 0

anrf II Po aI Po 1 Oal n aI 1

rfI1 rfItrn Trf

ao tl 1 co aI

1 tr aI aI rf 3 () aI ao l 1 PI PI

ltPo aorfilt ~~ tl shyaI a In T rf v ()rfrf

tr QTn 113rfaIT 3 - aI rf0 Pgt rf0 I n I nlt0rf0 11t1 Ttl T1IITrf -poundU~()1 T 1 aI 0aI III olt1Polt Po sQtlPo aI rf () trPo aI tI aI s TS s Pgt~1IItlaos rf () - Nlttl co to Nn lt 1 tl aI rf aIlt0 co

ntr aI rf trn

o Porf n T-tT

f- 1 Po () C) ~ ~

U)

I I r

II shy

I

I I~ I 1fJ

I)~ I~

0

0

10

J f

-h - _

~ ~

c

f

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 10: Audible Design - Diagrammatic Appendix

J)isfribuhor1 CJ FDrmanfs relafive to PdclL

FOrmfnt 511apc

Jat low feuro~ueVcies1 I 1 I I I I

STEPS

Formants equally Coding) ranges

have similar pitch-width If analysis channels are spaced in pitch (as they may be in Linear Predictive then formant envelopes are resolved equally well in all

LPC AND FORHANTS

14

Distribuhon

-too little data formarrt shape n~t resoveJ

~111 at low f~e~ueYdes)

~

I EQUAL PITCH STEPS I

-1 f -( rf l i_ ~-I

EQUAL FREIlUENC STEPS

Y

I

1 r f too muchdo(-Q individua( pO-rtoIs separoteo 61 chantlfls with low l-evel nosf Dhlu I I 1 IJ I

[at hiSh frequendes

1 1 1_ -f -1--1 1 I I 1 I

I I I I 1

I -shy -t- -+ I 1 I

1

Formants have similar pitch-width Hence their frequency-width increases as we go up the spectrum The phase vocoder channels are equally spaced in frequency so do not resolve the formant envelopes well in all ranges

PHASE VOCODER AND FORHANTS

15

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 11: Audible Design - Diagrammatic Appendix

CHANGING THE SPECTRUn

USUALLY CHANGES SPECTRAL CONTOUR

~p ()~~

Transpose (uplAlants) b~ syenuftinJ par-has

()fgt amp

1~ 7~

Iransfon (Sp~ct81 streIch bJ lessftta l = shrink) bj shift~ partials

pitcJ1 transpositol1 amp- spectral traniformation do not normalllj

preserve the spedYat contour

16

FORnANT PRESERVING SPECTRAL nANIPULATION

for evelj lJIiVldow in the- fre~ue1cy domain ap

speuroCtr-a1 COIItourpoundgtCtroct

(ft nEnCE formants) ~ MP~

P noro sllru~ fnJshy1shy - - -shy - -shy - -shy -

j

lshy___transform sp~drutn

0 +- il shy l 1- - --T r shy - - - - -

II I I III II ~frCJ

~ ni~fgtos~ WIOft or nal t f ~ 1~~cc1rarcontaur

I I )ofpound

FORHANT PRESERVING PITCH SHIFTING

works s1m11ar1y 17

I

II Ibull

I

j I

bull II I

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 12: Audible Design - Diagrammatic Appendix

ltD

SPECTRAL SHIFTING

~tltr

ori)inal Pltcshy

amp

I I

-r

origiTlal Sinsle pitch

a

11

ap no Shift

Shifted

~fY~LJL)~+I____~~_~ ~

2 pitches

I - - i

~)F WfSftfiD V ~ -v Ori8ihat - sound ~ ~ gt

~ - f~ - + ~ + ~ t - ~~F sect~~ ~

~hjfted - bull Sound _-shy

-~

J

I I + - shy r-shy ---- Y- ~ - - ~ -

L_ TIME gt p

~ I 1

(1) (2 ) ( 3) (4)

amp

I-rI 1+ J-I

frashy I I

Spectral shift Spectral shift of part of spectrum only_ Time-varying (increasing) spectral shift Formant-preserving spectral shift

I I ~frf-

18

SPECTRAL STRETCHING

1 I I

I

~r

~I-

Q)

bull stretch gt1

bull stretch lt 1

rl llhrn ) 11 51 ~hJ

I stretdt more attop than at bottoM

stretch more at bottom tha at -tor

~ I~~~~~~~ ~~ ~g~w~~~~-_- r w ~g ~-~ ~- shy 7 -

~ 1 I ~ I

I

MF

rlhlnlll middotmiddot (1) Spectral stretching total stretch greater than 1 (2) Total stretch less than 1 (spectral shrinking) (3) Different types of stretching (4) Time-variable (total stretch increasing) spectral stretching (5) Formant preserving spectral stretching

19

11 I I I

Ij

jbull 1

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 13: Audible Design - Diagrammatic Appendix

SPECTRAL FOCUSING

~

15 61 111 ~ ~ 2O 211 21 2JI 24 12 51 1 middot 2~ l2 i ll~ i 30b~ i

~ Iifiii Iiillli 111 iI~ iWi 1111 11111111111111

L~iil iiW(iIli n~I1 ~p~tllllin in Ifill j~C

20

i- shy --shy

~I-shy~

o J

---------t-shyt shy

o

PARTIAL TRACKING

- shy shy -~ I- shy -ishy~ t shy -- ~ --- -

Ishy - - Ishy-~ --- shy shy

~

- l- ~-

fshy

TIME

t--~ E~ E ) I ) I ~ --~iI

~

each partial regardless of which analysis channel

1 II I 4 11 [ I 2 3

1 m_ _ ~IY xtC~~ __~CI ~n~~_~C~~~d -- - -- -shy

l-I shy

~

-~

---

t

- A

gt g

it falls in

Fine-grained analysis extracts partials

1 1 - (

f I 11

I ~ ( I - I _ - - ---- -~~- - -

21

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 14: Audible Design - Diagrammatic Appendix

SPECTRAL SHAKING SPECTRAL FREEZING

orisinal sound

y 1 - -

--------

Ohjnal sound al transform 1 cr Qf transform 2 Ol1plitutle snakillj freflAe1(~ sJ 1lt

I o gt a I~~

1 centI

frt f~

~lIu I _ ---r

lpoundreeze onamplifude) ure~~f~~e~~~ shy

+shy

--I 1

11 I 1

r 11 [l1~f 11 ~ ~ f~

III II ID U ) v r

ft fro

23 22

I I qll

I

11shy

-1 +shy

l

f --t

lh

10--((111 ~ p

(JP

o gt t--t-- t-

f

I

~ r-- II

I

~ I ~ shy

I--

rshy Vf--

t

~ I-shy

1I p

1I--- r-J

--J

yr 1 1--1

7

i I v V ~ lshy

o gt

~

~ j--

r- V~ ~ t-- t--shy

J L I

I-shy v 1II ~ I ~ ~

rmtll ~

-+

+shy

+shyI I

II J~ ~ t-- I 7

I

1 I ~hy

+shy1 I I

~m ~~ Ii

-+14+ffn r

1

7 1 I

I ~ ~t

17

1 ~

[1I ~

r~ 7

l ~ ~

f- i r-- I

~ II

LJ I 1

I r- li l7 J~ h i

~ 1~ I

[ P 1 -r

If I II 1h i1V - frshy

+shy

fshy4

~

roc ~~rtr

1shy~I ~ r IV

-r -- --0

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 15: Audible Design - Diagrammatic Appendix

SPECTRAL ARPEGGIATION

Emphasize different elements of spectrum in sUccessive time-windows Emphasis-by-loudness may sweep over phase vocoder channels or composer-defined frequency ranges The sweeping motion may be in any composer-specified shape (e g sinewave ramp etc) and at any (possibly time-varying) speed

Ollata in rB_1e n1t cl1ln3ecl bull tWa i r8~E em~2ed 0 Data in ra~e zeroed ltt

0 9 8

ii4 3 2 I 0

10 9 Y 7

J 3 2shyI

0

As a6~ hutsl(ta (Oittill~01d~) frr~I f014J tt fIn tmiddot 0 if ~ ~ m

bull i bull I i7

d ~ e ~ 3 12shy middot d E 8I a middot B w ffibull LJ

Sef Inet fnc aIIS(~ fo ZHOJ unfil rphass aYri~middot[t SPufrv vlfo)~-

10 9 fl

5

7

2 ~ ~~ tI 0 0 tl tI d 24

-rUUl off frqmiddot TOJpound bfOtt t after Bmph4Sis (itttlds ~X4pt0 allo 5l(sta-~ ~J

SPECTRAL TRACING

LIJJIIrII -+ Spectral tracing for noise-reduction

OYlspna ( sound spect(aIiJi-tr(fced ~u(t

~~ II~~ II 1 I ILII 1111 Ii 1111 11111 11 t I I ft f

~I 11 I I 111-111 11111lid 11111111)11 hl lili

new ite~noqgtresentl T In prevIous Window Ill

bull frq frq

Spectral tracing creating revealed melodies as new partials enter the most-prominent set

In the above examples we retain only the 8 most prominent data items In a realistic situation we would be working with many more channels (eg 512) retaining perhaps 128 of these for noise-reduction and between 32 amp 8 to create revealed melodies 25

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 16: Audible Design - Diagrammatic Appendix

origlnal sound

I I I ~ I I I I

SPECTRAL BLURRING

selected Windows irrterpoded ffsult

cent I I - I I

Phase Vocoder windows are selected interpolated on channel-by-channel basis

Consequently some partials disappear and others emerge from nowhere (the low level noise the channels without partial s)

cent

Q~r 1I

and + a 11-

I

I

TI ftmiddot I I I~

in II - I~

I 1I 1I r Ilt ID I I

~ i Ii~~ I-

cent - I I t I~illT]

T

I I

TRACE AND BLUR

selected ltraced windows irtferpoaled result

ltf o~r lis7 II r I 1 cent

L ~ I I Igt I I i Ir shy

I I I I ~

r I f I

I

I i I I I Y

~

I

in i I 7

I I I

i~h I ) ~~j -I cent ~l

(TJI J I -+I I 17

I

I trl(Tracing retains only 4 most prominent data items in channels in the selected windows

Interpolation takes place between traced windows

27

~

l Il

SPECTRAL

orlcnal sound

-

-

J

- I

-

of

I

q

cent q

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 17: Audible Design - Diagrammatic Appendix

SPECTRAL UNDULATION

harmonic - inharmonicshy ormarrt wobbleogt tIp

Uht[ ( r[~J I -11111111111 iA fY~

tIlleTr-dl]) t11I1 II II II I)

111 II II II II I

1T lhJUhr-I-shyI11II II I I I bull ~

t11I1I I I III I bull IT[lrY(llh ~

IT1 tl-rTlTr [-~11111 II II II I)

11111 II II II I U1hrTrrlh t11I1 II 1II1 I Iflh-([llfh

o p

111111111111 l1lhdlTfhshyf-if- A

28

SPECTRAL SPLITTING

Divide spectrum into bands for e g (1) Filtering or noise removal (2) Splitting the aural image (here we add different vibrato to 2 bands)

2 3

bulll(h 1

CD

1

~ cent

~I f~

II J I shy

I

I

I J )

1

I I 1 ) )

I

11J1

29

)

)

~

I

- -t

I_gt~ YI~ATO A IVl8RATO 8

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 18: Audible Design - Diagrammatic Appendix

SPECTRAL TIHE-STRETCHING SPECTRAL TIHE-STRETCHING

Hethod 1

Q~

- ~ of1 r 011 ~

~ Ii1 ~ ~cp ~

~ II r shy

fy) ~ Method 1 Use data generated from input time-slicesto generate output time-slices which are larger(or smaller)

TItlE-WARPING

bullbull-==JbullbullC==-bullbull= orl~inal ti)1e-riime

bull gttime-ste1dtinr

bullbull=_lCgt tirne-shrnkinJ

- gt tirne-vC~inj tim( - sfretc~

Time-varying time-stretching (time-warping) is achieved __ _ Using method 1 by varying durations of output time-slices

generated from input windows Using method 2 by varying interpolation process which

generates new windows

30

Hethod 2

Original windows at CrEate inter~oi~ Wil1dgv~ fs4o1 Caws at $OP windOW-YJrit1in5j

ClJgt

IshyZ m ~ j

I

~ -IIl II 1 IQ

I ~

111 a p

1 - shy

rdSe ap algtJgt

~ fYQshy

oriainol wndowshy -samphng rare

~II 111

T ~

~ m

c) flf

Method 2 create new set of windows by interpolation Output windows eac h generate same sound-durati on a s input Windows but there are now more (or l ess) windows

ltm ~

~

l ~

~ ~L__~__~______~I~gt

I m

--shy

if-shy31

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 19: Audible Design - Diagrammatic Appendix

SPECTRAL INTERPOLATION

TIME

~ =j(l I

0

~

I~~ ~~ ~ -

n r -

~ f

~ d

r

---

~ 8L-- __ _ s

L ~

IIgt 0

i-- -shyr -sc ----J l - shy s ~~~~~ _-1shy

I VIL Lshy

~

11 o 5

VI

~ g ~

til o C s

oL

8 s ~

rshy ~

~ 0 c JI ~

~I I~~~-

I---- shy

shy8( -

+-

TYPES OF SPECTRAL INTERPOLATION

----

Different types of interpolation

a

~ i move eve I1llj

II raquo- 1OWCI frOl71 1st SoonO

l towand~ 211rJ sound

+gt i

imhal1 move slowllj awayfrorn 1st sound

the jnc~51~(1 qUiCiJy toLuaYCilt

2V1d SDuVll(

) Ihitally

f move ra~d(J sect aWllIj fl0n11Si sDlmd A hen iot4$iHjl~ rJowIlj

tDavd~ 2nd SOu d

iYliballj move V(rj ra~dll away

1 lt0

Independent interpolation

j-r

~ from 1st S DunG r tkeK ~ InCtt(lJY(jy VEry sDwJj

-0gtNmlS 211 d 5oun d

l ~ 1(1) 1) 0 )

o lt

S ll

III Ib n o

tl VI o c raquo-

VI l)

o l Q

lt0 o l ~

C) (1)

~

J ITl

j Z 11

-4

I f1

~ -I 11

of amplitude and frequency

~ I

~ I

I

-

W~ ~ I

~ I1

I

Clmpir-lJd e ~ frctutIKJ interpolafed at- II

cj fferent times

omp g free nte~laltd I or di ert1t

tim g in 11) drfampotf- I

WampIjS ~

op Qlnpg frltL Inter-polsted I ctt same tim~ b4t

in different

9~ amp irdepolatOtl beJlns ltS Defvre fYll intfrfxJafiD- ~

j) tutend5 after it ~ ~

~ I ~ ~ ~ ~ I ~

~

32 33

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 20: Audible Design - Diagrammatic Appendix

VOCODING

50Lll1dto do tt~ lIocod~ (1) sound to be vocoded (2) omp aMp

1

Frtt1(M~Jse dru~JI

~ im~5e s~6ral cOt1tour of sound 1 ~ on normalised ~Utll of ~ op

~ J

soun~~

I

frt

SPECTRAL HASKING

1I

-t r et8 il fott~e~ ~I in e8U1 anamp~s Cha~bull

~I rr-J

SPECTRAL INTERLEAVING

~ +shyI u4 ~4 ~~=J

shy

~

gtlt (j~ ltII I

(j (

C QJ

d A

x

QJ

QJ -E ~ ~~ a~ ~ E

Jl f ltl

nJ ltII I

( ~-) I ( E c -frat ~

r

I ~(

Nt

QJ

E nJ Ilt I IV E

nJ C tJ 0

c H -fr

4

is~ lt

Clt4 t

c~

)~ _ ( ( ~

~~ ---~- - ~

~~ I

I

~ t

It

~

Il II

D-_ pound-t

J

rAo~

)

I I

I-shy1J=5l

0 ~ ~

0 E cs

35 34

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 21: Audible Design - Diagrammatic Appendix

TAPE-SPEED VARIATION a

For transposition and time-stretchingtlIhr

tAAAA middot i I V [ ytlm~

rIQ~P Lengthen waves less cycles per second lower frequency

A A A A )V V V V V~I~e but also increases duration

Original spectrum and spectral contour

crman-C atlfgt I I I fltJrmant I

[- --r- -T--]---[-- r ~ -[- -t Tfrq

Pitch moves down partials move down formants move down

~lTlr[[-mh TAPE ACCELERATION I

[ I -L_- _____ J=gt edbme end~ed~~(I Mwf

Specify goal speeds I specifY goal speeds Ispecif Y end time amp end relative to source time relative to goal time speed relative to start

36

TAPE-SPEED VARIATION DIGITALLY

I I I I I I(L VI orj~yna( sound sMgtpled at onqin3r samphn~ infeYlIa( (time betweel sopes)

I I j I 1I II veOYlshyi I I I res~le ~~te~11rnrlt I bullbull 3 9 i(1L ----IJU UlLIL smpiJ

l~lll llL ~~ i - Ielt

I 111~ ) ~eFf~ ~5I I 1 11 j 1111 ] rIll J int-vI

37

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 22: Audible Design - Diagrammatic Appendix

HARl10NISER

I I

~

III r

amp--A

I

L--

Cut sound into disjoint or overlapping segments Segment length chosen so that after they have been expanded or contracted

(see below) they will rejoin into the exact duration of the original sound

~cent~ ~cent~~ Change segment pitch by tape-speed variation

which also modifies segment lengths

Rejoin changed-length segments with no overlaps or gaps They now fit exactly into the original duration

39

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 23: Audible Design - Diagrammatic Appendix

CUTTING

fAffff We cannot cut a sound simply by stopping it at an arbitrary fiMrpoint Usually this produces a sharp edge (see left) I eqUivalent to a fragment of a new waveform (see right) with many high partials In practice we hear a clickp f L normol rut i

~ a brief fade-outV 7 ~ to zero loudness

ZERO CUTTING

SPLICING

Unconventionally we may cut a sound at a zero-crossing

A normal splice is a very brief cross-fade

Unconventionally we may splice at a zero-crossing in both sounds r 14VVV V VVJ

2~ 3 ___--J

L--J 5J

6L____ ~

11 I etc

ant

I

I I

RANDOM CUTTING

time

Succe5Slve rondom cuts

SHREDDING

e

I I I I

-~~~ cut into random-let3th rorjoirrfse9ments 2 5 I 4shy 3 rearral15e in random order

[lT2r -3-shyr-4-[51

cut ere rearra~e etc

41

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 24: Audible Design - Diagrammatic Appendix

LOOPING AND ITERATION

Ilp

~~nOjjp QAll fI ~Q etc ~ ~ifijll r vv4J)T bull time

L-J IOOp~d elemet

r I f ( I te3lJla tilrlll1~

lnn]n]aAAnJ~~nllJl~~nnn~ LOOP I NGU0VifF~ U ifmiddot ~~Ulf ~ ifU If

I M II etc

I I I I I Slishttl tandomiHd tm3

~~n~~0n~~~~ nO ne~ ITERATION~lfurUvuoYu ~lfOUlf

~ ~ etc sl(nt~ ~c4ttercd pirchts

PROGRESSIVE LOOPING

~

11 ~etc v- step

This is in fact a simple

CUMp

SOUND REVERSING

Compare this with granular reversal

ZIGZAGING

Zigzag on a fixed element

reOld OIj~1 sound at or~i~aspped) ) General zigzaging

-=---~ CIS ShdwYl gt~

43

case of Brassage

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 25: Audible Design - Diagrammatic Appendix

BRASSAGE

segments must be in grain time-frame

1 to w o

OPTIONS

lt

~

B BRASSAGE (continued)

PITCH-SHIFTING OPTIONS bull

nJ~ pi(cJ

Harmoniser (only works with segments of grain dimension) or montage pitch-shifting without time-warping

Pitdl bull bull bullbull6 tm~ ~~--------~ime ~~--~~~~--~~------+~

Random pitch-shifts of segments

pitcrshy pi1th bullc t~ F~ -~ bull

Cyclical pitch-shifts of segments

Give small random delays to segments

and mix with original source

SPATIALISATION

ItLeft

I I I I I I I I

RI~ht

eg random spatialisation within a range

---------shy

[

eft

- ~~-~~~~-- RI5gth1

45

A For time-warping

1IIIIIfbullbull time~r-ink

~UI ~- cr ~ ~_tO

~ lt ~ Ul to

r 1- 0 (T 3 Ul to Ul

0 r 10 000 1 a IA to to

IA

a~ ~ ~ to

1 ~ 7 ~ IA to 3 to

0 (1)1r 00 030 1 to o 13 to 0 to ~ ~tO 3 ~ to

1 I ~a e ltD~

I I

t i

w o

VARIOUS

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 26: Audible Design - Diagrammatic Appendix

each mix element assigned a start time and a loudness

as a graphic icon on a screen

nIXING

resultant mix

Lefl(-l)

~ I t i~ It I~161 j ~ I ~ Gl 61~ 10

pa -b 2

Ri~ht (+1) c 4shy

rn~ source sound

C( 01degc

ltI~~t~_~~ each mix eleent assigned

a start tiae and a loudness on a line in a aixing score

spatialisation of mix

IN-BET~ING

Ngt~~ ~ ~ k~~

nIX SHUFFLING

Original mix Original mix

b

reve so-d 5li shy ~ d c A

h 9 J a

t i ampi ~ random scatter mix times

I I randomise sound Sfltluence

TIME WARPING lodflf5~ Original mix of ~ovnds b

add fixed CIJourn to mi)( brne S

Original mix

a_ ~ c ct ef h

Ri5nt Ar 3n lYli x I~ft t o ttght

a ~r

C E3 q ~~ (We

f )

b

etc REshy SPATIALISING

rondom-spatialise mix ~gt

a ~

ciOgt Jshy~~

~3C b

~~ e ~-- h lj

h n~

47

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 27: Audible Design - Diagrammatic Appendix

OCTAVE

Octave shift without change in duration

p(e g by spectral shifting)

for every window spectrum of original sound

spectrum of original sound

shifted octave down

etc

superimposition of the two

etc

STACKING

Octave shift with duration change

(eg by tape-speed variation)

~QP ~ r ~

till

pitch

octave up (shorter)

octave down (longer)

with non-sustained sounds the onset is enhanced in spectral brightness relative to the continuation

L_________toime

~ with sounds having ~ a sudden onset and a long attenuating ~ continuation (see right) stacking exagerates the loudness trajectory

t ime

If sounds do not have a sudden onset attack-synchronisation might be

are appropriate

ONSET SYNCHRONISATION

tape-speed varied versions d~fferent

1+-1+ jh~ ~ Qtf i +tilll~ ~bullbull ~ i e

t_ ~~ bull I

~~~ 1

49

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 28: Audible Design - Diagrammatic Appendix

I

50

TRANSPOSITION

pOOnhle 1~2

I~- I~V1 ~

I I

lwAVESET REVERSAL

r MP~W V~~W)o

II WAVESET SHAKING

VESET INVERSION

2 6

WAVESET OMISSION

VI - Y VI1-

WAVECyenCLES AND WAVESETS

WAVECYCLE Wavelength of sound where clearly pitchedWAVESET Distance from a zero-crossing to a 3rd zero-crossing

waveCllces tJ Ip-- I I I f ~

rpssuu ~ I

f I I 161 11 1 r

6me - ~I ~I _I I I i

wavesels J 1 I fit~~middot

Wlvesets I V V v V tim~ 1 I I I I

___JI 1___~I

tuavecJdes( I I i V ~~I I I

I ~

I V I -I I I v I I V I I VI

II I I I t I I It I I I I I I wa vesegt -II~ --I 1--1 --~V-l --IV -J

WAVESET SHUFFLING

blgtle IiI

I I I Iwaveseb~ 1 ~W--J

I I t I I I I I I I time 1111 II t 11 111 I

Wave~t~ --JUJI~U~IIIVL--lJULJlW

I V

I

51

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 29: Audible Design - Diagrammatic Appendix

I 1 I I I A I I

WAVESET HARHONIC DISTORTION

gtr(50S ure

~ I

1 I l 1M 1

~ ~ etc 1

I Orljrntl WIesees I

~II bull

poundPa etc I I I I12IId haoiltt

I 1I I

ffv1(1 ~ etc ~ I

13rd hucinc~u J etc -lo

each weitrteL

eneh Summed

I

I etc

WAVESET AVERAGING

prusure I

euro

Qvea~e over 3 wale~ds I

WAVESET ENVELOPING

lOUdness traiecfoj (poundnvflopt) Source sot(nd CslAddel1 ttack)5ra~va decaJ)

~ envelope 0 if sin~e tya yesets

rvvWANv~~ ~ envelope over 2 wavesets_

nrJl MhJl hJl ~D rJl rJl f+ ver 8 wavese6

no nnn n ~ 0nnOn Do 10)UU[ru OICr~ U0 U Lfu w ~

~gg4yenirj~~~wlillIiii127271222

~nnDnnnnnmJJlnnoRo0[J[f[JU LfO U [J u~LfU 0 0 ~

Am~~~~~M~tltO ~~ UPtVd t-tt~ set~

--D_D_t lJ tJ D_o~ nnll _ As pitch S~~U - u 1) ~ ~ laquoJ UVII tuaVl1t~ short~s Hence fnveofe sJloffel1s

~ 53

WAVESET DISTORTION

p ~ fV) I~QC~ WtM~

dsfogtflo~ by folole fa(jor

~ V amp OV -VZV0

C~bseJ power flcor

~v v q~AroA 4

WAVESET SUBSTITUTION

e~~(-dtft4tJ tv~~~~ ~

5Ub~btt+lQ ~ C-P~tmiddot~~J hetet

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 30: Audible Design - Diagrammatic Appendix

wavesel tim I I

WAVESET TRANSFER bull

bull

~ n2

00 Viv6V gt

~ tak =t f~

INTERLEAVING method 1

I

55

I

ETC

bull

TIHE-STRETCHING____________

~IV _ti~

tim x 2 repar eo Wdt lt2

(vj lln f 1 AW~lJuTv V V gt

timestrdcJ x 6 ~peat each wovesec x 6

bull I

TInE-SHRINKING

b1 c C1

~ it- i ~ II~ tV1 I ~ jgt V I~ d ) J J FO 1~1NJ7 V VI~ V Onl4

I

I I

wal~ets

timfshrink x3 retaif loudet of eeY~ 3 wtlllesen 0 bll c1 I el 3

I

source 1 c d

inta-~ve pairS of bJove~e~~ a bT eDT

I

method 2

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 31: Audible Design - Diagrammatic Appendix

the grains

GRANULAR REORDERING

d e f bullbullbull t t~ grains (pitch sequence is disturbed)

~ Time-shrink via grain deletion

GRANULAR REVERSAL

Source SDuhci

in the sound

Reverses the grains themselves

------~~----------~--

pir-~

r= Reorder

ptdo

Alter pitch of grains (retain original order)

Alter pitch and scatter timing of grains

TIHE-STRETCHING

Time-shrink by reducing Q bed e

57

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 32: Audible Design - Diagrammatic Appendix

ENVELOPE FOLLOWING ENVELOPING

Coarse-grained envelope following Follows overall contour ~ I

PO$Sibl~poundUrI~1 h extractedfroM~ ~ llnoiner soun(middot

~~~~~~2t~-L~~~~~------8~L tIme eg tracks rise and fall of loudness

of speakers voice

separates words or syllables

ENVELOPE SUBSTITUTION

eg in speak~s voice

to use ~bI1tUIt~~~JL-h_fwgt~ _(--J aother- sound ~ ~~~~~2Z~~~~~~~~-____ ~Ime

e g separates individual grains of a ro ~ led r ~

in speakers voice

Itle

58

Fine-grained envelope following

Follows finest detail of contour

59

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 33: Audible Design - Diagrammatic Appendix

ENVELOPE TRANSFORHATION amp SUBSTITUTIONENVELOPE TRANSFORHATION

~~-raquo

~ fi~ ---------

~ ----=-thrtSllold

~--0-~-eL )

~

l~~ ~ti~ I~-~

~N )

D~OUm

tlme )

()IP Transform enve lope e g by inversion

~)m~ SUBSTITUTE new envelope OR

on original sound

~ONI~ bt e

~o ~~e~_4J-~--4--~L~~tll~e

envelope for original sound

~iPte

-_t 60 61

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 34: Audible Design - Diagrammatic Appendix

TRIGGERING

Envelope generating triggers I a~pto be triggered a~ 1 Sound

I A bull A Ii tnreshold

e

6me I

Resulting sound

DUCKING

(backing) sound to be ducked

6 li T 0I-----fhreshold I I( K envelope of (lead) sound

to cause ducking

ducked (backing) sound

62

63

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 35: Audible Design - Diagrammatic Appendix

DELAY

Sound delayed and repeated with successive repetitions

quieter and quieter

COHB FILTERING

Very short delay time

1234618-

illfillEc --r---

number of repeats per second is very high

(eg 16+ per second) we hear a pitch

REVERBERATION

In an enclosed space sound reaching the ear directly

is mixed with many differently delayed

versions of the sound The delays are caused by the different distances the sound must travel

on its various paths to the ear The delayed versions

are (usually) quieter than the direct sound

amp have been changed in quality through the filtering effects

of reflection from different kinds of surfaces

C1gt

8~ 01

3ir (raquo

f 33 0 o s ~ ~

1lt VI lt) ~

~~ ~

lt

(t) ~ ()

(i) (T

~ ~

~D

~ cal

~i0 e 3 II

OJ

amp

~ -0shy

Q)

tLls ~ So

1

~

~~ tIlH(l tU~O O~l tIlttltJj H~ ~ H

o Z

65

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 36: Audible Design - Diagrammatic Appendix

VIBRATO

~ir-cl pirch

time~ lt=== ~t Iith natural vibrato Original pitch trajectory Slightly irregular

cyclical fluctuations of pitch

f~Vibrato created True vibrato

merely by pitch-variation retains original does not preserve formant position foraant position as pitch moves

TREHOLO

I _ lt lt lt )~me

Original loudness trajectory - -----

~ ~ ~~t~~~~i Iith (one example of) time-varying tremolo

For

_ 66 ~~shy67

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 37: Audible Design - Diagrammatic Appendix

TEXTURE

pitc ( l) r-n)a J)b ~middotmiddotJe

Jrr middotfJJ additionallymIn rp IJ1~j ~ ~

rp lJJ) pJrp ~ iJlm III)

specify variation of motif properties

1]J Tip III ~ 1]1 lJJ) rn

motif type number of motifs

time

lI Cgtab ~a bc Pklif r~Ptpoundumber

etc

WEDGEING

pif-cJ

I deJ1SJrY euroVOhbullbulltoY -~~lt

-- ~-----shy~~gy~gt~_ _~f I I bull ~~zt~ ~Zl ltE )

we~e dtuation

pitLlt1rojectorj

WEdfje WIdehshy

time

OF HOTXFS

( b Ubi t b )OR 0 u=a ~ s(rrnk =c

69

i

jl ~I

I f ~

TEXTURE

specify variation of

pitch bull event density

bull event regularity time quantisation grid number of sound sourcesbull bull hi amp 10 limits of loudness

bull hi amp 10 limits of event duration hi amp 10 limits of pitch harmonic field (or not) range of event spatial position

bull til1l

I111fQ1UttI ~ Idens ity I loudness

low pitdt range

time

pitch

hlgtt I low

~ quiet event otAd even

l(Janthed Cgtrandoltl

181f110111~random ied -V

TEXTURE OF GROUPS

additionallypitci1 -V specify variation of ~ group properties

Imiddot ~ loudness trajectory ~ hi amp 10 limits of size

( ~

hi amp 10 limits of speed I -yIV y hi amp 10 limits of pitchrange

~ N --v Y bull ~ h A --- ~ harmonic field (or not)- vv shy~ ~ - ~- spatial form (eg moving left)shy

A til-heuroshy

2-4q3-6 IqyoUP SIze

etc

68

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 38: Audible Design - Diagrammatic Appendix

11 il _ 11 I __llLJ I 11LampJ11i11

PITCH-TRACKING BY AUTO-CORRELATION

Example of an elementary auto-correlation procedure First clip the signal so local peaks remain but all other samples go to zero

tI ft)--J 11 1 ~ EL i ~ bullbull bull 1

Add the products of all pairs of samples separated by a step of N samples ~ For what value of N does this sum have its maximum value

(1) the window value] value]

are zero X zero zero must be a -ve value amp hence less than case 1

At twice the wavelength of the sound (12 samples) (1) nultiply each sample value by value 12 samples ahead within the window

(a) All these products are [+ve value] X (+ve value] [+ve value] (b) All these products are [-ve value] X [-ve value] [+ve value] (c) All other products are zero X zero = zero

(2) Add all products Only 4 +ve products in window (see above) Less than case 1

(1) nultiply each sample value by value N samples ahead within the window (a) All these products are [+ve or -ve value] X zero zero (b) All these products are zero X zero zero

(2) Add all products Sum 15 zero and therefore less than case 1

Hence sum has maximum value when step is wavelength of sound (case 1) We can thus track wavelength amp hence frequency With more complex waveforms transient subpeaks tend to cancel each other in the sum over a large window but as pitch may be changing window cant be too big Wrong-by-an-octave errors are a problem

7

4 _ N=5

l T bull -----f--I

~

At all other sample steps

PITCH-TRACKING BY PARTIAL ANALYSIS

i 11111 Ii I 111111111111 totones I quar itd-lSr dlfl veuro ua p

For each time-window transfer spectral data from the equal frequency grid of the Fourier Transform

to an equal-pitch grid spaced in quarter-tones

a -lll iii

bl bull II~II II

I rI 1 W ~~

I Iii

IIIlWmlJ ~U Ict t t t We cannot merely take the lowest peak in the spectrum as the pitch For example there may be no pitch present in the particular window

we are considering 50 that~he position of the minimum spectral peak has no pitch-tracking significance

To find a pitch effectively we must proceed as follows

For each quarter-tone channel on the pitch-grid

(1) Find the value in that channel (2) Find values in those channels which fallon the

harmonic-series template above that channel (abc) (3) Form a weighted sum of these values

nost sums of this kind will miss all or most partials in spectrum (eg a in diagram)

Templates which fallon the partials (eg b amp c in diagram) will yield significant values for the sum

The sums given by b amp c can only be differentiated if appropriate weightings are given to the contributions of each partial in

the (suspected) spectrum

Weightings for specific instruments (or other sound sources) may be well known

Difficulties arise with sources like the voice which can change their spectral contour from window to window 71

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So

Page 39: Audible Design - Diagrammatic Appendix

fliid bull (

I

seqrnent sttFshy

~-~

Ditto output times scattered

~GhdrtrE ampOn~

Pitch scatbred time

-shy b~

I OUtput denll1ty GREATER THlH 1

Segment size gt step

eg output times scattered

--- - --- shy ---shy - --shy ---shy-- -shydiagram shows time-placement of segments only

t Spatial location scattered within

H Left__e~~ng range

-shy -_-- RIht - - shy - -

73

j

+

II I I I

I

-

-

Inrvduc ed

SHEPARD TONES --------1 ~ GRANULAR RECONSTRUCTHgtN

Step is (average) distance between s~nt8 in source soundIn this example a tone of rising pitch always stays in the same tessitura output density = 1 joins segments end to end as in Brassage

Qmp

f~

pohltlh

hv mOlle-( up an ()t1tlVi) oYJdmol fa-ell is YeCreltthgtd bull

SOUND PLUCKING

or~lnal ~OIAI1d

MeJJ ~ - _ 11_---1 ~ ncteolSiJmiddot

( w(A1f~$etI~synstnctl 0 n So