convolution and correlation...how does convolution relate? •let’s say i want to make 4 copies of...
TRANSCRIPT
![Page 1: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/1.jpg)
Topic
Convolution and Correlation
Lecture by: Fatemeh Pishdadian
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 2: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/2.jpg)
Outline
• Convolution• Correlation
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 3: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/3.jpg)
Convolution• convolution is a mathematical operator which takes
two functions x and h and produces a third functionthat represents the amount of overlap between h anda reversed and translated version of x.
• In signal processing, one of the functions (h) is takento be a fixed filter impulse response, and the other (x)the input signal.
(h ∗ x )(t ) ≡ h (τ )x (t −τ )d τ−∞
∞
∫Convolution
operator
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 4: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/4.jpg)
Why do we care about convolution?
• You can digitally simulate having recorded something in a place you’ve never been by using convolution reverb.
• With a convolution reverb, you can record your voice in your bedroom and then make it sound like it was recorded in Carnegie Hall.
• The convolution operator makes convolution reverb possible (more on that next lecture).
Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 5: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/5.jpg)
Discrete Convolution• convolution is a mathematical operator which takes
two functions x and h and produces a third functionthat represents the amount of overlap between h anda reversed and translated version of x.
• In signal processing, one of the functions (h) is takento be a fixed filter impulse response, and the other (x)the input signal.
(h ∗ x )[n ]≡ h[k ]x[n − k ]k =−∞
∞
∑Convolution
operator
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 6: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/6.jpg)
Convolution In Python Code
import numpy as np
def convolution(A,B):
lengthA = np.size(A) lengthB = np.size(B)
C = np.zeros(lengthA + lengthB -1)
for m in np.arange(lengthA): for n in np.arange(lengthB):
C[m+n] = C[m+n] + A[m]*B[n]
return C
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 7: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/7.jpg)
Some things to note
• If you convolve two vectors of length n, you end up with a new vector of length 2n+1
• The algorithm on the previous slide takes n^2 time.
• Although that algorithm embodies the definition of convolution in code, there is a faster way of doing it involving the fast Fourier transform.
• We’ll get to that faster version later.
Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 8: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/8.jpg)
What is reverberation? • Say I have a recording of some audio (e.g. my voice)
that’s been turned into a digital waveform (i.e. a time-series f[n])
• I want to add echoes (aka reverberation) to it. What would I do?
• Think of an echo as a delayed and diminished copy of the original f[n].
• I would make several delayed and diminished copies of f[n] and then add them all together.
• Voila, I’ve added reverb!
Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 9: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/9.jpg)
How does convolution relate?• Let’s say I want to make 4 copies of my signal f[n],
each one delayed from the previous copy by 1 sample, and reduced in amplitude.
• Think of this as the original signal, plus three echoes.
• I can define a 2nd signal (call it g[n]) that looks like 4 impulses, each smaller than the previous one.
• Then I convolve f[n] and g[n].
• This makes 4 copies of f[n], with the kth copy being scaled and delayed by the kth element in g[n]. Then, they are all summed up.
• The result is a signal with 3 echoes added. Let’s look!Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 10: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/10.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Original signal Signal defining 4 echoes
Multiplying f[n] by the green element in g[n]
The final result, summing what we’ve done so far.
![Page 11: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/11.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Original signal Signal defining 4 echoes
Multiplying f[n] by the green element in g[n]
The final result, summing what we’ve done so far.
![Page 12: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/12.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Original signal Signal defining 4 echoes
Multiplying f[n] by the green element in g[n]
The final result, summing what we’ve done so far.
![Page 13: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/13.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Original signal Signal defining 4 echoes
Multiplying f[n] by the green element in g[n]
The final result, summing what we’ve done so far.
![Page 14: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/14.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Original signal Signal defining 4 echoes
Multiplying f[n] by the green element in g[n]
The final result, summing what we’ve done so far.
![Page 15: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/15.jpg)
Another way of looking at it.• The math of convolution is defined as flipping one of
the signals in time and then moving it across the other signal, multiplying and summing (go back and look at that Python code).
• On the next slides, you’ll see convolution worked through as describe above.
• Note that the end result is the same as what you saw in the previous demo.
Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 16: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/16.jpg)
Bryan Pardo, 2017, Northwestern University EECS 352: Machine Perception of Music and Audio
Let’s look at Convolution
![Page 17: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/17.jpg)
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
Let’s look at Convolution
![Page 18: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/18.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 19: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/19.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 20: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/20.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 21: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/21.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 22: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/22.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 23: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/23.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 24: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/24.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 25: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/25.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 26: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/26.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 27: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/27.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 28: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/28.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 29: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/29.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 30: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/30.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 31: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/31.jpg)
Let’s look at Convolution
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 32: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/32.jpg)
Cross-correlation• Cross-correlation is a measure of similarity of two
functions at time-lag t applied to one of them. It is aLOT like convolution…
(h x)(t ) ≡ h ∗(τ )x (t +τ )d τ−∞
∞
∫Cross-correlation operator
Should be a starCouldn’t find “star” in my font
Means “complex conjugate of h
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 33: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/33.jpg)
VERY Similar
• Convolution
• Cross-correlation
(h ∗ x )(t ) ≡ h (τ )x (t −τ )d τ−∞
∞
∫
(h x)(t ) ≡ h ∗(τ )x (t +τ )d τ−∞
∞
∫Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 34: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/34.jpg)
Cross-correlation in Python CodeWe can easily implement cross correlation withconvolution as follows:
Better yet, use the built in Python functions…
def crosscorrelation(A,B): return convolution(np.conj(A),B[::-1])
np.convolve(A,B,"full") # for convolutionnp.correlate(A,B,"full") # for cross correlation
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 35: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/35.jpg)
Auto-correlation• Auto-correlation is a measure of similarity of a function
to itself at time-lag t. It is a special case of cross-correlation (cross-correlation of a function with itself).
(x x)(t ) ≡ x ∗(τ )x (t +τ )d τ−∞
∞
∫Cross correlationShould be a star
Couldn’t find “star” in my font
Means “complex conjugate of f”
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 36: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/36.jpg)
Relating them all
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 37: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/37.jpg)
Convolution and Fourier transform
y(t) = h(t)* x(t)
y(t) = h(τ )x(t −τ )dτ∫Y (ω ) = H (ω )X (ω )
Convolution…
In the timedomain:
In frequencydomain:
• An important property of the Fourier transform: converts convolution in the time domain into multiplication in the frequency domain.
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio
![Page 38: Convolution and Correlation...How does convolution relate? •Let’s say I want to make 4 copies of my signal f[n], each one delayed from the previous copy by 1 sample, and reduced](https://reader035.vdocuments.mx/reader035/viewer/2022071414/610ccafd4024054acc55c4c6/html5/thumbnails/38.jpg)
Frequency selective filters
• Frequency selective filters we use in this course are a subset of LTI system.
• The output of filters will be computed via convolution in the time domain, or equivalently, via multiplication in the frequency domain.
• Can you name a real world example of convolution as summing up attenuated and delayed copies of a signal?
Bryan Pardo, 2019, Northwestern University EECS 352: Machine Perception of Music and Audio