6.003 signal processing · filtering: implementing convolution with dft let 𝐹 ... what is in the...
TRANSCRIPT
-
6.003 Signal Processing
Week 8, Lecture B:System Abstraction (IV): Filtering
6.003 Fall 2020
-
System AbstractionThree complete representations for linear, time-invariant systems.
The frequency response is Fourier transform of unit-sample/impulse response!
Difference/Differential Equations: relating output with input.
Unit-Sample/Impulse Response: responses across time for an impulse input.
Frequency Response: responses across frequencies for sinusoidal inputs.
-
Frequency Response
𝑥[𝑛] 𝑦[𝑛]
𝑌 Ω = 𝐻 Ω 𝑋 Ω
The frequency response provides insights regarding how a system modifies the frequency contents of input signals.
Examples: • Design a low-pass filter with certain cut off frequency Ω𝑐• Separating harmonics using band pass filters• Perform bass-boost• … Ω𝑐Ω𝑐
Today: Focusing on the filtering aspect of the systems
-
Filtering
We can view filtering in both the time and frequency domains:
Time domain:
Frequency domain:
Consider an example: perform low-pass filtering on a song (input 𝑥 𝑛 ), we can:
• Work in time domain: Obtain the unit-sample response ℎ 𝑛 of the filter system, perform convolution 𝑥 ∗ ℎ to obtain the filtered signal 𝑦 𝑛
• Work in frequency domain: Convert 𝑥 𝑛 to frequency domain 𝑋 Ω , perform frequency domain manipulation to obtain 𝑌 Ω and convert back to 𝑦 𝑛
-
Check yourself!Consider the system described by: 𝑦 𝑛 =
𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]
3
Sketch this system’s response to the following input: 𝑥 𝑛 = 𝛿 𝑛 − 1 + 2𝛿 𝑛 − 2 + 3𝛿[𝑛 − 3]
First let us find the system’s unit-sample response:
when 𝑥 𝑛 = 𝛿 𝑛 , 𝑦 𝑛 = ℎ 𝑛
when 𝑥 𝑛 = 𝛿 𝑛 − 1 + 2𝛿 𝑛 − 2 + 3𝛿[𝑛 − 3]
𝑦 𝑛 =𝛿 𝑛 − 2 + 𝛿 𝑛 − 1 + 𝛿[𝑛]
3+ 2 ∙
𝛿 𝑛 − 3 + 𝛿 𝑛 − 2 + 𝛿[𝑛 − 1]
3+ 3 ∙
𝛿 𝑛 − 4 + 𝛿 𝑛 − 3 + 𝛿[𝑛 − 2]
3
𝑦 𝑛 =𝛿 𝑛 + 3𝛿 𝑛 − 1 + 6𝛿 𝑛 − 2 + 5𝛿 𝑛 − 3 + 3𝛿 𝑛 − 4
3
=𝛿 𝑛 − 1 + 𝛿 𝑛 + 𝛿[𝑛 + 1]
3
-
Check yourself!Consider the system described by: 𝑦 𝑛 =
𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]
3
Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛
100)
If the output signal is a scalar multiple of the input signal, we refer to the signal as an eigenfunction, and the multiplier as the eigenvalue.
Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.𝑦 𝑛 = 𝑥 ∗ ℎ 𝑛 =
𝑘=−∞
∞
𝑥[𝑘] ℎ 𝑛 − 𝑘
-
Check yourself!Consider the system described by: 𝑦 𝑛 =
𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]
3
Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛
100)
Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.
Complex exponentials are eigenfunctions of LTI systems!
𝐻(𝛺) 𝑒𝑗Ω𝑛𝑒𝑗Ω𝑛
-
Check yourself!Consider the system described by: 𝑦 𝑛 =
𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]
3
Sketch this system’s response to the following input: 𝑥 𝑛 = cos(2𝜋𝑛
100)
Now that 𝑥 𝑛 is infinite long, not easy to perform 𝑥 ∗ ℎ.
𝐻(𝛺) 𝑒𝑗Ω𝑛𝑒𝑗Ω𝑛 ℎ 𝑛 =𝛿 𝑛 − 1 + 𝛿 𝑛 + 𝛿[𝑛 + 1]
3
𝐻 Ω =
𝑛=−∞
∞
ℎ 𝑛 𝑒−𝑗Ω𝑛 =𝑒𝑗Ω + 1 + 𝑒−𝑗Ω
3=1 + 2cos(Ω)
3
𝜋−𝜋 2𝜋
3−2𝜋
30
|𝐻 Ω |
𝑦 𝑛 = 𝐻2𝜋
100cos
2𝜋𝑛
100=
1+2cos(2𝜋
100)
3cos
2𝜋𝑛
100
0 < Ω <2𝜋
3: low pass
2𝜋
3< Ω < 𝜋: signal sign flip (phase delay by )
Since input is a single frequency within 0,2𝜋
3,
with 𝐻(2𝜋
100) just a constant
-
Check yourself!Consider the moving average filter from before, characterized by:
𝑦 𝑛 =𝑥 𝑛 − 1 + 𝑥 𝑛 + 𝑥[𝑛 + 1]
3
We have found this system’s frequency response:
Does this agree with the input/output relationship we see?
3-pt averager
𝑥[𝑛] 𝑦 𝑛
𝜋−𝜋 2𝜋
3−2𝜋
30
|𝐻 Ω |
-
Design a Low Pass FilterIf instead of the 3-pt averaging, we have the following unit-sample response:
What does the frequency response look like?ℎ1 𝑛 =1
2𝛿 𝑛 +
1
4𝛿 𝑛 − 1 +
1
4𝛿[𝑛 + 1]
𝐻1 Ω =
𝑛=−∞
∞
ℎ1 𝑛 𝑒−𝑗Ω𝑛 =
1
2+1
4(𝑒−𝑗Ω + 𝑒𝑗Ω)
𝐻1 Ω =1
2+
1
2cos(Ω)
𝜋−𝜋 0
-
Design a Low Pass FilterIf 𝐻2 Ω = (𝐻1 Ω )
2= (1
2+
1
2cos Ω )2, how does the shape of 𝐻2 Ω compare with 𝐻1 Ω ?
What does ℎ2[𝑛] look like? For what values of n is ℎ2[𝑛] nonzero?
ℎ2 𝑛 = ℎ1 ∗ ℎ1 𝑛 =1
16𝛿 𝑛 + 2 +
1
4𝛿 𝑛 + 1 +
3
8𝛿 𝑛 +
1
4𝛿 𝑛 − 1 +
1
16𝛿[𝑛 − 2]
This function is nonzero for n ∈ {-2, −1, 0, 1, 2}
𝜋−𝜋 0
𝐻1 Ω
𝐻2 Ω
-
Design a Low Pass FilterHow about 𝐻𝑁 Ω = (𝐻1 Ω )
𝑁= (1
2+
1
2cos Ω )𝑁 for N=3, 10, 100, 1000?
What does ℎ𝐿[𝑛] (N=1000) look like? How many nonzero values do we expect?
Many of the samples are small, but we have 2001 non-zero samples!
𝑯𝑳 𝜴
Plotting only a small portion of values
ℎ1 ∗ ℎ1 ∗∙∙∙∗ ℎ1 ∗ ℎ1
-
Bass BoostLet 𝑥[·] be our original input, and (𝑥 ∗ ℎ𝐿)[·] represent a low-passed version of 𝑥[·].
We want to use a single convolution to produce a new signal with the high frequencies still present, but the low frequencies amplified:
Find an expression for ℎ𝐵 𝑛 .
𝑥 ∗ ℎ𝐵 𝑛 = 𝑥 𝑛 + 𝐴 𝑥 ∗ ℎ𝐿 [𝑛]
𝑋 Ω 𝐻𝐵 Ω = 𝑋 Ω + A𝑋 Ω 𝐻𝐿 Ω
𝑋 Ω 𝐻𝐵 Ω = 𝑋 Ω (1 + A𝐻𝐿 Ω )
𝐻𝐵 Ω = 1 + A𝐻𝐿 Ω
In order to find ℎ𝐵 𝑛 , we consider perform the above operation in frequency domain:
ℎ𝐵 𝑛 = 𝛿[𝑛] + Aℎ𝐿[𝑛]
ℎ𝐵 𝑛 =1
2𝜋න2𝜋
𝐻𝐵(Ω) 𝑒𝑗Ω𝑛dΩ ℎ𝐿 𝑛 =
1
2𝜋න2𝜋
𝐻𝐿(Ω) 𝑒𝑗Ω𝑛dΩ 𝛿 𝑛 =
1
2𝜋න2𝜋
1 ∙ 𝑒𝑗Ω𝑛dΩ
-
Filtering: Comparison
Time domain:
Frequency domain:
Working in time domain: determine a desired frequency response H(·), compute the unit sample response h[·] of that filter, and then convolve an input signal with h[·].
• Required a frequency response for which h[·] is reasonably straightforward to calculate. • Long unit sample responses h[·] can mean that the convolution itself can take a long time.
• Integration with continuous frequency Ω hinders computation.
Working in frequency domain: from x[·] obtain X(·), then multiply X(·) by the desired frequency response H(·) to obtain Y(·), and then convert to y[·].
𝑦[𝑛] =1
2𝜋න2𝜋
𝑋 Ω 𝐻(Ω) ∙ 𝑒𝑗Ω𝑛 𝑑Ω
𝑋 Ω =
𝑛=−∞
∞
𝑥[𝑛] ∙ 𝑒−𝑗Ω𝑛
We can use DFT when working in frequency domain!
-
Filtering: Implementing Convolution
Time domain:
Frequency domain:
Let 𝐹 Ω = 𝐹𝑎 (Ω) ∙ 𝐹𝑏 (Ω), find 𝑓[𝑛].
multiplication of Fourier transforms equivalent to convolution in the time domain
𝑓[𝑛] =1
2𝜋න2𝜋
𝐹 Ω ∙ 𝑒𝑗Ω𝑛 𝑑Ω =1
2𝜋න2𝜋
𝐹𝑎 (Ω) ∙ 𝐹𝑏 (Ω) ∙ 𝑒𝑗Ω𝑛 𝑑Ω
=1
2𝜋න2𝜋
𝐹𝑎 (Ω) ∙
𝑚=−∞
∞
𝑓𝑏[𝑚] ∙ 𝑒−𝑗Ω𝑚 ∙ 𝑒𝑗Ω𝑛 𝑑Ω =
𝑚=−∞
∞
𝑓𝑏[𝑚]1
2𝜋න2𝜋
𝐹𝑎 (Ω) ∙ 𝑒𝑗Ω(𝑛−𝑚) 𝑑Ω
=
𝑚=−∞
∞
𝑓𝑏[𝑚] 𝑓𝑎[𝑛 − 𝑚] ≡ (𝑓𝑏∗ 𝑓𝑎)[𝑛]
𝑥 ∗ ℎ 𝑛DTFT
𝐻 Ω 𝑋 Ω
-
Filtering: Implementing Convolution with DFT
Let 𝐹[𝑘] = 𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘], find 𝑓[𝑛].
The expression in the parenthesis looks like 𝑓𝑎[𝑛 − 𝑚] since
𝑓[𝑛] =
𝑘=0
𝑁−1
𝐹[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛 =
𝑘=0
𝑁−1
𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛 =
𝑘=0
𝑁−1
𝐹𝑎[𝑘] ∙1
𝑁
𝑚=0
𝑁−1
𝑓𝑏[𝑚]𝑒−𝑗
2𝜋𝑁 𝑘𝑚 𝑒𝑗
2𝜋𝑁 𝑘𝑛
=1
𝑁
𝑚=0
𝑁−1
𝑓𝑏[𝑚]
𝑘=0
𝑁−1
𝐹𝑎[𝑘] ∙ 𝑒𝑗2𝜋𝑁 𝑘(𝑛−𝑚)
𝑓𝑎 [𝑛] =
𝑘=0
𝑁−1
𝐹𝑎[𝑘]𝑒𝑗2𝜋𝑁 𝑘𝑛
But 𝑓𝑎[𝑛] was only defined for 0 ≤ 𝑛 < 𝑁, and 𝑛 −𝑚 can fall outside that range.
How should we evaluate 𝑓𝑎[𝑛] when n is not between 0 and N−1?
-
Filtering: Implementing Convolution with DFTLet 𝐹[𝑘] = 𝐹𝑎[𝑘] ∙ 𝐹𝑏[𝑘], find 𝑓[𝑛].
The expression in the parenthesis looks like 𝑓𝑎 𝑛 −𝑚 , but 𝑓𝑎[𝑛] was only defined for 0 ≤𝑛 < 𝑁, and 𝑛 −𝑚 can fall outside that range. How should we evaluate 𝑓𝑎[𝑛] when n is not between 0 and N−1?
𝑓[𝑛] =
𝑘=0
𝑁−1
𝐹[𝑘]𝑒𝑗2𝜋𝑁𝑘𝑛
=1
𝑁
𝑚=0
𝑁−1
𝑓𝑏[𝑚]
𝑘=0
𝑁−1
𝐹𝑎[𝑘] ∙ 𝑒𝑗2𝜋𝑁 𝑘(𝑛−𝑚)
What is in the parenthesis is inverse DFT, remember iDFT gives the periodically extended signal (Please refer to slide #10 of Lec05A, slide #3 of Lec 05B).
So the expression in the parenthesis equal to 𝑓𝑎𝑝 𝑛 −𝑚 , where 𝑓𝑎𝑝 𝑛 is a periodically
extended version of 𝑓𝑎 𝑛 :
=1
𝑁
𝑚=0
𝑁−1
𝑓𝑏[𝑚] 𝑓𝑎𝑝[ 𝑛 − 𝑚 ]
≡1
𝑁(𝑓𝑏 ⊛𝑓𝑎)[𝑛]
𝑓𝑎𝑝 𝑛 =
𝑖=−∞
∞
𝑓𝑎[𝑛 + 𝑖𝑁]
𝑓[𝑛] =1
𝑁
𝑚=0
𝑁−1
𝑓𝑏[𝑚] 𝑓𝑎[ 𝑛 − 𝑚 𝑚𝑜𝑑 𝑁]
=1
𝑁(𝑓𝑏 ∗ 𝑓𝑎𝑝)[𝑛]
𝑓𝑎 𝑛
𝑓𝑎𝑝 𝑛
= 𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁
Circular Convolution:
-
Circular ConvolutionLet us use an example to illustrate the difference between circular convolution & conventional convolution.
(𝑓𝑏∗ 𝑓𝑎) 𝑛 =
𝑚=−∞
∞
𝑓𝑏 𝑚 𝑓𝑎 𝑛 − 𝑚 = 𝑓𝑏[0]𝑓𝑎 𝑛 + 𝑓𝑏[1]𝑓𝑎 𝑛 − 1 + 𝑓𝑏[2]𝑓𝑎 𝑛 − 2
(𝑓𝑏⊛𝑓𝑎) 𝑛 =
𝑚=0
𝑁−1
𝑓𝑏 𝑚 𝑓𝑎 𝑛 −𝑚 𝑚𝑜𝑑 𝑁
= 𝑓𝑏[0]𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁 + 𝑓𝑏[1]𝑓𝑎 𝑛 − 1 𝑚𝑜𝑑 𝑁+ 𝑓𝑏[2]𝑓𝑎 𝑛 − 2 𝑚𝑜𝑑 𝑁
𝑓𝑏[0]𝑓𝑎 𝑛
𝑓𝑏[1]𝑓𝑎 𝑛 − 1
𝑓𝑏[2]𝑓𝑎 𝑛 − 2
(𝑓𝑏∗ 𝑓𝑎) 𝑛
(𝑓𝑏∗ 𝑓𝑎)
𝑓𝑎𝑝 𝑛
(𝑓𝑏⊛𝑓𝑎)
𝑓𝑏[0]𝑓𝑎 𝑛 𝑚𝑜𝑑 𝑁
𝑓𝑏[1]𝑓𝑎 𝑛 − 1 𝑚𝑜𝑑 𝑁
𝑓𝑏[2]𝑓𝑎 𝑛 − 2 𝑚𝑜𝑑 𝑁
(𝑓𝑏⊛𝑓𝑎) 𝑛
= (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛]
The parts of the conventional convolution that would fall outside the DFT window “alias” to points inside the DFT window.
aliasing in time!
-
Different Ways to Consider Circular Convolution
The result of circular convolution is equivalent to:
(𝑓𝑏⊛𝑓𝑎) 𝑛 =
𝑚=−∞
∞
𝑓𝑏 𝑚 𝑓𝑎 𝑛 − 𝑚 𝑚𝑜𝑑 𝑁 =
𝑚=−∞
∞
𝑓𝑏 𝑚 𝑓𝑎𝑝 𝑛 − 𝑚 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛]
𝑓𝑎𝑝 𝑛
(𝑓𝑏⊛𝑓𝑎)(𝑓𝑏∗ 𝑓𝑎)
• A periodically-extended version (periodic in N) of the result of convolving the two signals:
(𝑓𝑏⊛𝑓𝑎) 𝑛 =
𝑚=−∞
∞
(𝑓𝑏∗ 𝑓𝑎)[𝑛 − 𝑚𝑁]
• A convolution of one of the signals with a periodically-extended version of the other:
(𝑓𝑏⊛𝑓𝑎) 𝑛 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛], 𝑓𝑎𝑝 𝑛 = σ𝑚=−∞∞ 𝑓𝑎[𝑛 − 𝑚𝑁]
-
Implementing Convolution
With DTFT:
If
𝑥 𝑛𝐷𝑇𝐹𝑇
𝑋(Ω)
and
ℎ 𝑛𝐷𝑇𝐹𝑇
𝐻(Ω)
then
(𝑥 ∗ ℎ) 𝑛𝐷𝑇𝐹𝑇
𝐻(Ω)𝑋(Ω)
With DFT:
If
𝑥 𝑛𝐷𝐹𝑇
𝑋[𝑘]
and
ℎ 𝑛𝐷𝐹𝑇
𝐻[𝑘]
then
1
𝑁(𝑥 ⊛ ℎ) 𝑛
𝐷𝐹𝑇𝐻 𝑘 𝑋[𝑘]
-
SummaryWe can perform filtering either from time or frequency domain:
Time domain:
Frequency domain:
𝑥 ∗ ℎ 𝑛𝐷𝑇𝐹𝑇
𝐻 Ω 𝑋 Ω
Freq. domain multiplication correspond to time domain (circular) convolution:
1
𝑁(𝑥 ⊛ ℎ) 𝑛
𝐷𝐹𝑇𝐻 𝑘 𝑋[𝑘]
The concept of circular convolution:
(𝑓𝑏⊛𝑓𝑎) 𝑛 = (𝑓𝑏∗ 𝑓𝑎𝑝)[𝑛], 𝑓𝑎𝑝 𝑛 = σ𝑚=−∞∞ 𝑓𝑎[𝑛 − 𝑚𝑁]
(𝑓𝑏⊛𝑓𝑎) 𝑛 =
𝑚=−∞
∞
(𝑓𝑏∗ 𝑓𝑎)[𝑛 − 𝑚𝑁]