Download - 6.003 Signal Processing - 6.003 Spring 2021
6.003 Signal Processing
Week 10, Lecture A: 2D Signal Processing (I): 2D Fourier Representation
6.003 Fall 2020
Signals: Functions Used to Convey Informationβ’ Signals may have 1 or 2 or 3 or even more independent variables.
A 1D signal has a one-dimensional domain. We usually think of it as time t or discrete time n.
A 2D signal has a two-dimensional domain. We usually think of the domains as x and y or ππ₯ and ππ¦ (or r and c).
Signals: Continuous vs. Discrete
6.003 Fall 2020
Signals from physical systems are often of continuous domain:β’ continuous time β measured in seconds, etcβ’ continuous spatial coordinates β measured in meters, cm, etc
Computations usually manipulate functions of discrete domain: β’ discrete time β measured in samplesβ’ discrete spatial coordinates β measured in pixels
Sampling
6.003 Fall 2020
(nT)
0 T
Continuous βtimeβ (CT) versus discrete βtimeβ (DT)
Important for computational manipulation of physical data. β’ digital representations of audio signals (as in MP3)β’ digital representations of images (as in JPEG)
From Time to Space
6.003 Fall 2020
Turn now to development of βfrequency domainβ representations in 2D.
So far, our signals have been a function of time: f(t), f[n]
Now, start to consider functions of space: f(x, y), f[r, c]
Our goal is still the same: β’ Extract meaningful information from a signal, β’ Manipulate information in a signal.We still resort to Fourier representations for these purposes.
Fourier RepresentationsFrom βContinuous Timeβ to βContinuous Space.β
1D Continuous-Time Fourier Transform
Synthesis equation
Analysis equationπΉ π = ΰΆ±ββ
β
π(π‘) β πβπππ‘ ππ‘
π π‘ =1
2πΰΆ±ββ
β
πΉ(π) β ππππ‘ ππ
Two dimensional CTFT:
πΉ ππ₯, ππ¦ = ΰΆ±ββ
β
ΰΆ±ββ
β
π(π₯, π¦) β πβπ(ππ₯π₯+ππ¦π¦) ππ₯ππ¦
π π₯, π¦ =1
4π2ΰΆ±ββ
β
ΰΆ±ββ
β
πΉ ππ₯, ππ¦ β ππ(ππ₯π₯+ππ¦π¦) πππ₯πππ¦
π₯ and π¦ are continuous spatial variables (units: cm, m, etc.)
ππ₯ and ππ¦ are spatial frequencies
(units: radians / length)
β’ integrals β double integrals; β’ sum of π₯ and π¦ exponents in kernal function.
Fourier RepresentationsFrom βDiscrete Timeβ to βDiscrete Space.β
1D Discrete-Time Fourier Transform
Synthesis equation
Analysis equation
Two dimensional DTFT:
πΉ Ξ©π , Ξ©π =
π=ββ
β
π=ββ
β
π[π, π] β πβπ(Ξ©ππ+Ξ©ππ)
π[π, π] =1
4π2ΰΆ±2π
ΰΆ±2π
πΉ Ξ©π , Ξ©π β ππ(Ξ©ππ+Ξ©ππ) πΞ©ππΞ©π
π and π are discrete spatial variables (units: pixels)
Ξ©π and Ξ©π are spatial frequencies (units: radians / pixel)
β’ sum β double sums; integral β double integrals; β’ sum of π and π exponents in kernal function.
π[π] =1
2πΰΆ±2π
π(Ξ©) β ππΞ©π πΞ©
πΉ Ξ© =
π=ββ
β
π[π] β πβπΞ©π
Fourier Representations1D DFT to 2D DFT
1D Discrete Fourier Transform
Synthesis equation
Analysis equation
Two dimensional DFT:
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
π[π, π] β πβπ(2ππππ π+
2ππππΆ π)
π[π, π] =
π=0
π β1
π=0
πΆβ1
πΉ[ππ , ππ] β πβπ(
2ππππ π+
2ππππΆ π)
π and π are discrete spatial variables (units: pixels)
ππ and ππ are integers representing frequencies
π[π] =
π=0
πβ1
π[π] β ππ2πππ π
πΉ[π] =1
π
π=0
πβ1
π[π] β πβπ2πππ π
OrthogonalityDFT basis functions are orthogonal to each other in 1D and 2D.
1D DFT basis functions: ππ π = πβπ2ππ
ππ
βInner productβ of 1D basis functions:
π=0
πβ1
ππβ π ππ π =
π=0
πβ1
ππ2πππ π β πβπ
2πππ π =
π=0
πβ1
ππ2π(πβπ)
π π = απ ππ π = π0 ππ‘βπππ€ππ π
2D DFT basis functions: πππ,ππ π, π = πβπ2ππππ
π πβπ2ππππΆ
π
βInner productβ of 2D basis functions:
π=0
π β1
π=0
πΆβ1
πππ,ππβ [π, π]πππ,ππ π, π =
π=0
π β1
π=0
πΆβ1
(ππ2ππππ πβ ππ
2ππππΆ π) β (πβπ
2ππππ πβ πβπ
2ππππΆ π)
=
π=0
π β1
ππ2π(ππβππ)
π π
π=0
πΆβ1
ππ2π(ππβππ)
πΆ π= α
π πΆ ππππ = ππ πππ ππ = ππ0 ππ‘βπππ€ππ π
(0 β€ π, π < π)
(0 β€ ππ, ππ < π , 0 β€ ππ, ππ < πΆ)
See slide #9 of Lec 3B.
Python Representation of 2D ImagesCalculating DFTs is most efficient in NumPy (Numerical Python).
β’ NumPy arrays are homogeneous: their elements are of the same type.
β’ NumPy operators (+, -, abs, .real, .imag) combine elements to create new arrays. e.g., (f+g)[n] is f[n]+g[n].
β’ 2D NumPy arrays can be indexed by tuples: e.g., f[r,c] = f[r][c].
β’ 2D NumPy arrays support negative indices as in lists: e.g., a[-1] = a[len(a)-1]
β’ 2D indices address row then column:
NumPy indexing is consistent with linear algebra (row first then column with rows increasing downward and columns increasing to the right). But it differs from physical mathematics (x then y with x increasing to the right and y increasing upward). You may do calculations either way, but be mindful of this difference.
Python Representation of 2D ImagesWe provide methods in the lib6003.image module for saving/loading PNG images:
β’ png_read(filename) loads an image from a file into a NumPy arrayβ’ png_write(array, filename) saves image data to a file β’ show_image(array) displays an image
arrays are indexed in row, column order, and the values are brightness, usually in the range [0, 1]
We also provide functions for computing 2D DFTs in the lib6003.fft module: fft2 and ifft2In the frequency domain, index 0, 0 into a Numpy array corresponds to ππ = 0, ππ = 0 (the DC component).
Example: Make a white square on a black background.
π
π
Check yourself!The 2D DFT basis functions have the form πππ,ππ π, π = πβπ
2ππππ
π πβπ2ππππΆ
π
π
π 0,0 is at top left corner, black correspond to lowest value, white correspond to highest value.
Check yourself!The 2D DFT basis functions have the form:
πππ,ππ π, π = πβπ2ππππ
π πβπ2ππππΆ
π
= πππ 2ππππ
π +2ππππΆ
π β ππ ππ2ππππ
π +2ππππΆ
π
If 2πππ
π π +
2πππ
πΆπ is constant, the real and imaginary parts will be constant.
Example: Let ππ =3 and ππ = β 4 when π = πΆ = 128.
Then the exponent is 2π3
128π β
2π4
128π . This exponent is zero if 3 π = 4 π.
If the exponent is zero, then cosine is at its peak value of 1.
Thus the real part of the 2D basis function is 1 along the line π =4
3π.
Therefore the real part of the 2D basis function will be 1 along the lines π = βππ
πππ.
Check yourself!The 2D DFT basis functions have the form πππ,ππ π, π = πβπ
2ππππ
π πβπ2ππππΆ
π
π
π
π π(πππ,ππ π, π ) = πππ 2ππππ
π +2ππππΆ
π
0,0 is at top left corner, black correspond to lowest value, white correspond to highest value.
Fourier Transform Pairs
In 1D, we found that it was useful to know how the transforms of simple shapes looked (for example delta β constant), in part because it was often possible to use that understanding to simplify thinking about bigger problems.
The same will be true in 2D!
The rest of today: 2D Fourier analysis of simple shapes.
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
π π, π = πΏ π πΏ π = α1, π = 0 πππ π = 00, ππ‘βπππ€ππ π
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
πΏ π πΏ π β πβπ(2ππππ π+
2ππππΆ π)
=1
π πΆπβπ(
2ππππ 0+
2ππππΆ 0)
=1
π πΆ
πΏ π πΏ ππ·πΉπ 1
π πΆ
π π = πΏ π = α1, π = 00, ππ‘βπππ€ππ π
Ξ΄[n]
0
πΉ π =1
π
π=0
πβ1
πΏ[π] β πβπ2πππ
π =1
π
1D unit sample
2D Discrete Fourier TransformGenerally, implement a 2D DFT as a sequence of 1D DFTs:
Alternatively, we can start with columns and then do rows just as well.
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
π π, π β πβπ(2ππππ π+
2ππππΆ π)
=1
π
π=0
π β11
πΆ
π=0
πΆβ1
π π, π β πβπ2ππππΆ π β πβπ
2ππππ π
first, obtain the DFT for each row
then, take the DFT of each resulting columns
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:
2D Discrete Fourier TransformExample: Find the DFT of a 2D constant.
π π, π = 1
=1
π πΆ
π=0
π β1
πβπ2ππππ π
π=0
πΆβ1
πβπ2ππππΆ π
= πΏ[ππ]πΏ[ππ]
1π·πΉπ
πΏ[ππ]πΏ[ππ]
π=0
πΆβ1
πβπ2ππππΆ π = α
πΆ, ππ= 00, ππ‘βπππ€ππ π
And we know:
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
1 β πβπ(2ππππ π+
2ππππΆ π)
πΉ[ππ , ππ] =1
π πΆβ π β πΏ[ππ] β πΆ β πΏ[ππ]
π=0
π β1
πβπ2ππππ π = α
π , ππ= 00, ππ‘βπππ€ππ π
π π = 1
πΉ π =1
π
π=0
πβ1
π[π] β πβπ2πππ π
=1
π
π=0
πβ1
πβπ2πππ π
= α1, π = 00, ππ‘βπππ€ππ π
= πΏ[π]
1D constant
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a constant.
2D Discrete Fourier TransformExample: Find the DFT of a vertical line.
π π, π = πΏ π = α1, π = 00, ππ‘βπππ€ππ π
=1
π πΆ
π=0
π β1
πβπ(2ππππ π+
2ππππΆ 0)
=1
πΆπΏ[ππ]
πΏ ππ·πΉπ 1
πΆπΏ[ππ]
=1
π πΆ
π=0
π β1
πβπ(2ππππ π)
= απ , ππ= 00, ππ‘βπππ€ππ π
π=0
π β1
πβπ2ππππ
πAnd:
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
πΏ π β πβπ(2ππππ π+
2ππππΆ π)
πΉ[ππ , ππ] =1
π πΆβ π β πΏ[ππ]
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a vertical line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
π π, π = πΏ π = α1, π = 00, ππ‘βπππ€ππ π
=1
π πΆ
π=0
πΆβ1
πβπ(2ππππ 0+
2ππππΆ π)
=1
π πΏ[ππ]
πΏ ππ·πΉπ 1
π πΏ[ππ]
=1
π πΆ
π=0
πΆβ1
πβπ(2ππππΆ π)
= απΆ, ππ= 00, ππ‘βπππ€ππ π
π=0
πΆβ1
πβπ2ππππΆ
πSince:
πΉ[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
πΏ π β πβπ(2ππππ π+
2ππππΆ π)
πΉ[ππ , ππ] =1
π πΆβ πΆ β πΏ[ππ]
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:
Circularly Translating/Shifting an ImageEffect of image translation/shifting on its Fourier transform.
Let: ππ = π β π0 πππ π , ππ = π β π0 πππ πΆ. Then:
πΉ1[ππ , ππ] =1
π πΆ
π=0
π β1
π=0
πΆβ1
π1 π, π β πβπ(2ππππ π+
2ππππΆ π)
= πβπ2ππππ
π0 β πβπ2ππππΆ
π0 β πΉ0[ππ , ππ]
π0 π, ππ·πΉπ
πΉ0[ππ , ππ]Assume that
Find the 2D DFT of π1 π, π = π0 π β π0 πππ π , π β π0 πππ πΆ
=1
π πΆ
π=0
π β1
π=0
πΆβ1
π0 π β π0 πππ π , π β π0 πππ πΆ β πβπ(2ππππ π+
2ππππΆ π)
πΉ1[ππ , ππ] =1
π πΆ
ππ=0
π β1
ππ=0
πΆβ1
π0 ππ , ππ β πβπ(2ππππ (ππ+π0 πππ π )+
2ππππΆ ( ππ+π0 πππ πΆ)
Circularly translating an image adds linear phase to its transform.
= πβπ2ππππ π0β πβπ
2ππππΆ π0
1
π πΆ
ππ=0
π β1
ππ=0
πΆβ1
π0 ππ, ππ β πβπ(2ππππ ππ+
2ππππΆ ππ)
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample: π π, π = πΏ π πΏ π
π·πΉππΉ[ππ, ππ] =
1
π πΆ
2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample. π π, π = πΏ π πΏ π β 1
π·πΉππΉ[ππ, ππ] = πβπ
2ππππΆ
β1 1
π πΆ
ππβπ
π
2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample. π π, π = πΏ π πΏ π β 2
π·πΉππΉ[ππ, ππ] = πβπ
2ππππΆ
β2 1
π πΆ
ππβπ
π
2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample. π π, π = πΏ π πΏ π β 3
π·πΉππΉ[ππ, ππ] = πβπ
2ππππΆ
β3 1
π πΆ
ππβπ
π
2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample: π π, π = πΏ π πΏ π
π·πΉππΉ[ππ, ππ] =
1
π πΆ
2D Discrete Fourier TransformExample: Find the DFT of a shifted2D unit sample: π π, π = πΏ π + 1 πΏ π
π·πΉππΉ[ππ, ππ] = πβπ
2ππππ
β(β1) 1
π πΆ
ππβπ
π
2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample: π π, π = πΏ π + 2 πΏ π
π·πΉππΉ[ππ, ππ] = πβπ
2ππππ β(β2) 1
π πΆ
ππβπ
π
2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample: π π, π = πΏ π + 3 πΏ π
π·πΉππΉ[ππ, ππ] = πβπ
2ππππ
β(β3) 1
π πΆ
ππβπ
π
Summary
6.003 Fall 2020
Introduced 2D signal processing: β’ Mostly simple extensions of 1D ideas β’ Some small differences
Introduced 2D Fourier representations: β’ Fourier kernal comprises the sum of an π₯ part and a π¦ part (or π, π)β’ Basis functions look like sinusoids turned at angles determined by the ratio of ππ to ππ.
Multiple 2D Fourier Transform pairs: β’ 2D unit sample => 2D constantβ’ 2D constant => 2D unit sampleβ’ vertical line => horizontal lineβ’ horizontal line => vertical line
(Circularly) translating an image does not change the magnitude but adds linear phase to its transform.