6.003 signal processing - 6.003 spring 2021

6.003 Signal Processing

Week 10, Lecture A: 2D Signal Processing (I): 2D Fourier Representation

6.003 Fall 2020

Signals: Functions Used to Convey Information• Signals may have 1 or 2 or 3 or even more independent variables.

A 1D signal has a one-dimensional domain. We usually think of it as time t or discrete time n.

A 2D signal has a two-dimensional domain. We usually think of the domains as x and y or 𝑛𝑥 and 𝑛𝑦 (or r and c).

Signals: Continuous vs. Discrete

6.003 Fall 2020

Signals from physical systems are often of continuous domain:• continuous time – measured in seconds, etc• continuous spatial coordinates – measured in meters, cm, etc

Computations usually manipulate functions of discrete domain: • discrete time – measured in samples• discrete spatial coordinates – measured in pixels

Sampling

6.003 Fall 2020

(nT)

0 T

Continuous “time” (CT) versus discrete “time” (DT)

Important for computational manipulation of physical data. • digital representations of audio signals (as in MP3)• digital representations of images (as in JPEG)

From Time to Space

6.003 Fall 2020

Turn now to development of “frequency domain” representations in 2D.

So far, our signals have been a function of time: f(t), f[n]

Now, start to consider functions of space: f(x, y), f[r, c]

Our goal is still the same: • Extract meaningful information from a signal, • Manipulate information in a signal.We still resort to Fourier representations for these purposes.

Fourier RepresentationsFrom “Continuous Time” to “Continuous Space.”

1D Continuous-Time Fourier Transform

Synthesis equation

Analysis equation𝐹 𝜔 = න−∞

∞

𝑓(𝑡) ∙ 𝑒−𝑗𝜔𝑡 𝑑𝑡

𝑓 𝑡 =1

2𝜋න−∞

∞

𝐹(𝜔) ∙ 𝑒𝑗𝜔𝑡 𝑑𝜔

Two dimensional CTFT:

𝐹 𝜔𝑥, 𝜔𝑦 = න−∞

∞

න−∞

∞

𝑓(𝑥, 𝑦) ∙ 𝑒−𝑗(𝜔𝑥𝑥+𝜔𝑦𝑦) 𝑑𝑥𝑑𝑦

𝑓 𝑥, 𝑦 =1

4𝜋2න−∞

∞

න−∞

∞

𝐹 𝜔𝑥, 𝜔𝑦 ∙ 𝑒𝑗(𝜔𝑥𝑥+𝜔𝑦𝑦) 𝑑𝜔𝑥𝑑𝜔𝑦

𝑥 and 𝑦 are continuous spatial variables (units: cm, m, etc.)

𝜔𝑥 and 𝜔𝑦 are spatial frequencies

(units: radians / length)

• integrals → double integrals; • sum of 𝑥 and 𝑦 exponents in kernal function.

Fourier RepresentationsFrom “Discrete Time” to “Discrete Space.”

1D Discrete-Time Fourier Transform

Synthesis equation

Analysis equation

Two dimensional DTFT:

𝐹 Ω𝑟 , Ω𝑐 =

𝑟=−∞

∞

𝑐=−∞

∞

𝑓[𝑟, 𝑐] ∙ 𝑒−𝑗(Ω𝑟𝑟+Ω𝑐𝑐)

𝑓[𝑟, 𝑐] =1

4𝜋2න2𝜋

න2𝜋

𝐹 Ω𝑟 , Ω𝑐 ∙ 𝑒𝑗(Ω𝑟𝑟+Ω𝑐𝑐) 𝑑Ω𝑟𝑑Ω𝑐

𝑟 and 𝑐 are discrete spatial variables (units: pixels)

Ω𝑟 and Ω𝑐 are spatial frequencies (units: radians / pixel)

• sum → double sums; integral → double integrals; • sum of 𝑟 and 𝑐 exponents in kernal function.

𝑓[𝑛] =1

2𝜋න2𝜋

𝑓(Ω) ∙ 𝑒𝑗Ω𝑛 𝑑Ω

𝐹 Ω =

𝑛=−∞

∞

𝑓[𝑛] ∙ 𝑒−𝑗Ω𝑛

Fourier Representations1D DFT to 2D DFT

1D Discrete Fourier Transform

Synthesis equation

Analysis equation

Two dimensional DFT:

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝑓[𝑟, 𝑐] ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+

2𝜋𝑘𝑐𝐶 𝑐)

𝑓[𝑟, 𝑐] =

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝐹[𝑘𝑟 , 𝑘𝑐] ∙ 𝑒−𝑗(

2𝜋𝑘𝑟𝑅 𝑟+


𝑟 and 𝑐 are discrete spatial variables (units: pixels)

𝑘𝑟 and 𝑘𝑐 are integers representing frequencies

𝑓[𝑛] =

𝑘=0

𝑁−1

𝑓[𝑛] ∙ 𝑒𝑗2𝜋𝑘𝑁 𝑛

𝐹[𝑘] =1

𝑁

𝑛=0

𝑁−1

𝑓[𝑛] ∙ 𝑒−𝑗2𝜋𝑘𝑁 𝑛

OrthogonalityDFT basis functions are orthogonal to each other in 1D and 2D.

1D DFT basis functions: 𝜙𝑘 𝑛 = 𝑒−𝑗2𝜋𝑘

𝑁𝑛

“Inner product” of 1D basis functions:

𝑛=0

𝑁−1

𝜙𝑘∗ 𝑛 𝜙𝑙 𝑛 =

𝑛=0

𝑁−1

𝑒𝑗2𝜋𝑘𝑁 𝑛 ∙ 𝑒−𝑗

2𝜋𝑙𝑁 𝑛 =

𝑛=0

𝑁−1

𝑒𝑗2𝜋(𝑘−𝑙)

𝑁 𝑛 = ቊ𝑁 𝑖𝑓 𝑘 = 𝑙0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

2D DFT basis functions: 𝜙𝑘𝑟,𝑘𝑐 𝑟, 𝑐 = 𝑒−𝑗2𝜋𝑘𝑟𝑅

𝑟 𝑒−𝑗2𝜋𝑘𝑐𝐶

𝑐

“Inner product” of 2D basis functions:

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝜙𝑘𝑟,𝑘𝑐∗ [𝑟, 𝑐]𝜙𝑙𝑟,𝑙𝑐 𝑟, 𝑐 =

𝑟=0

𝑅−1

𝑐=0

𝐶−1

(𝑒𝑗2𝜋𝑘𝑟𝑅 𝑟∙ 𝑒𝑗

2𝜋𝑘𝑐𝐶 𝑐) ∙ (𝑒−𝑗

2𝜋𝑙𝑟𝑅 𝑟∙ 𝑒−𝑗

2𝜋𝑙𝑐𝐶 𝑐)

=

𝑟=0

𝑅−1

𝑒𝑗2𝜋(𝑘𝑟−𝑙𝑟)

𝑅 𝑟

𝑐=0

𝐶−1

𝑒𝑗2𝜋(𝑘𝑐−𝑙𝑐)

𝐶 𝑐= ቊ

𝑅𝐶 𝑖𝑓𝑘𝑟 = 𝑙𝑟 𝑎𝑛𝑑 𝑘𝑐 = 𝑙𝑐0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

(0 ≤ 𝑘, 𝑙 < 𝑁)

(0 ≤ 𝑘𝑟, 𝑙𝑟 < 𝑅, 0 ≤ 𝑘𝑐, 𝑙𝑐 < 𝐶)

See slide #9 of Lec 3B.

Python Representation of 2D ImagesCalculating DFTs is most efficient in NumPy (Numerical Python).

• NumPy arrays are homogeneous: their elements are of the same type.

• NumPy operators (+, -, abs, .real, .imag) combine elements to create new arrays. e.g., (f+g)[n] is f[n]+g[n].

• 2D NumPy arrays can be indexed by tuples: e.g., f[r,c] = f[r][c].

• 2D NumPy arrays support negative indices as in lists: e.g., a[-1] = a[len(a)-1]

• 2D indices address row then column:

NumPy indexing is consistent with linear algebra (row first then column with rows increasing downward and columns increasing to the right). But it differs from physical mathematics (x then y with x increasing to the right and y increasing upward). You may do calculations either way, but be mindful of this difference.

Python Representation of 2D ImagesWe provide methods in the lib6003.image module for saving/loading PNG images:

• png_read(filename) loads an image from a file into a NumPy array• png_write(array, filename) saves image data to a file • show_image(array) displays an image

arrays are indexed in row, column order, and the values are brightness, usually in the range [0, 1]

We also provide functions for computing 2D DFTs in the lib6003.fft module: fft2 and ifft2In the frequency domain, index 0, 0 into a Numpy array corresponds to 𝑘𝑟 = 0, 𝑘𝑐 = 0 (the DC component).

Example: Make a white square on a black background.

𝑟

𝑐

Check yourself!The 2D DFT basis functions have the form 𝜙𝑘𝑟,𝑘𝑐 𝑟, 𝑐 = 𝑒−𝑗

2𝜋𝑘𝑟𝑅


𝑐

𝑟

𝑐 0,0 is at top left corner, black correspond to lowest value, white correspond to highest value.

Check yourself!The 2D DFT basis functions have the form:

𝜙𝑘𝑟,𝑘𝑐 𝑟, 𝑐 = 𝑒−𝑗2𝜋𝑘𝑟𝑅


𝑐

= 𝑐𝑜𝑠2𝜋𝑘𝑟𝑅

𝑟 +2𝜋𝑘𝑐𝐶

𝑐 − 𝑗𝑠𝑖𝑛2𝜋𝑘𝑟𝑅


𝑐

If 2𝜋𝑘𝑟

𝑅𝑟 +

2𝜋𝑘𝑐

𝐶𝑐 is constant, the real and imaginary parts will be constant.

Example: Let 𝑘𝑟 =3 and 𝑘𝑐 = − 4 when 𝑅 = 𝐶 = 128.

Then the exponent is 2𝜋3

128𝑟 −

2𝜋4

128𝑐 . This exponent is zero if 3 𝑟 = 4 𝑐.

If the exponent is zero, then cosine is at its peak value of 1.

Thus the real part of the 2D basis function is 1 along the line 𝑟 =4

3𝑐.

Therefore the real part of the 2D basis function will be 1 along the lines 𝑟 = −𝑘𝑐

𝑘𝑟𝑐.

Check yourself!The 2D DFT basis functions have the form 𝜙𝑘𝑟,𝑘𝑐 𝑟, 𝑐 = 𝑒−𝑗

2𝜋𝑘𝑟𝑅


𝑐

𝑟

𝑐

𝑅𝑒(𝜙𝑘𝑟,𝑘𝑐 𝑟, 𝑐 ) = 𝑐𝑜𝑠2𝜋𝑘𝑟𝑅


𝑐

0,0 is at top left corner, black correspond to lowest value, white correspond to highest value.

Fourier Transform Pairs

In 1D, we found that it was useful to know how the transforms of simple shapes looked (for example delta → constant), in part because it was often possible to use that understanding to simplify thinking about bigger problems.

The same will be true in 2D!

The rest of today: 2D Fourier analysis of simple shapes.

2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:

𝑓 𝑟, 𝑐 = 𝛿 𝑟 𝛿 𝑐 = ቊ1, 𝑟 = 0 𝑎𝑛𝑑 𝑐 = 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝛿 𝑟 𝛿 𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


=1

𝑅𝐶𝑒−𝑗(

2𝜋𝑘𝑟𝑅 0+

2𝜋𝑘𝑐𝐶 0)

=1

𝑅𝐶

𝛿 𝑟 𝛿 𝑐𝐷𝐹𝑇 1

𝑅𝐶

𝑓 𝑛 = 𝛿 𝑛 = ቊ1, 𝑛 = 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

δ[n]

0

𝐹 𝑘 =1

𝑁

𝑛=0

𝑁−1

𝛿[𝑛] ∙ 𝑒−𝑗2𝜋𝑘𝑁

𝑛 =1

𝑁

1D unit sample

2D Discrete Fourier TransformGenerally, implement a 2D DFT as a sequence of 1D DFTs:

Alternatively, we can start with columns and then do rows just as well.

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝑓 𝑟, 𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


=1

𝑅

𝑟=0

𝑅−11

𝐶

𝑐=0

𝐶−1

𝑓 𝑟, 𝑐 ∙ 𝑒−𝑗2𝜋𝑘𝑐𝐶 𝑐 ∙ 𝑒−𝑗

2𝜋𝑘𝑟𝑅 𝑟

first, obtain the DFT for each row

then, take the DFT of each resulting columns

2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample:

2D Discrete Fourier TransformExample: Find the DFT of a 2D constant.

𝑓 𝑟, 𝑐 = 1

=1

𝑅𝐶

𝑟=0

𝑅−1

𝑒−𝑗2𝜋𝑘𝑟𝑅 𝑟

𝑐=0

𝐶−1

𝑒−𝑗2𝜋𝑘𝑐𝐶 𝑐

= 𝛿[𝑘𝑐]𝛿[𝑘𝑟]

1𝐷𝐹𝑇

𝛿[𝑘𝑟]𝛿[𝑘𝑐]

𝑐=0

𝐶−1

𝑒−𝑗2𝜋𝑘𝑐𝐶 𝑐 = ቊ

𝐶, 𝑘𝑐= 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

And we know:

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

1 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶∙ 𝑅 ∙ 𝛿[𝑘𝑟] ∙ 𝐶 ∙ 𝛿[𝑘𝑐]

𝑟=0

𝑅−1

𝑒−𝑗2𝜋𝑘𝑟𝑅 𝑟 = ቊ

𝑅, 𝑘𝑟= 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝑓 𝑛 = 1

𝐹 𝑘 =1

𝑁

𝑛=0

𝑁−1

𝑓[𝑛] ∙ 𝑒−𝑗2𝜋𝑘𝑁 𝑛

=1

𝑁

𝑛=0

𝑁−1

𝑒−𝑗2𝜋𝑘𝑁 𝑛

= ቊ1, 𝑘 = 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

= 𝛿[𝑘]

1D constant

2D Discrete Fourier TransformExample: Find the DFT of a constant.

2D Discrete Fourier TransformExample: Find the DFT of a vertical line.

𝑓 𝑟, 𝑐 = 𝛿 𝑐 = ቊ1, 𝑐 = 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

=1

𝑅𝐶

𝑟=0

𝑅−1

𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+

2𝜋𝑘𝑐𝐶 0)

=1

𝐶𝛿[𝑘𝑟]

𝛿 𝑐𝐷𝐹𝑇 1

𝐶𝛿[𝑘𝑟]

=1

𝑅𝐶

𝑟=0

𝑅−1

𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟)

= ቊ𝑅, 𝑘𝑟= 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝑟=0

𝑅−1

𝑒−𝑗2𝜋𝑘𝑟𝑅

𝑟And:

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝛿 𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶∙ 𝑅 ∙ 𝛿[𝑘𝑟]

2D Discrete Fourier TransformExample: Find the DFT of a vertical line:

2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:

𝑓 𝑟, 𝑐 = 𝛿 𝑟 = ቊ1, 𝑟 = 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

=1

𝑅𝐶

𝑐=0

𝐶−1

𝑒−𝑗(2𝜋𝑘𝑟𝑅 0+


=1

𝑅𝛿[𝑘𝑐]

𝛿 𝑟𝐷𝐹𝑇 1

𝑅𝛿[𝑘𝑐]

=1

𝑅𝐶

𝑐=0

𝐶−1

𝑒−𝑗(2𝜋𝑘𝑐𝐶 𝑐)

= ቊ𝐶, 𝑘𝑐= 00, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝑐=0

𝐶−1

𝑒−𝑗2𝜋𝑘𝑐𝐶

𝑐Since:

𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝛿 𝑟 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


𝐹[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶∙ 𝐶 ∙ 𝛿[𝑘𝑐]

2D Discrete Fourier TransformExample: Find the DFT of a horizontal line:

Circularly Translating/Shifting an ImageEffect of image translation/shifting on its Fourier transform.

Let: 𝑙𝑟 = 𝑟 − 𝑟0 𝑚𝑜𝑑 𝑅, 𝑙𝑐 = 𝑐 − 𝑐0 𝑚𝑜𝑑 𝐶. Then:

𝐹1[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝑓1 𝑟, 𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


= 𝑒−𝑗2𝜋𝑘𝑟𝑅

𝑟0 ∙ 𝑒−𝑗2𝜋𝑘𝑐𝐶

𝑐0 ∙ 𝐹0[𝑘𝑟 , 𝑘𝑐]

𝑓0 𝑟, 𝑐𝐷𝐹𝑇

𝐹0[𝑘𝑟 , 𝑘𝑐]Assume that

Find the 2D DFT of 𝑓1 𝑟, 𝑐 = 𝑓0 𝑟 − 𝑟0 𝑚𝑜𝑑 𝑅, 𝑐 − 𝑐0 𝑚𝑜𝑑 𝐶

=1

𝑅𝐶

𝑟=0

𝑅−1

𝑐=0

𝐶−1

𝑓0 𝑟 − 𝑟0 𝑚𝑜𝑑 𝑅, 𝑐 − 𝑐0 𝑚𝑜𝑑 𝐶 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑟+


𝐹1[𝑘𝑟 , 𝑘𝑐] =1

𝑅𝐶

𝑙𝑟=0

𝑅−1

𝑙𝑐=0

𝐶−1

𝑓0 𝑙𝑟 , 𝑙𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 (𝑙𝑟+𝑟0 𝑚𝑜𝑑 𝑅)+

2𝜋𝑘𝑐𝐶 ( 𝑙𝑐+𝑐0 𝑚𝑜𝑑 𝐶)

Circularly translating an image adds linear phase to its transform.

= 𝑒−𝑗2𝜋𝑘𝑟𝑅 𝑟0∙ 𝑒−𝑗

2𝜋𝑘𝑐𝐶 𝑐0

1

𝑅𝐶

𝑙𝑟=0

𝑅−1

𝑙𝑐=0

𝐶−1

𝑓0 𝑙𝑟, 𝑙𝑐 ∙ 𝑒−𝑗(2𝜋𝑘𝑟𝑅 𝑙𝑟+

2𝜋𝑘𝑐𝐶 𝑙𝑐)

2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample: 𝑓 𝑟, 𝑐 = 𝛿 𝑟 𝛿 𝑐

𝐷𝐹𝑇𝐹[𝑘𝑟, 𝑘𝑐] =

1

𝑅𝐶

2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample. 𝑓 𝑟, 𝑐 = 𝛿 𝑟 𝛿 𝑐 − 1

𝐷𝐹𝑇𝐹[𝑘𝑟, 𝑘𝑐] = 𝑒−𝑗

2𝜋𝑘𝑐𝐶

∙1 1

𝑅𝐶

𝑘𝑐−𝜋

𝜋



2𝜋𝑘𝑐𝐶

∙2 1

𝑅𝐶

𝑘𝑐−𝜋

𝜋



2𝜋𝑘𝑐𝐶

∙3 1

𝑅𝐶

𝑘𝑐−𝜋

𝜋

2D Discrete Fourier TransformExample: Find the DFT of a 2D unit sample: 𝑓 𝑟, 𝑐 = 𝛿 𝑟 𝛿 𝑐

𝐷𝐹𝑇𝐹[𝑘𝑟, 𝑘𝑐] =

1

𝑅𝐶

2D Discrete Fourier TransformExample: Find the DFT of a shifted2D unit sample: 𝑓 𝑟, 𝑐 = 𝛿 𝑟 + 1 𝛿 𝑐


2𝜋𝑘𝑟𝑅

∙(−1) 1

𝑅𝐶

𝑘𝑟−𝜋

𝜋

2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample: 𝑓 𝑟, 𝑐 = 𝛿 𝑟 + 2 𝛿 𝑐


2𝜋𝑘𝑟𝑅 ∙(−2) 1

𝑅𝐶

𝑘𝑟−𝜋

𝜋

2D Discrete Fourier TransformExample: Find the DFT of a shifted 2D unit sample: 𝑓 𝑟, 𝑐 = 𝛿 𝑟 + 3 𝛿 𝑐


2𝜋𝑘𝑟𝑅

∙(−3) 1

𝑅𝐶

𝑘𝑟−𝜋

𝜋

Summary

6.003 Fall 2020

Introduced 2D signal processing: • Mostly simple extensions of 1D ideas • Some small differences

Introduced 2D Fourier representations: • Fourier kernal comprises the sum of an 𝑥 part and a 𝑦 part (or 𝑟, 𝑐)• Basis functions look like sinusoids turned at angles determined by the ratio of 𝑘𝑐 to 𝑘𝑟.

Multiple 2D Fourier Transform pairs: • 2D unit sample => 2D constant• 2D constant => 2D unit sample• vertical line => horizontal line• horizontal line => vertical line

(Circularly) translating an image does not change the magnitude but adds linear phase to its transform.

6.003 signal processing - 6.003 spring 2021

Documents