2d discrete-space signals and systemshyunjinp/notes/n-disc.pdf · 2d discrete-space signals and...

Chapter DS

2D Discrete-Space Signals and Systems

Contents

2D discrete-space signals and systems . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . DS.2

Discrete-space signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . DS.3

Classification of discrete-time signals . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . DS.5

2D discrete-space systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . DS.8

Classification of systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . DS.9

LSI systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . DS.11

2D convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . DS.12

2D discrete-space Fourier transform . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . DS.16

Properties of the DSFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . DS.17

Transforms of elementary functions . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . DS.19

Sampling revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . DS.20

Frequency-domain characteristics of LSI systems . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . DS.23

Introduction to filter design . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . DS.24

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . DS.29

(Related to Ch. 1 of Lim.)

DS.1

DS.2 c© J. Fessler, January 13, 2005, 17:44 (student version)

2D discrete-space signals and systems

Using optical devices like lenses, gratings, transparencies, etc., one can perform a very wide variety ofreal-timeanalog imageprocessing operations. However, often there are enormous benefits todigital approaches to image processing, the most importantof which is flexibility. (It is usually easier to reprogram an image processing algorithm than to change the design of an opticalsystem.)

Having addressed the issues of image formation and samplingin previous sections, we now turn to the world of discrete-space(DS) images:g[n,m] for n,m ∈ Z. A discrete-space image is definedonly on the integers. (It isnot correct to think ofg[n,m] asbeing zero for non-integer values ofn,m.)

In this chapter we will consider the middle box of the following type of imaging system:

Image formation / Sampling→ f [n,m]→ Image Processing→ g[n,m]→ Image Display

For this chapter we will continue to considercontinuous-valued(butdiscrete-space) 2D signals, saving issues of quantization forlater.

Our first goal is a brief review of 2D discrete-space signals and systems. (Essentially EECS 451 in 2D).

Overview• Kronecker impulse function• Notation and coordinate systems• Signal classes (even, odd, periodic, separable).• Circular symmetry• System classes: (linear, stable, invertible, causal, static, shift-invariant).• Impulse response• Linear shift-invariant (LSI ) systems• convolution sum, cross correlation sum, and properties.• LSI System properties in terms of impulse response• Fourier series / eigenfunctions / properties• 2D Fourier transform• 2D FT properties (convolution etc.).• Filtering

What major 1D topics are absent?

• ??• ??

This review will emphasize the similarities anddifferencesbetween the 1D and 2D formulae. This treatment serves to reinforcesignals and systems concepts learned previously, while simultaneously introducing a few new concepts and properties that areunique to 2D (and higher) problems.

References: Lim, Jain [1,2].

c© J. Fessler, January 13, 2005, 17:44 (student version) DS.3

Discrete-space signals

Unit impulse function

The 2DKronecker delta function or impulse function is defined by:

δ2[n,m] =

{

1, n = 0,m = 00, otherwise.

This really is a (discrete-space) function.

Properties:

• Unity sum:∑∞

n=−∞∑∞

m=−∞ δ2[n,m] = 1, which is analogous to the unity integral property of the Dirac impulse function.

• Sampling property: δ2[n− n0,m−m0] f [n,m] = δ2[n− n0,m−m0] f [n0,m0]

• Sifting property :∑∞

n=−∞∑∞

m=−∞ δ2[n− n0,m−m0] f [n,m] = f [n0,m0]

• Symmetry property: δ2[−n,−m] = δ2[n,m]

• Scaling property: δ2[2n, 2m] = δ2[n,m]

Note that this property is quite different from that of theDirac impulse.

• Separability property : δ2[n,m] = δ[n] δ[m], whereδ[n] is the 1D Kronecker delta function.

In the literature one often sees identical or almost identical notation for the Kronecker delta functionδ2[n,m] and the Dirac impulseδ2(x, y). The reader must determine from context which form is meant (i.e., whether the arguments are discrete or continuous).

MATLAB ’s inline function is handy for creating the Kronecker delta function:

delta = inline(’(n==0) & (m==0)’, ’n’, ’m’);

Theunit-step sequenceis given by

u2[n,m] =

{

1, n ≥ 0, m ≥ 00, otherwise

=

∞∑

k=0

∞∑

l=0

δ2[n− k,m− l] .

Note thatδ2[n,m] = (u2[n,m]−u2[n− 1,m]) (u2[n,m]−u2[n,m− 1]) ,

which is analogous to the 2D CS formulaδ2(x, y) = ∂2

∂x∂y u2(x, y).

Furthermore,δ2[n,m] = u2[n,m]−u2[n− 1,m]−u2[n,m− 1] +u2[n− 1,m− 1] .


Signal notation

Here are three equivalent methods for denoting the 2D discrete-space signal that is zero everywhere except in the 3 by 3 neighbor-hood of the origin:

f [n,m] =

{

1, |n| ≤ 1, |m| ≤ 10, otherwise

=

1∑

k=−1

1∑

l=−1

δ2[n− k,m− l] =

1 1 11 1 11 1 1

.

All three expressions are useful.

Coordinate systems

For 1D signals, by convention we almost always display negative values oft or n to the left, and positive values to the right.For 2D signals, there is more variability in how the two coordinates are used and displayed, depending on the situation.

For theoretical analysis, it is usually convenient to consider an infinite lattice coordinate system centered at the origin (0, 0), withthe first index, sayn, increasing from left to right, and the second index, saym, increasing from bottom to top, as follows:

f [n,m] =

. .....

f [−1, 1] f [0, 1] f [1, 1]

. . . f [−1, 0] f [0, 0] f [1, 0] . . .

f [−1,−1] f [0,−1] f [1,−1]...

. . .

. (DS-1)

This convention is particularly natural when we consider the valuesf [n,m] to be samples of a CS signalf(x, y), since it matchesthe usual Cartesian coordinates(x, y).

These notes will use the convention (DS-1) whenever a DS is displayed in the text. For example

0 0 12 3 00 4 0

= δ2[n− 1,m− 1] +2 δ2[n + 1,m] +3 δ2[n,m] +4 δ2[n,m + 1] .

However, when a digital image is stored in a computer, only a finite number of samples can be stored. In MATLAB and Fortran, theusual indexing for aN by M 2D array A is as follows:

A =

a11 . . . a1M

.... . .

...aN1 . . . aNM

.

The relationship between array indices and image indices isentirely arbitrary, and up to the discretion of the programmer. WheneverFFT’s are going to be used, the following relationship is often appropriate:

aij = f(i−N/2− 1, j −M/2− 1),

so that the(0, 0) value of the image is stored in theN/2 + 1,M/2 + 1 location of the array, which is just the right place so thatfftshift will relocate it to the1, 1 array position.

Unfortunately, MATLAB ’s imagesc command displays the array columns as image columns, even though the column index ismore naturally associated with thex coordinate. So it is often helpful to include atranspose operation when callingimagescin MATLAB . There are also commands likeaxis image , axis xy , andaxis ij in MATLAB to deal with the variety ofoptions for coordinate systems.

Of course in programming languages lke ANSI C the array indices begin from 0 rather than 1. And in fact it is often most efficientto store 2D images as a 1D array rather than using double indexing in C.

In summary, one must always think carefully about relationships between array indices and image coordinates when programming!


Classification of discrete-time signals

Theenergyof a 2D DS signal is defined as

Eg ,

∞∑

n=−∞

∞∑

m=−∞|g[n,m]|2 .

Theaverage powerof a 2D signal is defined as

Pg , limN→∞

1

(2N + 1)2

N∑

n=−N

N∑

m=−N

|g[n,m]|2 .

• If E is finite (E <∞) theng[n,m] is called anenergy signalandP = 0.• If E is infinite, thenP can be either finite or infinite. IfP is finite and nonzero, theng[n,m] is called apower signal.• More generally, the energy and power of the signal are computed only over the support of the signal.

Periodic signals

A 2D discrete-space signalg[n,m] is periodic with period (N,M) if and only if

g[n,m] = g[n + N,m] = g[n,m + M ], ∀n,m ∈ Z.

To find the period ofg[n,m] = cos(ωXn + ωYm + θ), writeωX/2π = k/N wherek andN are integers with no common divisors,and likewise writeωY/2π = l/M . Then(N,M) will be a period ofg[n,m]. If no such ratios exist, theng[n,m] is aperiodic.

Big differences compared to continuous-space sinusoids:• A DS sinusoidal signalg[n,m] = cos(ωXn + ωYm + θ) is periodic if and only ifωX/2π andωY/2π arerational .

In contrast, a CS sinusoidal signalcos(2π(νXx + νYy)+θ) is periodic foranyvaluesνX andνY.• Two DS sinusoidal signals with frequencies separated by integer multiples of2π are identical (indistinguishable).

For example,g[n,m] = cos(ωn) andf [n,m] = cos((ω + 2π)n + 4πm) are identical. This is a form ofaliasing.For such reasons, when considering sinusoids, we usual focus on frequencies in the interval[0, 2π) or [−π, π).

Periodic signals are power signals withPg = 1NM

∑N−1n=0

∑M−1m=0 |g[n,m]|2 .

Separability

A signalg[n,m] is separableiff we can writeg[n,m] = g1[n] g2[m] .

Example. δ2[n,m] = δ[n] δ[m]Example. u2[n,m] = u[n]u[m]

To check if a finite-support image is separable, simply verify that all rows are a multiple of the center row.

Symmetry• g[n,m] is symmetric or eveniff g[−n,−m] = g[n,m]• g[n,m] is antisymmetric or odd iff g[−n,−m] = − g[n,m]

We can decompose any signal into even and odd components:

g[n,m] = Ev {g[n,m]}+ Od {g[n,m]}

Ev {g[n,m]} ,1

2(g[n,m] + g[−n,−m])

Od {g[n,m]} ,1

2(g[n,m]− g[−n,−m])

Rotational symmetry

For DS signals, the concept ofrotational symmetry is essentially limited to the 2-, 4-, and 8-fold cases as follows:• 2-fold g[n,m] = g[−n,−m]• 4-fold g[n,m] = g[−n,−m] = g[m,−n] = g[−m,n]• 8-fold g[n,m] = g[−n,−m] = g[m,−n] = g[−m,n] = g[m,n] = g[−n,m] = g[−m,−n] = g[n,−m]

Note the preceding definition for the 8-fold case is somewhatdifferent than for the CS situation.


Circular symmetry

In continuous-space analysis, we said a signalg(x, y) hascircular symmetry iff it is a function solely of the square root of the

sum-of-squares of its arguments,i.e., g(x, y) = gR

(

√

x2 + y2)

for some functiongR : [0,∞) 7→ C.

An equivalent definition is to say thatg(x, y) hascircular symmetry iff g(x, y) is invariant to rotations, i.e., iff

g(x, y) = g(x cos θ + y sin θ,−x sin θ + y cos θ), ∀x, y, θ. (DS-2)

For discrete-space signals, there is no canonical rotationformula. In fact, MATLAB ’s imrotate command has several differentoptions (for interpolation). Furthermore, although MATLAB ’s imrotate command can “rotate” a DS image by any specifiedangle, if you rotate an image (by any angle other than a multiple of π/2) and then rotate it back, you will not get back the originalimage exactly, due to the interpolations made in the rotation process. This property can range from being a minor inconvenience tobeing a serious nuisance in various applications.

Nevertheless, it seems desirable to have some definition of circular symmetry for DS signals, since rotation invarianceis a verydesirable property in many (if not most) applications. I propose the following two definitions. I have never seen them in anytextbooks, but I do not claim that they are new. They are natural definitions, so it seems likely that someone will have thought aboutsimilar things before, but for whatever reason they have never made it into the (few) image processing books that I have seen.

Definition. I will say that a DS signalg[n,m] is weak-sense circularly symmetriciff it satisfies

g[n,m] = gR

(

√

n2 + m2)

, (DS-3)

for some functiongR(r) : [0,∞) 7→ C. Equivalently,g[n,m] corresponds to samples of some CS circularly symmetric image.

Example. Sampled-disk signals of the formg[n,m] = rect(√

n2+m2

d0

)

are weak-sense circularly symmetric.

The DS impulse signalg[n,m] = δ2[n,m] also falls into this class by choosing any0 < d0 < 2.

The reason I call this “weak sense,” is that such DS signals are not necessarily invariant to sinc-based rotations,i.e., they will not(exactly) satisfy a condition analogous to (DS-2).

Definition. I will say a DS signalg[n,m] is strong-sense circularly symmetriciff g[n,m] satisfies (DS-3) for a functiongR(r)whose Hankel transform is zero forρ > 1/2, i.e., an appropriately bandlimited 2D circularly symmetric analog image.

This definition is equivalent to requiring that if we form an analog signal fromg[n,m] by jinc-based interpolation, rotate that signalby any angleθ, and then resample the result, then we get back the originalg[n,m]; this property is analogous to (DS-2).

Given a 2D DS signalg[n,m], how can we determine whether it is strong-sense circularlysymmetric? In principle, one couldverify that the following equality holds for alln,m ∈ Z and for allθ:

g[n,m] =∞∑

k=−∞

∞∑

l=−∞g[k, l] jinc

(

√

(x− k)2 + (y − l)2)

∣

∣

∣

∣

∣x = n cos θ + m sin θy = −n sin θ + m cos θ

=∞∑

k=−∞

∞∑

l=−∞g[k, l] jinc

(

√

(n cos θ + m sin θ − k)2 + (−n sin θ + m cos θ − l)2)

.

The right-hand expression contains jinc-interpolation, continuous-space rotation, then resampling.

Example. The signalg[n,m] = jinc(√

n2 + m2)

− 14 jinc

(

12

√n2 + m2

)

is strong-sense circularly symmetric, since it corre-sponds to samples of the bandlimited circularly symmetric functionga(x, y) = jinc(r)− 1

4 jinc(r/2) with spectrumGa(νX, νY) =rect(ρ)− rect(2ρ), which has maximum frequencyρmax = 1/2.

Challenge. Isg[n,m] = δ2[n,m] strong-sense circularly symmetric?


Example. Here is an example that uses jinc-based interpolation to rotate (by 24◦) and resample two types of images: a circ function,and a jinc function.

The 3rd column shows the error images after rotation. As expected, there is much more error with the sampled circ function.(Why?) However, the error is small in both cases, interestingly. Something like samples ofgR(r) = rect

(

r−100.4

)

(a ring) wouldpresumably show more dramatically the problem with attempting to rotate samples of a non-bandlimited image.

x

y

Sampled Circ Image

−5 0 5

−5

0

5

x

y

jinc−based rotate

−5 0 5

−5

0

5

0

0.005

0.01

0.015

0.02

NRMS = 0.95%

|orig − rotate 60o|

−5 0 5

−5

0

5

x

y

bilinear−based rotate

−5 0 5

−5

0

5

0

0.005

0.01

0.015

0.02

NRMS = 2.38%


−5 0 5

−5

0

5

x

y

Sampled Jinc Image

−5 0 5

−5

0

5

x

y

jinc−based rotate

−5 0 5

−5

0

5

0

0.005

0.01

0.015

0.02

NRMS = 0.09%


−5 0 5

−5

0

5

x

y

bilinear−based rotate

−5 0 5

−5

0

5

0

0.005

0.01

0.015

0.02

NRMS = 0.34%


−5 0 5

−5

0

5

What causes the error when rotating the circ function? ??What causes the error when rotating the jinc function? ??


2D discrete-space systems

A 2D discrete-space (DS) system, or justsystem, operates on a 2D DS signal called theinput image to form a 2D DS signal calledtheoutput image.

f [n,m]→ SystemS → g[n,m] .

Notation options:

g = S[f ], g[n,m] = S{f [n,m]}, g[n,m] = (S{f})[n,m], f [n,m]S−→ g[n,m] .

Example. 2D DSmoving averageover a3× 3 window or neighborhood:

g[n,m] =1

9

n+1∑

k=n−1

m+1∑

l=m−1

f [k, l] =1

9

1∑

k=−1

1∑

l=−1

f [n− k,m− l] .

Example. Imagesegmentationbased on a simplethreshold:

g[n,m] =

{

1, f [n,m] > 100, otherwise.

Example. Thedown-samplingoperation1 (trivial form of image compression):

g[n,m] = f [2n, 2m] =

. .....

f [−2, 2] f [0, 2] f [2, 2]. . . f [−2, 0] f [0, 0] f [2, 0] . . .

f [−2,−2] f [0,−2] f [2,−2]...

. . .

. (DS-4)

Example. Theup-samplingoperation (by zero insertion):

g[n,m] =

{

f [n/2,m/2], n even andm even0, otherwise

=

.. ....

0 0 0 0 0 0 00 f [−1, 1] 0 f [0, 1] 0 f [1, 1] 00 0 0 0 0 0 0

. . . 0 f [−1, 0] 0 f [0, 0] 0 f [1, 0] 0 . . .

0 0 0 0 0 0 00 f [−1,−1] 0 f [0,−1] 0 f [1,−1] 00 0 0 0 0 0 0

..... .

. (DS-5)

1Also calledsubsamplingor sometimesdecimation, a term that originated in the Roman practice of killing every10th soldier of a defeated army [3, p. 66].


Classification of systems

We can divide system characteristics into the following twocategories.• Amplitude properties• A-1 linearity• A-2 stability• A-3 invertibility

• Spatial properties• S-1 causality• S-1’ separability• S-2 memory• S-3 shift invariance• S-4 rotation invariance

We will save linearity (the most important?) for last.

A-2 Stability• A system isbounded-input bounded-output (BIBO) stableiff every bounded input produces a bounded output.

∀f , if ∃Mf s.t.|f [n,m]| ≤Mf <∞ ∀n,m, then there must exist anMg s.t.|g[n,m]| ≤Mg <∞ ∀n,m.UsuallyMg will depend onMf .• Otherwise the system is calledunstable, and it is possible that a bounded input signal will make the output “blow up.”

Example. The moving average filter, the down-sampler, and up-sampler are all BIBOstable.

Example. Consider the following recursively defined filter:g[n,m] = g[n− 1,m− 1] + f [n,m]. This system isunstablesincethe response to a unit-step input signal is an unbounded ramp.

We will derive a simple test for BIBO stability shortly that applies to LSI systems (but not more generally).

A-3 Invertibility• A systemS is calledinvertible iff each (possible) output signal is the response to onlyoneinput signal.• OtherwiseS is not invertible.

If a systemS is invertible, then there exists a systemS−1 such that

f [n,m]→ S → g[n,m]→ S−1 → f [n,m] .

Design ofS−1 is important in many image processing applications.

Mathematically:S−1[S[f ]] = f.

Invertibility is a particularly important concept in imaging since often we would like to “undo” degrading effects likeblurring.

Example. Is the down-sampling operation invertible? ?? Is the up-sampling operation invertible? ??


S-1 Causal systems

The concept of causality is very important in 1D signal processing (when the independent variable usually corresponds to timesamples).

In 2D signal processing, the independent variables usuallycorrespond to spatial positions, not time, so causality is usually unim-portant. (Of course in “real time” systems like video, wherethe independent variables are both spaceand time, one must considercausality with respect to the time variable.)

There are some (mostly older) papers in the image processingliterature that address raster-scan based image processing methods,where the image is filtered, for example, by starting in the upper left hand corner and working lexicographically down towardsthe lower right hand corner. Such processing could be described as “causal” (whereas the moving average filter above would be“noncausal” by such a definition).

S-1’ Separable systems

A system is calledseparableif it first acts independently on each image column, and then independently on each row, or viceversa. Separable operations are collections of 1D actions,so are computationally attractive.

S-2 Memory

A system whose outputg[n,m] at any point(n,m) only depends on the input imagef [n,m] at the same location(n,m) couldbe calledmemoryless, if we stretch the English usage of “memory” somewhat. Thereare many simple, but important and useful,image processing operations that are memoryless.

Example. To compensate for nonlinearities in I/O devices like photographic film, scanners, printers, monitors, and the eye, oneoften appliesgamma correctionto an image prior to display:

g[n,m] = |f [n,m]|γ .

MATLAB ’s cmgammais related to this operation.

S-3 Shift-invariance

A systemS is calledshift invariant or space invariant iff

f [n,m]S−→ g[n,m] implies that f [n− n0,m−m0]

S−→ g[n− n0,m−m0] (DS-6)

for everyinput imagef [n,m] and shiftsn0,m0 ∈ Z. Note that only integer shifts are allowed and required!

Otherwise the system is calledshift variant or space variant.

Example. The moving average system is shift invariant. Proof:

f [n,m]S−→ g[n,m] =

1

9

1∑

k=−1

1∑

l=−1

f [n− k,m− l],

f [n− n0,m−m0]S−→ 1

9

1∑

k=−1

1∑

l=−1

f [(n− k)− n0, (m− l)−m0]

=1

9

1∑

k=−1

1∑

l=−1

f [(n− n0)− k, (m−m0)− l]

= g[n− n0,m−m0] .

Example. Is the thresholding system shift invariant? ??

Example: Are up-sampling and down-sampling shift-invariant operations? ??


S-4 Rotation invariance

We defined continuous-space systems to berotation invariant if rotating any input image caused the output image to rotateby thesame amount. This definition is inconvenient for discrete-space analysis, since “rotating” discrete-space images is nontrivial.

An alternative weaker definition for continuous-space systems would be to require that the response of the system to every circularlysymmetric input signal would be some circularly symmetric output signal. This alternative definition would more naturally extendto discrete-space systems, but it does not seem general enough.

In practical terms, what we would really like is something like the following. Consider the following image processing system.

analog inputf(x, y)

→ sampler→ digital inputf [n,m]

→ DigitalProcessing

→ digital outputg[n,m]

→ display(D/A)

→ analog outputg(x, y)

(DS-7)What we would like is to specify properties that the digital processing system must have to ensure that the overall end-to-end systemis rotationally invariant in the CS sense,i.e., rotating the analog input image causes the output image to simply rotate (except ofcourse for edge effects in a real system with finite numbers ofsamples).

We will return to this topic after discussing the 2D DSFT, where will see that certain forms of rotation invariance are possible(theoretically at least) under certain conditions.

A-3 Linear Systems

Many image processing operations are linear (but many of themore interesting operations are not). In the context of 2D discrete-space systems, we will sayS is a linear system(function) iff it is a linear operator on2 `2 (Z2), i.e., iff S satisfies

S[αf1 + βf2] = αS[f1] + βS[f2] superposition property

for all imagesf1[n,m], f2[n,m] and all real and complex constantsα, β.

Example. The moving average system above is linear.

Example. The thresholding system above is nonlinear.

Example. Are the up-sampling and down-sampling operations linear? ??

LSI systems

In 2D CS systems, we derived the generalsuperposition integral for linear systems, and then derived theconvolution integral asa special case for linearshift invariant(LSI ) systems. We did this because many 2D CS systems (e.g., mirrors) are linear but notshift invariant. Most 2D DS systems are “human made” to our specifications, rather than being intrinsically physical systems thatmust obey natural laws like Maxwell’s equations. So we can deliberately choose to design 2D DS LSI systems, and we usuallydo!Thus we proceed directly to the LSI case and derive the convolution sum.

Impulse response

For a 2D DS LSI system, leth[n,m], called theimpulse response, denote the response to a unit impulse signal:

δ2[n,m]→ S → h[n,m] .

Assuming the system is shift invariant, the response to a shifted impulse input will be simply the impulse response shifted:

δ2[n− k,m− l]→ S (SI) → h[n− k,m− l] .

Example. The impulse response of the 3 by 3 moving average filter (our only example of an LSI system thus far!) is

h[n,m] =1

9

n+1∑

k=n−1

m+1∑

l=m−1

δ2[k, l] =1

9

1∑

k=−1

1∑

l=−1

δ2[n− k,m− l] =

1/9 1/9 1/91/9 1/9 1/9

1/9 1/9 1/9

.

2`2 (Z2) denotes the vector space of square summable 2D sequences,i.e.,n

f : Z2 7→ C : P

∞

n=−∞

P

∞

m=−∞|f [n, m]|2 < ∞

o

.


2D convolution

We can use the sifting property of the impulse function to represent any 2D DS signal as a sum:

f [n,m] =

∞∑

k=−∞

∞∑

l=−∞f [k, l] δ2[n− k,m− l] .

For a linear, shift-invariant (LSI ) system, we can use thesuperposition property to see that

f [n,m] =∞∑

k=−∞

∞∑

l=−∞f [k, l] δ2[n− k,m− l]→ S LSI →

∞∑

k=−∞

∞∑

l=−∞f [k, l] h[n− k,m− l] .

Thus we have derived the following generalinput-output relationship for 2D DS LSI systems:

f [n,m]→ LSI h[n,m] → g[n,m] =

∞∑

k=−∞

∞∑

l=−∞f [k, l] h[n− k,m− l] = f [n,m] ∗∗ h[n,m],

which is called theconvolution sum.

An LSI system is completely specified by its impulse response.

2D convolution recipe• Foldh[n,m] about origin to formh[−k,−l]• Shift the folded results byn,m to formh[n− k,m− l]• Multiply h[n− k,m− l] by f [k, l]• Sum over allk,l• Repeat for everyn,m

Example.

1

2

2

4

3

6

h

0

0

0

0

0

1

0

0

0

1

0

0

0

1

0

0

0

0

0

1

f

−−> k

−−>

l

6

3

4

2

2

1

h flipped

0

0

0

0

0

0

1

2

0

0

0

3

6

0

0

0

6

12

0

0

0

5

10

1

2

0

3

6

2

4

0

0

0

3

6

g = h ** f

−−> n

−−>

m

N1 = 3

M1

=2

N2 = 5

M2

=4

N1 + N2 − 1 = 7

M1

+M

2−

1=

5


Example. Find the response of the 3 by 3 moving average system to the 2Dinput signalf [n,m] = cos(πn) cos(πm) = (−1)n+m.What does this signal look like? (Picture)The output isg[n,m] = 1

9 (−1)n+m = 19 f [n,m].

Image support

On paper we can convolve infinite sequences. A computer will only convolvefinite support sequences,i.e., images that are(implicitly) zero forn,m outside some rectangular region.

MATLAB ’s conv2 routine performs 2D DS convolution of finite-support signals.However, the user must keep track of the origin (if it is important).

Convolving aN1 ×M1 image with aN2 ×M2 image results in a(N1 + N2 − 1)× (M1 + M2 − 1) image.Sometimes this “expansion” is inconvenient, so MATLAB ’s conv2 routine includes a’same’ option that discards the “extra”pixels at the edge so that the output is the same size as one of the input arguments.

Computation and separability

The computational cost of 2D convolution of aN1 ×M1 image with aN2 ×M2 image is approximately(N1 + N2 − 1)(M1 +M2 − 1)min(N1M1, N2M2) adds/multiplies, or aboutN2M2 if N1 = M1 = N � N1 = M2 = M .

Using aseparable filter can greatly reduce computation. Ifh[n,m] = h1(n)h2(m), then we can rewrite the convolution sum asfollows:

∞∑

k=−∞

∞∑

l=−∞f [k, l] h1[n− k]h2[m− l] =

∞∑

k=−∞h1[n− k]

( ∞∑

l=−∞f [k, l] h2[m− l]

)

.

The inner sum is simply 1D convolution of each column of the image with the 1D kernelh2. The outer sum is then the 1Dconvolution of each row of that intermediate result with the1D kernelh1. For convolving a largeN × N image with a smallseparableM ×M kernel, the computational requirement is aboutN2(M + M) = 2N2M , so we save about a factor ofM/2computation. SinceM is at least 3 usually, that means at least a 33% reduction in computation, and often much more.

Example. The filterh[n,m] in the preceding example is separable. We could convolve each row with [1 2 3], then convolve eachcolumn of the result with[1 2].

To be more precise, 1D convolution (if implemented carefully) of aN -point signal with aM -point signal, withN ≥M , requires

1 + 2 + · · ·+ (M − 1) + (N −M + 1)M + (M − 1) + · · ·+ 2 + 1 = M(M − 1) + (N −M + 1)M = NM

multiplications, and nearly the same number of additions.

Edge effects

In 1D audio signal processing, usually there are enough zeros padded at the start and end of a signal that edge effects are relativelyunimportant. Furthermore, audio signals have a zero DC value, so zero padding is the natural choice when edge effects arerelevant.

Although any image stored in a computer has finite support, most images come from samples of analog images of real-world scenesthat have essentially unbounded support (like a photographof part of a mountain). When processing such images using convolution(e.g., filtering), one must think about edge effects.

MATLAB ’s conv2 routine assumes that the unavailable values outside the image are zero when computing convolution. This isonly one of several options.• zero “padding” (implicit or explicit)• toroidal extension or circular extension (2D wrap around, happens with 2D FFT)• edge value replication• mirror extension• truncating and renormalizing the filter kernel near the edges (e.g., for a 5by5 moving average, when close to an edge you can

use 5by4 or 5by3 etc. and adjust the denominator factor accordingly.) This is not “true” convolution of course, since it is aspace-varying operation.• Acquire more input samples than actually needed and discardthe output values near the edges where the (shifted) impulse

response would overlap with the unavailable samples. MATLAB ’s conv2 routine includes avalid option precisely for thispurpose. Thevalid returns an smaller image consisting only of those pixels where the shifted impulse response does notoverlap beyond the input image edges.


2D convolution sum properties

The following properties of 2D convolution are easily verified directly from the 2D convolution sum by manipulations that areessentially identical to those for the 1D convolution sum.• Commutative property:

f ∗∗ h = h ∗∗ f

• Associative property:(f ∗∗ h1) ∗∗ h2 = f ∗∗ (h1 ∗∗ h2)

• Distributive property :f ∗∗ (h1 + h2) = (f ∗∗ h1) + (f ∗∗ h2) .

• The order of serial connection of LSI systems does not affectthe overall impulse response, sinceh1 ∗∗ h2 = h2 ∗∗ h1.• f [n,m] ∗∗ δ2[n,m] = f [n,m]• Shift property:f [n,m] ∗∗ δ2[n− n0,m−m0] = f [n− n0,m−m0]• Shift-invariance:

g[n,m] = f [n,m] ∗∗ h[n,m] =⇒ f [n− l1,m− l1] ∗∗ h[n− l2,m− l2] = g[n− l1 − l2,m− l1 − l2]

• Separability property : (f1[n] f2[m]) ∗∗ (h1[n]h2[m]) = (f1[n] ∗h1[n]) · (f2[m] ∗h2[m]) .• A “scaling” property.

If g[n,m] = f [n,m] ∗∗ h[n,m], then is f [2n, 2m] ∗∗ h[2n, 2m] = 14 g[2n, 2m]? ??

LSI system properties via impulse response• causal:h[n,m] = 0 whenevern < 0 or m < 0? That would be the natural analog to the 1D case, but it is essentially irrelevant

for imaging.• memoryless (static):h[n,m] = α δ2[n,m], otherwise dynamic• BIBO stable:

∑∞n=−∞

∑∞m=−∞ |h[n,m]| <∞

• invertible: For what type of h[n,m] is an LSI system invertible?h[n,m] ∗∗ hi[n,m] = δ2[n,m] for somehi

If h[n,m] ∗∗f [n,m] = 0signal for some nonzero signalf [n,m], then the system with impulse responseh[n,m] is not invertible.

2D correlation

Thecross correlationof two 2D signalsf [n,m] andg[n,m] is defined by

rfg[k, l] ,

∞∑

n=−∞

∞∑

m=−∞f [n,m] g∗[n− k,m− l] =

∞∑

n=−∞

∞∑

m=−∞f [n + k,m + l] g∗[n,m], k, l ∈ Z,

where(k, l)is called thelag. This operation is very useful for finding features of known shape but unknown position in images.

Properties of cross correlation• Hermitian symmetry:rfg[−k,−l] = r∗fg[k, l]• rfg[n,m] = f [n,m] ∗∗ g∗[−n,−m] (convolution without the flip)• Schwarz inequality: |rfg[k, l]| ≤

√

EfEg whereEf =∑∞

n=−∞∑∞

m=−∞ |f [n,m]|2 is the signal energy.

Theautocorrelation of a signal is the cross correlation of the signal with itself:

rff [k, l] ,

∞∑

n=−∞

∞∑

m=−∞f [n,m] f∗[n− k,m− l] =

∞∑

n=−∞

∞∑

m=−∞f [n + k,m + l] f∗[n,m] .

It inherits the properties of cross correlation. In addition:• |rff [k, l]| ≤ rff [0, 0]• rff [0, 0] = Ef


Example. Here is an example application using MATLAB ’s xcorr2 function to detect a pattern of known size, orientation, andshape (square) but unknown location in a noisy image.

1 128

f1

1280

1

1 128

g=f+noise1

128−5

5

1 128

r = xcorr2(g, rect2)

1

128−0.2

1.2

1 128

g > 0.51

1280

1

1 128

r > 0.51

1280

1

Eigenfunctions of 2D DS systems

2D DS complex exponential signals are eigenfunctions of 2D DS LSI systems, even if they are aperiodic!

f [n,m] = eı(ωXn+ωYm) → LSI h[n,m] → g[n,m] =

∞∑

k=−∞

∞∑

l=−∞h[k, l] f [n− k,m− l]

=

∞∑

k=−∞

∞∑

l=−∞h[k, l] eı[ωX(n−k)+ωY(m−l)]

= eı(ωXn+ωYm) H(ωX, ωY)

where the 2D DSFT ofh[n,m] (akafrequency response) is given by:

H(ωX, ωY) =

∞∑

n=−∞

∞∑

m=−∞h[n,m] e−ı(ωXn+ωYm) .


2D discrete-space Fourier transform

The2D discrete-space Fourier transform(DSFT) of a 2D discrete-space signalg[n,m] is defined as follows:

G(ωX, ωY) =∞∑

n=−∞

∞∑

m=−∞g[n,m] e−ı(ωXn+ωYm) (analysis).

(This is the 2D space domain analog of the DTFT).The inverse 2D DSFTis given by

g[n,m] =1

(2π)2

∫ π

−π

∫ π

−π

G(ωX, ωY) eı(ωXn+ωYm) dωX dωY (synthesis).

Proof of the inverse 2D DSFT relationship:

1

(2π)2

∫ π

−π

∫ π

−π

G(ωX, ωY) eı(ωXn+ωYm) dωX dωY

=1

(2π)2

∫ π

−π

∫ π

−π

[ ∞∑

k=−∞

∞∑

l=−∞g[k, l] e−ı(ωXk+ωYl)

]

eı(ωXn+ωYm) dωX dωY

=∞∑

k=−∞

∞∑

l=−∞g[k, l]

[

1

(2π)2

∫ π

−π

∫ π

−π

eı[ωX(n−k)+ωY(m−l)] dωX dωY

]

=

∞∑

k=−∞

∞∑

l=−∞g[k, l] δ2[n− k,m− l] = g[n,m] .

The proof is valid,i.e., the Fourier transform exists and the synthesis formula holds, if g[n,m] has finite energy or is absolutelysummable,i.e.,

∑∞n=−∞

∑∞m=−∞ |g[n,m]| <∞.

Example. The frequency response of the (separable!) moving averagefilter is given by

H(ωX, ωY) =∞∑

n=−∞

∞∑

m=−∞h[n,m] e−ı(ωXn+ωYm) =

1

9

1∑

n=−1

1∑

m=−1

e−ıωXn e−ıωYm =1

9[1 + 2 cos ωX][1 + 2 cos ωY].

Convergence

Any time one encounters infinite sums or integrals, the subject of convergenceshould be considered.

We say the FT ofg[n,m] converges (pointwise)iff G(ωX, ωY) is everywhere finite and

limN→∞

N∑

n=−N

N∑

m=−N

g[n,m] e−ı(ωXn+ωYm) = G(ωX, ωY)

for all ωX andωY.

Sufficient condition: ifg[n,m] is absolutely summable, i.e., if∑∞

n=−∞∑∞

m=−∞ |g[n,m]| < ∞, then the 2D FT ofg[n,m]converges, andG(ωX, ωY) is ananalytic function and is infinitely differentiable with respect toωX andωY.

Alternatively, if g[n,m] is square summable(an energy signal), then the 2D DSFT converges in amean-squaresense:∫ π

−π

∫ π

−π

|GN (ωX, ωY)−G(ωX, ωY)|2 dωX dωY → 0

asN → ∞, whereGN (ωX, ωY) is the truncated sum given above. This is theoretically weaker than pointwise convergence, but itmeans that the error energy diminishes with increasing N, sopractically the two functions become physically indistinguishable.


Properties of the DSFT

The table on the next page summarizes the properties of the 2DDSFT. Most of these are straightforwardly derivable and very muchlike the analogous property of the 2D continuous-space FT. In the following we mention properties of the DSFT that differfromthose of the continuous-space FT.

Periodicity

The 2D DSFT is periodic with period(2π, 2π):

G(ωX + 2π, ωY) =∞∑

n=−∞

∞∑

m=−∞gd[n,m] e−ı[(ωX+2π)n+ωYm] =

∞∑

n=−∞

∞∑

m=−∞gd[n,m] e−ı[ωXn+ωYm] = G(ωX, ωY),

and likewiseG(ωX, ωY + 2π) = G(ωX, ωY).

Multiplication property

Since the 2D DSFT is periodic, the space-domainmultiplication property is a little more involved than simply “convolution” inthe frequency domain.

f [n,m] g[n,m]DSFT←→ 1

(2π)2F (ωX, ωY) ∗∗

2πG(ωX, ωY) =

1

(2π)2

∫ π

−π

∫ π

−π

F (λ1, λ2) G(ωX − λ1, ωY − λ2) dλ1 dλ2,

which is (2π) periodic convolution. (Notice that the integrand is periodic with period2π.) This can be derived by substitutingthe summations forF (ωX, ωY) andG(ωX, ωY) into the convolution sum and reducing the expression to an integral that can berecognized as the DSFT off [n,m] g[n,m].

Duality

There is noduality property since the arguments are fundamentally different.

Transpose property

g[n,m]DSFT←→ G(ωX, ωY) =⇒ g[m,n]

DSFT←→ G(ωY, ωX)

Reflection property

g[±n,±m]DSFT←→ G(±ωX,±ωY)

Rotational symmetry propertyIf g[n,m] has 2- 4- or 8-fold rotational symmetry, then so does its 2D DSFT.

Circular symmetry properties

If a signal g[n,m] is weak-sense circularly symmetric, what can we say about its spectrum G(ωX, ωY)? ??

If a signal g[n,m] is strong-sense circularly symmetric, what can we say about its spectrum G(ωX, ωY)? ??

Exercise.Find a FT property for upsampling.

Exercise.Find a FT property for downsampling.


Properties of the 2D Discrete-Space Fourier Transform

Space Fourier

Synthesis g[n,m] = 1(2π)2

∫ π

−π

∫ π

−πG(ωX, ωY) eı(ωXn+ωYm) dωX dωY

Analysis G(ωX, ωY) =∑∞

n=−∞∑∞

m=−∞ g[n,m] e−ı(ωXn+ωYm)

Eigenfunction h[n,m] ∗∗ eı(an+bm) = H(a, b) eı(an+bm) H(ωX, ωY)(2π)2 δ2((ωX − a, ωY − b))(2π,2π)

= H(a, b)(2π)2 δ2((ωX − a, ωY − b))(2π,2π)

Linearity a1 f1[n,m] +a2 f2[n,m] a1 F1(ωX, ωY) +a2 F2(ωX, ωY)

Shift g[n− n0,m−m0] G(ωX, ωY) e−ı(ωXn0+ωYm0)

Reversal g[−n,−m] G(−ωX,−ωY)

Convolution f [n,m] ∗∗ h[n,m] F (ωX, ωY) H(ωX, ωY)

Multiplication (windowing) f [n,m] g[n,m]1

(2π)2F (ωX, ωY) ∗∗

2πG(ωX, ωY)

Cross-Correlation f [n,m]?? g[n,m] = f [n,m] ∗∗ g∗[n,m] F (ωX, ωY) G∗(ωX, ωY)

Frequency shift g[n,m] eı(an+bm) G(ωX − a, ωY − b)

Freq. Differentiation −ı n g[n,m] ∂∂ωX

G(ωX, ωY)

Conjugation g∗[n,m] G∗(−ωX,−ωY)

Symmetry properties g[n,m] real G(ωX, ωY) = G∗(−ωX,−ωY)

g[n,m] = g∗[−n,−m] G(ωX, ωY) real

Separability g[n,m] = g1[n] g2[m] G(ωX, ωY) = G1(ωX)G2(ωY)

Parseval’s Theorem∑∞

n=−∞∑∞

m=−∞ f [n,m] g∗[n,m] =1

(2π)2

∫ π

−π

∫ π

−π

F (ωX, ωY) G∗(ωX, ωY) dωX dωY

Parseval/Rayleigh Theorem E =∑∞

n=−∞∑∞

m=−∞ |g[n,m]|2 =1

(2π)2

∫ π

−π

∫ π

−π

|G(ωX, ωY)|2 dωX dωY

DC Value∑∞

n=−∞∑∞

m=−∞ g[n,m] = G(0, 0)

Examples of the 2D Discrete-Space Fourier Transform

Space Fourier

δ2[n,m] 1

1 (2π)2 δ2((ωX, ωY))(2π,2π)

eı(an+bm) (2π)2 δ2((ωX − a, ωY − b))(2π,2π)∑∞

k=−∞∑∞

l=−∞(2π)2 δ2(ωX − a− 2πk, ωY − b− 2πl)

{

1, |n| ≤ N, |m| ≤M0, otherwise

sin(ωX(N + 1/2))

sin(ωX/2)

sin(ωY(M + 1/2))

sin(ωY/2)

ωmax

Xωmax

Y

π2 sinc2

(

ωmax

X

π n,ωmax

Y

π m)

“ rect2

(

ωX

2ωmax

X

, ωY

2ωmax

Y

)

”

The functionsin(ω(N + 1/2))

sin(ω/2)is sometimes called theDirichlet kernel or theperiodic sinc function.

The MATLAB functiondiric implements something close to this function.

Note that the DSFT of “rect(

nN

)

” is not N sinc(Nω). This is a common mistake! Exercise. Derive the rect/Dirichlet pair above.


Transforms of elementary functions

Impulse

δ2[n,m]DSFT←→ 1

Complex exponential and constant

In CS, we have the FT paireı2π(ax+by) F2←→ δ2(νX − a, νY − b) .

What about in DS?Can the spectrum of a DS signal consist of a single impulse? ??

Consider the following periodic spectrum:

G(ωX, ωY) = (2π)2 δ2((ωX − a, ωY − b))(2π,2π) ,

∞∑

k=−∞

∞∑

l=−∞(2π)2 δ2(ωX − a− k2π, ωY − b− l2π)

(using Dirac impulses), then by the inverse FT (assuminga, b ∈ (−π, π]):

g[n,m] =1

(2π)2

∫ π

−π

∫ π

−π

(2π)2 δ2(ωX − a, ωY − b) eı(ωXn+ωYm) dωX dωY

=

∫ π

−π

∫ π

−π

δ2(ωX − a, ωY − b) eı(ωXn+ωYm) dωX dωY = eı(an+bm) .

So we have shown

eı(an+bm) DSFT←→∞∑

k=−∞

∞∑

l=−∞(2π)2 δ2(ωX − a− k2π, ωY − b− l2π) = (2π)2 δ2((ωX − a, ωY − b))(2π,2π),

where the subscript(2π, 2π) means, as indicated by the summation formula, that the stated function is repeated periodically withhorizontal period2π and vertical periodπ.

One special case of this is a constant function (a = b = 0), for which we obtain

1DSFT←→ (2π)2 δ2((ωX, ωY))(2π,2π) .

Periodic signals

If g[n,m] is periodic with period(N,M), then using the 2D DS Fourier series we can writeg[n,m] in terms of complex exponen-tials:

g[n,m] =1

NM

N−1∑

k=0

M−1∑

l=0

G[k, l] e−ı2π(nk/N+ml/M) ,

whereG[k, l] denotes theN M -point DFT ofg[n,m]:

G[k, l] =

N−1∑

n=0

M−1∑

m=0

g[n,m] e−ı2π(kn/N+lm/M) .

Thus the Fourier transform of any periodic signalg[n,m] impulsive:

g[n,m]DSFT←→ 1

NM

N−1∑

k=0

M−1∑

l=0

G[k, l](2π)2 δ2((ωX − 2πk/N, ωY − 2πl/M))(2π,2π)

=

(

1

NM

N−1∑

k=0

M−1∑

l=0

G[k, l](2π)2 δ2(ωX − 2πk/N, ωY − 2πl/M)

)

(2π,2π)

.


Sampling revisited

Using the 2D DSFT, we can now derive 2D sampling theorywithout any mention of Dirac impulse functions.

Consider ideal rectilinear point sampling:gd[n,m] = ga(n∆X,m∆Y) .

Using the inverse 2D FT we have

gd[n,m] = ga(n∆X,m∆Y)

=

∫ ∞

−∞

∫ ∞

−∞Ga(νX, νY) e−ı2π[n∆XνX+m∆YνY] dνX dνY

=∞∑

k=−∞

∞∑

l=−∞

∫ (k+1/2)/∆X

(k−1/2)/∆X

∫ (l+1/2)/∆Y

(l−1/2)/∆Y

Ga(νX, νY) e−ı2π[n∆XνX+m∆YνY] dνX dνY

let ωX = 2π(νX∆X − k) andωY = 2π(νY∆Y − l)

=∞∑

k=−∞

∞∑

l=−∞

∫ π

−π

∫ π

−π

Ga

(

ωX/(2π) + k

∆X

,ωY/(2π) + l

∆Y

)

e−ı[(ωX+2πk)n+(ωY+2πl)m] dωX dωY

(2π∆X)(2π∆Y)

=1

(2π)2

∫ π

−π

∫ π

−π

[

1

∆X∆Y

∞∑

k=−∞

∞∑

l=−∞Ga

(

ωX/(2π) + k

∆X

,ωY/(2π) + l

∆Y

)

]

e−ı(ωXn+ωYm) dωX dωY .

This final expression is in the form of the inverse 2D DSFT. Thus, the bracketed expression must be the 2D DSFT ofgd[n,m]. Inother words, we have shown the following relationship between the spectrum of the sampled signalgd[n,m] and the original CSsignalga(x, y):

Gd(ωX, ωY) =1

∆X∆Y

∞∑

k=−∞

∞∑

l=−∞Ga

(

ωX/(2π) + k

∆X

,ωY/(2π) + l

∆Y

)

. (DS-8)

This expressions holdsregardlessof whether the CS signalga(x, y) is band-limited!

-νX

6νY

10/mm

5/mm Ga(νX, νY)

To avoid aliasing, what is maximum sample spacing? ??

Suppose we sample with∆X = ∆Y = 1/30 mm. Find spectrum ofgd[n,m].Using DS-8 above, we have thatGd(ωX, ωY) is (replicates of) the following, where the amplitude will be scaled by1/∆2 = 302.

-ωX

6ωY

2π/3 π

Gd(ωX, ωY)

Note:ωX = 2π∆XνX.


1.4Magnitude and Phase (an introduction)

The 2D DSFT is generally complex, which can be expressed using the real and imaging parts, or in terms of magnitude and phase:

G(ωX, ωY) = GR(ωX, ωY) +ı GI(ωX, ωY) = |G(ωX, ωY)| eıθ(ωX,ωY) whereθ(ωX, ωY) , ∠G(ωX, ωY) .

In applications such as transform image coding, it is usefulto think about the relative importance of the magnitude and phasecomponents. Furthermore, in some imaging contexts one can only make measurements of the magnitude of the Fourier transformof an object, and the phase terms must be estimated somehow from the magnitude using prior information. This latter problem isknown asphase retrieval, and wemay discuss it in more detail along with image restoration later.

The basic question is: can we recover an image from just the magnitude or phase of its 2D DSFT? In general of course the answeris no. But if the object isreal, or belongs to some other constrained class, then it may be possible.

Example. Fig. DS.1 shows an image of Fourier and of the magnitude and phase of Fourier’s transform. I displayed the magnitudeon logarithmic scale so that the details are more clear.

The naive approach to trying to determineg[n,m] from its magnitude or phase alone would be simply to compute the inverse FT:

∫ π

−π

∫ π

−π

|G(ωX, ωY)| eı(ωXn+ωYm) dωX dωY, or∫ π

−π

∫ π

−π

eı∠ G(ωX,ωY) eı(ωXn+ωYm) dωX dωY .

Fig. DS.1 shows that this simple approach fails completely for the magnitude, but some recognizable information is seenin theinverse FT of the phase.

We can get some insight into why the magnitude fails by thinking about the magnitude squared (which is closely related to themagnitude). What is the inverse 2D DSFT of |G(ωX, ωY)|2? The autocorrelation ofg[n,m], which, as illustrated in Fig. DS.1is also visually uninterpretable!

So image recovery via a simple inverse FT clearly is suboptimal. There are better approaches!

Suppose the image is known to bereal and has knownsupport, i.e., it is known to be zero except forn,m pairs in a finite set. Then

tan θ(ωX, ωY) =real(G(ωX, ωY))

imag(G(ωX, ωY))=

∑

n,m g[n,m] sin(ωXn + ωYm)∑

n,m g[n,m] cos(ωXn + ωYm),

where the sums are over the support set. Cross multiplying and subtracting yields

0 =∑

n,m

g[n,m] [sin(ωXn + ωYm)− tan θ(ωX, ωY) cos(ωXn + ωYm)] .

If we have measured the phasetan θ(ωX, ωY) for at least as manyωX, ωY pairs as there are elements in the support set, then we can(try to) solve the above set of linear equations up to a scale factor.

Solving such a large systems of equations is likely to be verynoise sensitive.

Later we will discussiterative methodsfor image recovery from either magnitude or phase information, including noise control.


0

50

100

150

200

250

n

m

Jean Baptiste Joseph Fourier

50 100 150 200 250

50

100

150

200

2500.5

1

1.5

2

2.5

x 109

∆ n

∆ m

Fourier’s autocorrelation

−200 −100 0 100 200

−250

−200

−150

−100

−50

0

50

100

150

200

250

2

4

6

8

10

12

14

16

Fourier’s Transform: log(Magnitude)

π−π π/2−π/2 ωX

π

−π

π/2

−π/2

ωY

−3

−2

−1

0

1

2

3

Fourier’s Transform: Phase

π−π π/2−π/2 ωX

π

−π

π/2

−π/2

ωY

500

1000

1500

2000

2500

3000

3500

4000

n

m

IFFT2(Magnitude)

50 100 150 200 250

50

100

150

200

250

−0.01

−0.005

0

0.005

0.01

n

m

IFFT2(exp(i*Phase))

50 100 150 200 250

50

100

150

200

250

Figure DS.1: Fourier and his transforms.


Frequency-domain characteristics of LSI systems

A LSI system is characterized completely by itsfrequency responseH(ωX, ωY) (provided it exists).

By the convolution property, if the input signalf [n,m] has transformF (ωX, ωY), then

F (ωX, ωY)→ LSI H(ωX, ωY) → G(ωX, ωY) = F (ωX, ωY) H(ωX, ωY) .

This property is the foundation forfiltering .

The output of an LSI system will only contain frequency components that are present in the input signal.

Recall a LSI is BIBOstable iff its impulse responseh[n,m] is absolutely summable. Fortunately, the DSFT always exists forabsolutely summable signals, soH(ωX, ωY) always exists for stable LSI systems.

In terms of the frequency response H(ωX, ωY), when is a LSI system invertible? ??

For what a is the system with impulse response h[n,m] =

0 a 0a 1 a0 a 0

invertible? ??

Rotation invariance

Consider the end-to-end system (DS-7), and restrict attention to LSI systems.

For what type of frequency response H(ωX, ωY) is the end-to-end system rotation invariant? ??


Introduction to filter design

A common image processing task is to filter an image to performtasks such as removing noise (lowpass filtering) or enhancingedges (highpass filtering). Perhaps it is not the most glamorous task, but it is usually indispensable as a preprocessingstep beforemore exciting image processing.Many of the filter design principles for images parallel those of filter design in 1D DSP,e.g., tradeoffs between transition bandwidthand width of the impulse response. Differences include the desirability of rotation invariance, our indifference tocausality, andthe lack of very useful Z-transform tools.

Introduction

We treat the subject of filter design using a running example.Consider the 1D filter with impulse responseh[n] = [1/4 1/2 1/4]. What is the frequency response of this filter?

H(ω) =

∞∑

n=−∞h[n] e−ıωn = (1/4) + (1/2) e−ıω + (1/4) e−ı2ω = (1/4) e−ıω (eıω + 2 + e−ıω) = (1/2) e−ıω (1 + cos ω).

Since1 + cos ω ≥ 0, the phase response is∠H(ω) = −ω,

so there is aunit delay or shift associated with this filter. The magnitude response is

|H(ω)| = (1/2)(1 + cos ω), (Picture) with |H(0)| = 1 and|H(π)| = 0,

which is a reasonable simple lowpass filter. Note that the highest discrete-space frequency (±π) is eliminated by this filter. (So itis in that sense a better design than the moving average [1/3 1/3 1/3].)

How do we make a simple 2D lowpass filter with similar characteristics?

Design 1

A first attempt makesh[n,m] look like h[n] along each axis:h[n,m] =

0 1/4 01/4 1/2 1/4

0 1/4 0

.

Is this separable? No. So we must use the full 2D formula to compute the frequencyresponse:

H(ωX, ωY) =

∞∑

n=−∞

∞∑

m=−∞h[n,m] eı(ωXn+ωYm) =

1

2+

1

4

(

e−ıωX + eıωX + e−ıωY + eıωY

)

=1

2(1 + cos ωX + cos ωY) .

H(ωX, ωY) is real (due to Hermitian symmetry ofh[n,m]). Is the phase response zero? Almost:

∠H(ωX, ωY) =

{

±π, 1 + cos ωX + cos ωY < 00, otherwise.

So there is noshift! This is one pleasant difference between image processing and 1D signal processing:zero phasefilters aretrivial to design, and feasible to implement in 2D, since “causality” is generally unimportant.

The magnitude response is|H(ωX, ωY)| = (1/2) |1 + cos ωX + cos ωY| .

Is this a lowpass filter? Where does the magnitude response drop to zero? Let us look at the slice of this magnituderesponse whereωY = 0:

H(ωX, 0) = (1/2)[2 + cos ωX], (Picture) with |H(0, 0)| = 3/2 and|H(π, 0)| = 1/2

2D (Picture) of H(ωX, ωY) with H(π, π) = −1/2.

So this filter behaves somewhat differently than the 1D design. In particular, the response to the checkerboard input imagef [n,m] = (−1)n+m with frequency(π, π) is nonzero.

We probably would like a lowpass design that is zero along theedge of the±π,±π box, i.e.,

H(±π, ωY) = H(ωX,±π) = 0, ∀ωX, ωY.


Design 2

Next we try aseparablefilter design, based on the 1D filter:

h[n,m] = h[n + 1]h[m + 1] =

1/16 1/8 1/161/8 1/4 1/8

1/16 1/8 1/16

.

Since this is separable, the corresponding frequency response is simply the product of the 1D frequency response (without thephase shift):

H(ωX, ωY) = (1/4)(1 + cos ωX)(1 + cos ωY), (Picture) with H(0, 0) = 1 andH(±π, ωY) = H(ωX,±π) = 0.

(Since this one is real and nonnegative, we can just plotH(ωX, ωY) rather than|H(ωX, ωY)|.)This design looks good so far. But what if we look at the slice along the 45◦ line whereωX = ωY?

(Picture) of H(ωX, ωY) vsω =√

ω2X

+ ω2Y

going to zero atω = π√

2, with H(

π/√

2, π/√

2)

= (1/4)[1+cos(

π/√

2)

]2 ≈ 0.04.

This filter design is not circularly symmetric! So rotation of the analog input image, followed by sampling and then filtering withthis filter will yield different results (not just a rotationof the output).

Example. To demonstrate this lack of rotation invariance we sampledthe following analog image as shown in Fig. DS.2:

f [n,m] = fa(x, y)|x=n, y=m wherefa(x, y) = cos(

0.9 ∗ π ∗√

x2 + y2)

.

Filtering thisf [n,m] with the above impulse responseh[n,m] yields the output image shown on the bottom left in Fig. DS.2.

After filtering, more of the sinusoidal signal components inthe diagonal directions remain than in the horizontal and verticaldirections, even though the spatial frequency is essentially 0.9π in all directions.

Design 3

Idea: start in the frequency domain, then work backwards to find the impulse response. Suppose we like the 1D behavior of

H(ω) = (1/2)[1 + cos ω].

Then it would be reasonable to consider a 2D filter with the following circularly symmetric frequency response:

H(ωX, ωY) = (1/2)

(

(

1 + cos√

ω2X

+ ω2Y

)

rect

(

√

ω2X

+ ω2Y

2π

))

(2π,2π)

, (DS-9)

where the inner expression shows the central replicate, andthe subscript reminds us thatH(ωX, ωY) is (2π, 2π)-periodic.

A completely precise formula would be An alternate expression is

H(ωX, ωY) =

∞∑

k=−∞

∞∑

l=−∞4(

1 + cos√

(ωX − 2πk)2 + (ωY − 2πl)2)

rect

(

√

(ωX − 2πk)2 + (ωY − 2πl)2

2π

)

,

which is clearly(2π, 2π)-periodic. However, this form is probably harder to grasp quickly than the preceding “shorthand” formula.

(Picture) of H(ωX, ωY) with zero ring at radiusπ.

The filter with frequency response (DS-9) is IIR. To my knowledge, there is no analytical solution for the inverse 2D DSFT of thisspectrum.


n

m

f(n,m)

−20 0 20−20

−10

0

10

20

−1

−0.5

0

0.5

1

x

y

fa(x,y)

−20 0 20−20

−10

0

10

20

−0.1

−0.05

0

0.05

0.1

n

m

g2(n,m) = f(n,m) ** h

2(n,m)

−20 0 20−20

−10

0

10

20

−0.1

−0.05

0

0.05

0.1

n

m

g3(n,m) = f(n,m) ** h

3(n,m)

−20 0 20−20

−10

0

10

20

Figure DS.2: Circularly symmetric image sampled and filtered. The designh3[n,m] has a frequency response that is more invariantto the orientation of the “sinusoidal” components.


However, it is simple to use MATLAB to computeh[n,m] approximately, as follows.

% fig_fir3an = 128;u = [-n/2:n/2-1]/n*2*pi;v = [-n/2:n/2-1]/n*2*pi;[uu vv] = ndgrid(u,v);rect = inline(’abs(t) < 1/2’);om = sqrt(uu.ˆ2+vv.ˆ2);H = (1/2)*(1 + cos(om)) .* rect(om/2/pi);im(H)

h = fftshift(ifft2(fftshift(H)));h = real(h);im(h)

−0.00

0.00

−0.00

−0.00

−0.00

0.00

−0.00

0.00

−0.00

−0.00

0.01

−0.00

−0.00

0.00

−0.00

−0.00

0.06

0.13

0.06

−0.00

−0.00

−0.00

0.01

0.13

0.23

0.13

0.01

−0.00

−0.00

−0.00

0.06

0.13

0.06

−0.00

−0.00

0.00

−0.00

−0.00

0.01

−0.00

−0.00

0.00

−0.00

0.00

−0.00

−0.00

−0.00

0.00

−0.00

h[n,m]

n

m

For practical use we would prefer an FIR filter. For example, we take the central5× 5 block of this impulse response, and quantizethe coefficients to one decimal place. The following figure shows contour plots of the resulting frequency response of this practicalfilter (dashed lines) compared to the ideal circular contours (solid lines). The practical frequency response is remarkably circularlysymmetric. Althoughexactrotation invariance is impossible to achieve with an FIR filter, this example shows that a reasonableapproximation is achievable with only a5× 5 mask, even with the filter coefficients quantized to two base 10 digits or about 5 bits.

−3 −2 −1 0 1 2 3−3

−2

−1

0

1

2

3

ωX

ωY

Contours of H(ωX,ω

Y)

Fig. DS.2 shows the result of applying this truncated, quantized filter to the circles image. The response is much more uniform inall directions.


Design 4

If we are going to abandon FIR considerations for specifyingthe frequency response, then perhaps we should just be optimisticand consider theideal lowpassfrequency response:

H(ωX, ωY) = rect

(

√

ω2X

+ ω2Y

2ωc

)

(Picture) ,

(and periodic of course) where0 < ωc < π.

What is the impulse response of this filter?

We could attempt to integrate to find the inverse FT. A more efficient method makes use of the fact that we already know that

jinc(r)F2←→ rect(ρ), soh[n,m] is samples of someha(x, y) where, by the sampling theorem:

h[n,m]DSFT←→ H(ωX, ωY) =

1

∆2

∞∑

k=−∞

∞∑

l=−∞Ha

(

ωX/2π − k

∆,ωY/2π − l

∆

)

.

using∆X = ∆Y = ∆ for simplicity. By thescaling property of the 2D CS FT:

ha(x, y) = (2ρc)2 jinc(2ρcr)

F2←→ Ha(νX, νY) = rect

(

ρ

2ρc

)

.

Defineω =√

ω2X

+ ω2Y

as the digital radial frequency component. Considering thecentral replicate, we need to makerect(

ω2ωc

)

match with 1∆2 Ha

(

ω2π∆

)

= 1∆2 rect

(

ω4π∆ρc

)

. Thus we need soρc = ωc

2π∆ .

Choosing∆ = 1 for simplicity, the impulse response is:

h[n,m] = ha(n,m) = ha

(

√

n2 + m2)

=(ωc

π

)2

jinc(ωc

π

√

n2 + m2)

.

Clearly thiscircularly symmetric , ideal lowpass filterhas an IIR impulse response, as expected from similar results in 1D.

0.0

−0.1

0.0

0.1

0.0

−0.1

0.0

−0.1

0.1

0.0

−0.1

0.0

0.1

−0.1

0.0

0.0

−0.2

0.4

−0.2

0.0

0.0

0.1

−0.1

0.4

2.0

0.4

−0.1

0.1

0.0

0.0

−0.2

0.4

−0.2

0.0

0.0

−0.1

0.1

0.0

−0.1

0.0

0.1

−0.1

0.0

−0.1

0.0

0.1

0.0

−0.1

0.0

h(n,m) for ideal lowpass

n

m


Summary

This chapter has emphasized LSI systems and frequency analysis.

Many image processing operations, such as median filtering (to be discussed soon) arenonlinear, so the LSI analysis is inapplica-ble.

Bibliography[1] J. S. Lim. Two-dimensional signal and image processing. Prentice-Hall, New York, 1990.[2] A. K. Jain. Fundamentals of digital image processing. Prentice-Hall, New Jersey, 1989.[3] M. Vetterli and J. Kovacevic.Wavelets and subband coding. Prentice-Hall, New York, 1995.

2d discrete-space signals and systemshyunjinp/notes/n-disc.pdf · 2d discrete-space signals and...

Documents