lecture 14: convolution and frequency domain filtering › class › simg782 › lectures ›...

Lecture 14: Convolution and FrequencyDomain Filtering

Harvey RhodyChester F. Carlson Center for Imaging Science

Rochester Institute of [email protected]

October 27, 2005

AbstractThe impulse response of a filter describes its spatial processing

behavior. This is closely tied to the global processing characteristicsthat are obtained using Fourier transforms. Here we focus onthe relationship between the spatial and frequency domains andprovide examples of alternative implementations of filters with variousdesirable characteristics.

DIP Lecture 14

Filtering by Convolution

We will first examine the relationship of convolution and filtering byfrequency-domain multiplication with 1D sequences.

Let f(n), 0 ≤ n ≤ L− 1 be a data record.

Let h(n), 0 ≤ n ≤ K − 1 be the impulse response of a discrete filter.

If the sequence f(n) is passed through the discrete filter then the output is

g(m) =k2∑

k=k1

h(k)f(m− k), −∞ < m < ∞

where k1 and k2 are appropriate limits on the index.

DIP Lecture 14 1

Index Values

The values of k1 and k2 depend upon m. Getting this dependence right isthe detail that causes the most trouble in the implementation of convolution.

Condition Restrictionh(k) = 0 for k < 0 k1 ≥ 0f(m− k) = 0 for m− k ≥ L k1 ≥ m− L + 1h(k) = 0 for k ≥ K k2 ≤ K − 1f(m− k)=0 for m− k < 0 k2 ≤ m

These restrictions can be combined with the requirement that k1 ≤ k2 toyield

m ≥ k2 ≥ k1 ≥ 0 ⇒ m ≥ 0 (1)

K − 1 ≥ k2 ≥ k1 ≥ m− L + 1 ⇒ m ≤ K + L− 2 (2)

Hence, g(m) = 0 outside the interval 0 ≤ m ≤ K + L− 2.

DIP Lecture 14 2

Convolution Sum

g(m) =k2(m)∑

k=k1(m)

h(k)f(m− k), 0 ≤ m ≤ K + L− 2

= 0, elsewhere

where the limits are described by the following tables.Case K < L

Range of m k1 k2

0 ≤ m ≤ K − 1 0 m

K ≤ m ≤ L− 1 0 K − 1

L ≤ m ≤ K + L− 2 m− L + 1 K − 1

Case K ≥ L

Range of m k1 k2

0 ≤ m ≤ L− 1 0 m

L ≤ m ≤ K − 1 m− L + 1 m

K ≤ m ≤ K + L− 2 m− L + 1 K − 1

DIP Lecture 14 3

Periodic Convolution

The complexity of the convolution calculation can be reduced by usingperiodic functions.

Let f(n) and h(n) be padded with zeros to length P ≥ L + K − 1 andthen repeated with period P . The resulting periodic functions will be calledfP (n) and hP (n).

The periodic convolution is defined as

gP (m) =P−1∑k=0

hP (k)fP (m− k)

The result is periodic with period P .

gP (m + jP ) =P−1∑k=0

hP (k)fP (m + jP − k) =P−1∑k=0

hP (k)fP (m− k)

DIP Lecture 14 4

Example

Shown at the right are twofunctions that we would liketo process by convolution.

f(n) has length L = 6 andh(n) has length K = 5.

The functions have beenextended by zero-padding tolength P = L + K − 1 = 10.

2 4 6 8

1

2

3

4

5

Function f(n).

2 4 6 8

-1

-0.5

0.5

1

1.5

2

Function h(n).

DIP Lecture 14 5

Example

The convolution of fP (n) withhP (n) is shown at the right.One period of fP (n) is shownat the top of each column.The cyclic shifts hP (n−m) ofthe filter response are shownin rows 2-6.

The value of gP (m) is foundby multiplying hP (m−n) withfP (n) and summing. Theresult for each shift is listedbeside the corresponding filtershift.

2 4 6 8

1

2

3

4

5

2 4 6 8

1

2

3

4

5

g(0) = 02 4 6 8

-1

-0.5

0.5

1

1.5

2

2 4 6 8

-1

-0.5

0.5

1

1.5

2

g(5) = 15

g(1) = 12 4 6 8

-1

-0.5

0.5

1

1.5

2

2 4 6 8

-1

-0.5

0.5

1

1.5

2

g(6) = 12

g(2) = 42 4 6 8

-1

-0.5

0.5

1

1.5

2

2 4 6 8

-1

-0.5

0.5

1

1.5

2

g(7) = 2

g(3) = 82 4 6 8

-1

-0.5

0.5

1

1.5

2

2 4 6 8

-1

-0.5

0.5

1

1.5

2

g(8) = −4

g(4) = 122 4 6 8

-1

-0.5

0.5

1

1.5

2

2 4 6 8

-1

-0.5

0.5

1

1.5

2

g(9) = −5

DIP Lecture 14 6


It is readily demonstrated that

gP (m) = g(m), 0 ≤ m ≤ P − 1

Thus, we can use circular convolution to compute the a finite convolution.

The periodic functions fP (n) and hP (n) have the discrete Fourier transforms

F (u) =1P

P−1∑n=0

fP (n)e−i2πun/P H(u) =1P

P−1∑n=0

hP (n)e−i2πun/P

fP (n) =P−1∑u=0

F (u)ei2πun/P hP (n) =P−1∑u=0

H(u)ei2πun/P

Note the dependence on P , rather than on L or K. Note also that F (u)and H(u) are periodic with period P .

DIP Lecture 14 7

Example

10 20 30 40 50

1

2

3

4

5

10 20 30 40 50

-1

-0.5

0.5

1

1.5

2

10 20 30 40 50

-5

5

10

15

Function fP (n). Function hP (n). Function gP (n).

The functions involved in the periodic convolution of f and h are shownabove. The period P = 10 is the minimum length for these functions.The result is a periodic function that is in agreement with g in the interval0 ≤ n ≤ K + L− 1.

DIP Lecture 14 8

DFTThe DFT of gP (m) can be computed by

G(u) =1P

P−1∑m=0

gP (m)e−i2πum/P

=1P

P−1∑k=0

P−1∑m=0

hP (k)fP (m− k)e−i2πum/P

Make the change of index m = k + n and rearrange.

G(u) =1P

P−1∑k=0

hP (k)P−k−1∑n=−k

fP (n)e−i2πu(n+k)/P

=

(1P

P−1∑k=0

hP (k)e−i2πuk/P

)P−k−1∑n=−k

fP (n)e−i2πun/P

The terms n = −k, . . . ,−1 in the last sum can be replaced by n = P −k, . . . , P − 1

because the summand is periodic. Hence . . .

DIP Lecture 14 9

DFT

The sum then simplifies to the product

G(u) = P

(1P

P−1∑k=0

hP (k)e−i2πuk/P

)(1P

P−1∑n=0

fP (n)e−i2πun/P

)= PH(u)F (u)

1. By construction, fP (n), hP (n) and gP (n) are all periodic with period P .

2. It is required that P ≥ L + K − 1.

3. F (u), H(u) and G(u) are all complex periodic functions with period P .

4. F (u) can be computed by padding f(n) with zeros to length P and thendoing a DFT.

5. A similar statement is true for H(u) and G(u).6. u takes on integer values.

DIP Lecture 14 10

Example

Function F (u). Function H(u). Function G(u).

The transforms F (u), H(u) and G(u) can be displayed in the complex plane.The points are numbered with the value of the index u, 0 ≤ u ≤ P − 1.Because the functions are periodic, they follow the same cycle repetitively.

DIP Lecture 14 11

Power Relationships

The mean-squared value of f(n) over the interval of length P is

1P

P−1∑n=0

fP (n)f∗P (n) =1P

P−1∑n=0

P−1∑u=0

P−1∑v=0

F (u)F ∗(v)ei2π(u−v)n/P

=P−1∑u=0

P−1∑v=0

F (u)F ∗(v)

(1P

P−1∑n=0

ei2π(u−v)n/P

)

The term in parentheses is δu,v Hence,

1P

P−1∑n=0

|fP (n)|2 =P−1∑u=0

|F (u)|2

1. Because f(n) = fP (n) in 0 ≤ n ≤ P − 1, the above statement is alsovalid for f(n).

2. An equivalent statement is true for hP (n) and gP (n).

DIP Lecture 14 12

Example

The power calculations for the waveforms and their transforms are shown inthe table below. In each case, the same result is obtained in both domains.

1P

P−1∑n=0

|fP (n)|2 = 5.51P

P−1∑n=0

|hP (n)|2 = 0.71P

P−1∑n=0

|gP (n)|2 = 63.9

P−1∑u=0

|F (u)|2 = 5.5P−1∑u=0

|H(u)|2 = 0.7P−1∑u=0

|G(u)|2 = 63.9

DIP Lecture 14 13

Filtering of 2D Functions

Let f(x, y) be a L1 × L2 array that represents the brightness values of animage.

Let h(x, y) be a K1 ×K2 array that represents the impulse response of aspatial filter.

The result of filtering f with h is the array g(x, y). By analogy with the 1Dexample, we find that g(x, y) has size (L1 + K1 − 1, L2 + K2 − 1).

The convolution operation can be most conviently written in terms offunctions that are periodic with period (P1, P2), where

P1 ≥ L1 + K1 − 1

P2 ≥ L2 + K2 − 1

DIP Lecture 14 14

Periodic ConvolutionThe periodic convolution is defined as

gP (x, y) =P1−1∑j=0

P2−1∑k=0

hP (j, k)fP (x− j, y − k)

The result is periodic with period P1, P2.

gP (x + aP1, y + bP2) =P1−1∑j=0

P2−1∑k=0

hP (j, k)fP (x + aP1 − j, y + bP2 − k)

=P1−1∑j=0

P2−1∑k=0

hP (j, k)fP (x− j, y − k)

= gP (x, y)

DIP Lecture 14 15


1. Assume that f(x, y) is given only for 0 ≤ x ≤ L1−1 and 0 ≤ y ≤ L2−1.

2. Assume that h(x, y) is given only for 0 ≤ x ≤ K1−1 and 0 ≤ y ≤ K2−1.

3. Without loss of generality, we can assume any extension we wish outsidethe region of definition.

4. The convolution g = f ? h has size (L1 + K1 − 1, L2 + K2 − 1).5. We can construct periodic functions fP (x, y) and hP (x, y) that are

defined on the primary frame (P1 ≥ L1 + K1− 1, P2 ≥ L2 + K2− 1) bypadding with zeros. In the examples we will use P1 = L1 +K1−1, P2 =L2 + K2 − 1

6. Periodic convolution of fP (x, y) with hP (x, y) produces a periodicfunction gP (x, y) that is equal to g(x, y) for 0 ≤ x ≤ P1 − 1, 0 ≤y ≤ P2 − 1.

DIP Lecture 14 16


7. Periodic convolution is equivalent to multiplication of the transforms ofthe periodic spatial functions.

8. The discrete Fourier transforms are periodic with period (P1, P2).

F (u, v) =1

P1P2

P1−1∑x=0

P2−1∑y=0

fP (x, y)e−i2π�

xuP1

+vyP2

�

=1

P1P2

L1−1∑x=0

L2−1∑y=0

f(x, y)e−i2π�

xuP1

+vyP2

�

Similar expressions hold for the other variables.

DIP Lecture 14 17

Algorithm

1. Read the image array f(x, y) and determine the dimensions (L1, L2).2. Define the filter h(x, y) with dimensions (K1,K2).3. Construct zero-padded arrays fP (x, y) and hP (x, y) with dimensions

(P1, P2).4. Compute F (u, v) = FFT(fP (x, y)) and H(u, v) = FFT(hP (x, y)).5. Compute G(u, v) = P1P2F (u, v)H(u, v)6. Compute gP (x, y) = FFT−1(G(u, v)).

DIP Lecture 14 18

Algorithm Example

;STEP 1: DEFINE THE IMAGE ARRAY WITH SIZE 20X20.N=20 & M=20x=(findgen(N)-N/2)#Replicate(1,M)y=(findgen(M)-M/2)##Replicate(1,N)fxy=exp(-(xˆ2/15+yˆ2/5))

;STEP 2: DEFINE THE FILTER ARRAY WITH SIZE 3X3.hxy=[[-1,0,-1],[0,4,0],[-1,0,-1]]

;STEP 3: PAD THE ARRAYS TO (22,22).hp=zeropad(hxy,22,22)fp=zeropad(fxy,22,22)

;STEP 4: DO THE FFT’sff=FFT(fp)hf=fft(hp)

;STEP 5: COMPUTE G(u,v)gf=hf*ff*22*22

;STEP 6: COMPUTE gpgp=float(fft(gf,/inverse))

DIP Lecture 14 19

Example

Function f(x, y). Function hP (x, y). Function gP (x, y).

Function |F (u, v)|. Function |H(u, v)|. Function |G(u, v)|.

DIP Lecture 14 20

ZeropadFunction zeroPad,A,N,M,Center=center;+;B=zeroPad(A,N,M) returns an array B of size NxM such that B[i,j]=A[i,j];where A is defined and B[i,j]=0 where A[i,j] is not defined. B is the same;type as A.;;If A is a scalar or a row vector then zeroPad can be called with a single;dimension argument. B=zeroPad(A,N) returns a row vector of length N.;;B=zeropad(A) returns an array that has a single ring of zeros around A.;;It is an error if N or M is smaller than the corresponding dimension of A.;;KEYWORDS;;CENTER: Set the keyword center if the array should be centered in the;field of zeros.;;HISTORY;Written by H. Rhody September, 1999;Modified to include additional error checks and CENTER keyword;January 2001.;Modified to handle B=zeropad(A) case. 9.23.2005 HR;-

DIP Lecture 14 21

Zeropad ExamplesIDL> print,zeropad(4)

0 0 00 4 00 0 0

IDL> print,zeropad([1,2,3])0 0 0 0 00 1 2 3 00 0 0 0 0

IDL> print,zeropad([1,2,3],7)1 2 3 0 0 0 0

IDL> print,zeropad([1,2,3],7,/center)0 0 1 2 3 0 0

IDL> print,zeropad(indgen(3,4),7,8,/center)0 0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 2 0 00 0 3 4 5 0 00 0 6 7 8 0 00 0 9 10 11 0 00 0 0 0 0 0 00 0 0 0 0 0 0

DIP Lecture 14 22

Comparison with Convolution

1. Spatial filtering can be carried out by convolution.

2. IDL contains a fast convolution routine, CONVOL.

3. Care must be taken to do mathematical convolution with CONVOL.

Steps to do convolution of arrays A and B with CONVOL.

1. Determine the dimensions (L1, L2) and (K1,K2) of A and B.

2. Construct Ap by zero-padding A to size (L1 + K1 − 1, L2 + K2 − 1).Ap=zeropad(A,L1+K1-1,L2+K2-1)

3. Compute CP=CONVOL(Ap,B,CENTER=0,/EDGE ZERO).

This procedure replaces a more complicated method that was requiredbefore the EDGE ZERO keyword was introduced in IDL 6.2.

The “old” method is used in the CONVOLVE program below, which makes itcompatible with older versions of IDL .

DIP Lecture 14 23

CONVOLVE ProgramFUNCTION CONVOLVE,A,M,SCALE,_EXTRA=extra;+;B=CONVOLVE(A,M,S) returns an array B that is the 2D;convolution of the arrays A and M. The convolution calculation;is actually carried out by the IDL CONVOL routine.;;An array AP is constructed by padding A, then;C=CONVOL(AP,M,S,CENTER=0,/EDGE_TRUNCATE) is computed;and then C is trimmed to provide the proper output.;;The size of the returned array is equal to;Ac+Mc-1 by Ar+Mr-1, where Ac x Ar is the size of;A and Mc x Mr is the size of M.;;If S is provided then the results are scaled by;passing S to the CONVOL routine. Otherwise, the;scale value S=1 is used.;;CONVOL keywords are passed via _EXTRA.;;H. Rhody;September, 2001;-

DIP Lecture 14 24

CONVOLVE (cont)IF N_PARAMS() LT 2 THEN MESSAGE,’Incorrect number of parameters.’SA=Size(A)SM=Size(M)typeA=SA[SA[0]+1]typeM=SM[SM[0]+1]IF N_PARAMS() LT 3 THEN SCALE=1

;Determine the width and height parameters. The case of a 1D array;has to be handled differently.CASE SA[0] OF1: BEGIN

WA=SA[1] & HA=1END

2: BEGINWA=SA[1] & HA=SA[2]END

ENDCASE

CASE SM[0] OF1: BEGIN

WM=SM[1] & HM=1END

2: BEGINWM=SM[1] & HM=SM[2]END

ENDCASE

DIP Lecture 14 25

CONVOLVE (cont);Construct a zero-padded array to use in the convolution operationSAP=[2,WA+2*(WM-1),HA+2*(HM-1),MAX([typeA,typeM]),(WA+2*(WM-1))*(HA+2*(HM-1))]AP=Make_Array(Value=0,Size=SAP)AP[WM-1:WM+WA-2,HM-1:HM+HA-2]=A

;Do the convolution of AP with MAC=CONVOL(AP,M,SCALE,CENTER=0,/EDGE_TRUNCATE,_EXTRA=extra)

;Trim and return the convol result.CSTART=WM-1RSTART=HM-1RETURN,AC[CSTART:CSTART+WM+WA-2,RSTART:RSTART+HM+HA-2]END

DIP Lecture 14 26

Comparison of Transform and Convolution Methods

Output Difference

Shown above is the output array and the magnitude of the differencebetween the transform and convolution methods. All of the differences areless than 3× 10−7.

DIP Lecture 14 27

Power Calculations (cont)

The output image has the frequency-domain relationship

G(u, v) = P1P2F (u, v)H(u, v)

Hence, the power can be computed by

1P1P2

P1−1∑x=0

P2−1∑y=0

|g(x, y)|2 =P1−1∑u=0

P2−1∑v=0

|G(u, v)|2

= (P1P2)2P1−1∑u=0

P2−1∑v=0

|F (u, v)|2|H(u, v)|2

= .0147 for the example above

DIP Lecture 14 29

Image Filtering Example

Image arrays can be processed by either spatial or transform filtering. Justsubstitute the image array for f(x, y) in the above techniques.

You may need to rescale the amplitude of the final array to fit the range[0,255] for display.

Original Image Transform Filtering Convolution Filtering

DIP Lecture 14 30

lecture 14: convolution and frequency domain filtering › class › simg782 › lectures ›...

Documents