multiresolution image processingee225b/fa12/lectures/multi... · bernd girod: ee368 digital image...

Bernd Girod: EE368 Digital Image Processing Multiresolution Image Processing no. 1

Multiresolution image processing

Laplacian pyramidsSome applications of Laplacian pyramidsDiscrete Wavelet Transform (DWT)Wavelet theoryWavelet image compression


Image pyramids

[Burt, Adelson, 1983]


Image pyramid example

original image

Gaussian pyramid Laplacian pyramid


Overcomplete representation

2

Number of samples in Laplacian or Gaussian pyramid =1 1 1 41 ... x number of original image samples4 4 4 3P

⎛ ⎞+ + + + ≤⎜ ⎟⎝ ⎠


LoG vs. DoG

Laplacian of Gaussian Difference of Gaussians


Image processing with Laplacian pyramid

Analysis Synthesis

Interpolator

Interpolator

Subsampling

+

+ -

-

Filtering

Filtering

Interpolator

InterpolatorSubsampling

Inputpicture

Processedpicture

• • • • • • • • • • • •


Expanded Laplacian pyramid

Analysis Synthesis

+

+ -

-

Filtering

Filtering

Inputpicture

Processedpicture

• • • • • • • • • • • •


Mosaicing in the image domain


Mosaicing by blending Laplacian pyramids


Expanded Laplacian pyramids

Original: begoniaOriginal: begonia

Original: dahliaOriginal: dahlia

Gaussian level 2Gaussian level 2

Gaussian level 2Gaussian level 2

Laplacian level 2Laplacian level 2

Laplacian level 2Laplacian level 2


Blending Laplacian pyramidsLevel 0Level 0 Level 2Level 2

Level 4Level 4 Level 6Level 6


Image analysis with Laplacian pyramid

Analysis

Interpolator

Interpolator

Subsampling

+

+ -

-

Filtering

Filtering

Subsampling

Inputpicture

•

Recognition/detection/segmentationresult


Multiscale edge detection

Zero-crossings of Laplacian images of different scales

Spurious edges removed


Multiscale face detection

InputNetwork Outputsu

bsam

plin

g

Preprocessing Neural network

pixels20 by 20

Extracted windowInput image pyramid(20 by 20 pixels)

Correct lighting Histogram equalization Receptive fieldsHidden units

[Rowley, Baluja, Kanade, 1995]


Example: multiresolution noise reduction

Original Noisy Noise reduction:3-level Haar transform

no subsamplingw/ soft coring


1-d Discrete Wavelet Transform

Recursive application of a two-band filter bank to the lowpass band of the previous stage yields octave band splitting:

frequency


Cascaded analysis / synthesis filterbanks

0h0g

1g

1h

0g

1g

0h

1h



ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

...etc


2-d Discrete Wavelet Transform example


Review: Z-transform and subsampling

Generalization of the discrete-time Fourier transform

Fourier transform on unit circle: substituteDownsampling and upsampling by factor 2

( ) [ ] ; ; n

n

x z x n z z r z r∞

− − +

=−∞

= ∈ < <∑ C

jz e ω=

2 2( )x z ( ) ( )12

x z x z+ −⎡ ⎤⎣ ⎦


Two-channel filterbank

Aliasing cancellation if :0 1

1 0

( ) ( )( ) ( )

g z h zg z h z

= −− = −

[ ] [ ]

[ ] [ ] ( )

0 0 0 1 1 1

0 0 1 1 0 0 1 1

1 1ˆ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 21 1( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 2

x z g z h z x z h z x z g z h z x z h z x z

h z g z h z g z x z h z g z h z g z x z

= + − − + + − −

= + + − + − −

Aliasing

2 2

2 2

0h

1h

0g

1g

( )x z ( )x̂ z


Example: two-channel filter bank with perfect reconstruction

Impulse responses, analysis filters:Lowpass highpass

Impulse responses, synthesis filtersLowpass highpass

Mandatory in JPEG2000Frequency responses:

|g |0

|h |0

1|h |

1|g |

π2

π

1

2

00

FrequencyFr

eque

ncy

resp

onse

1 1 3 1 1, , , ,4 2 2 2 4− −⎛ ⎞

⎜ ⎟⎝ ⎠

1 1 1, ,4 2 4

−⎛ ⎞⎜ ⎟⎝ ⎠

1 1 3 1 1, , , ,4 2 2 2 4

−⎛ ⎞⎜ ⎟⎝ ⎠

1 1 1, ,4 2 4

⎛ ⎞⎜ ⎟⎝ ⎠

“Biorthogonal 5/3 filters”“LeGall filters”


Lifting

Analysis filters

L “lifting steps”First step can be interpreted as prediction of odd samples from the even samples

K0

1λ 2λ 1Lλ − LλΣ Σ

Σ Σ

K1

[ ]even samples 2x n

[ ]odd samples

2 1x n +

0low band y

1high band y

[Sweldens 1996]


Lifting (cont.)

Synthesis filters

Perfect reconstruction (biorthogonality) is directly build into lifting structurePowerful for both implementation and filter/wavelet design

1λ 2λ 1Lλ − LλΣ Σ

Σ Σ[ ]even samples 2x n

[ ]odd samples

2 1x n +

0low band y

1high band y

10K −

11K −

-

-

-

-


Example: lifting implementation of 5/3 filters

[ ]even samples 2x n

( )12

z− + 114z −+

Σ

Σ

1/2[ ]

odd samples 2 1x n +

0low band y

1high band y

Verify by considering response to unit impulse in even and odd input channel.


Conjugate quadrature filters

Achieve aliasing cancelation by

Impulse responses

With perfect reconstruction: orthonormal subband transform!Perfect reconstruction: find power complementary prototype filter

( )( ) ( )

10 0

1 11 1

( ) ( )

( )

h z g z f z

h z g z zf z

−

− −

= ≡

= = − [Smith, Barnwell, 1986]

Prototype filter

[ ] [ ] [ ][ ] [ ] ( ) ( )

0 0

11 1 1 1k

h k g k f k

h k g k f k+

= − =

= − = − − +⎡ ⎤⎣ ⎦

( ) ( )( ) 222jjF e F e ω πω ±+ =


Wavelet bases

( ) ( )

( )( ) ( ) ( )

( )

2

2

2

Consider Hilbert space of finite-energy functions .

Wavelet basis for : family of linearly independent functions

2 2

that span . Hence any signal

m m mn

x t

t t nψ ψ− −

=

= −

∈

x

x

L

L

L ( )( ) [ ] ( )

2 can be written as

m mn

m ny n ψ

∞ ∞

=−∞ =−∞

= ∑ ∑x

L

“mother wavelet”


Multi-resolution analysis

( ) ( ) ( ) ( ) ( ) ( )

( ) ( )( ) { }

( ) ( )

2 1 0 1 2 2

2

0

Upward compl

Nested subspaces

eteness

Downward complete

ness

Self-similari t fy i

m

m

m

m

V V V V V

V

V

x t V

− −

∈

∈

⊂ ⊂ ⊂ ⊂ ⊂ ⊂ ⊂

=

=

∈

0

… …

∪∩

Z

Z

L

L

( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ){ } ( )

0 0

0

f 2

iff for all

There exists a "scaling function" with integer translates -

such that forms an orthonormal basis f

Translation i

or

nvaria

nce

mm

n

n n

x t V

x t V x t n V n

t t t n

V

ϕ ϕ ϕ

ϕ

−

∈

∈

∈ − ∈ ∈

=

Z


Multiresolution Fourier analysis

ω

( ){ } ( )span pn

p

nVϕ =

( ){ } ( )11span p pn n

Vϕ − −=

( ){ } ( )22span p pn n

Vϕ − −=

( ){ } ( )33span p pn n

Vϕ − −=

( ){ } ( )span pn

p

nWψ =

( ){ } ( )11span p pn n

Wψ − −=

( ){ } ( )22span p pn n

Wψ − −=

( ){ } ( )33span p pn n

Wψ − −=


Relation to subband filters( ) ( )

( ) [ ] ( ) ( )

( )

[ ] ( )

[ ] ( ) ( )

1

0 1

10 0

0 00

linear combinationof scaling functions in

Since , recursive definition of scaling function

2 2

Orthonormality

n ,

nn n

n

V

V V

t g n t g n t nϕ ϕ ϕ

δ ϕ ϕ

−

−

∞ ∞−

=−∞ =−∞

⊂

= = −

=

∑ ∑

[ ] ( ) ( ) [ ] ( ) ( )

[ ] [ ] ( ) ( ) [ ] [ ][ ]0

1 10 0 2

1 10 0 0 0

,

k unit norm and orthogonalto its 2-translates: correspondsto synthesis lowpass filter oforthonormal subband tra

2 , 2

i j ni j

i ji j i

g

g i t g j t dt

g i g j n g i g i n

ϕ ϕ

ϕ ϕ

∞− −

+−∞

− −

⎛ ⎞= ⎜ ⎟

⎝ ⎠

= − = −

∑ ∑∫

∑ ∑

nsform


Wavelets from scaling functions( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

( ){ } ( ) ( )

( ) [ ] ( ) ( )

1

1

0 0 1

11

linear combinationof scaling functions

is orthogonal complement of in

and

Orthonormal wavelet basis for

p p p

p p p p p

n

nn

W V V

W V W V V

W V

t g n t

ψ

ψ ϕ

−

−

−

∞−

=−∞

⊥ ∪ =

⊂

= ∑

( )

[ ] ( )

[ ] ( ) ( )( ){ } ( ){ } ( )

1

1

11 0

0 0 1

in

2 2

Using conjugate quadrature high-pass synthesis filter

The mutually orthonormal

1 1

Easy

functions, and , together span .

to e

nn

n

n

n n n

V

g n t n

g n g n

Vϕ

ϕ

ψ

−

−

∞

∞

∈

+

∈

=−

= −

= − − −⎡ ⎤⎣ ⎦

∑

Z Z

( )( ){ } ( )2

,

xtend to dilated versions of to construct orthonormal wavelet basis

for .

mn n m

tψ

ψ∈Z

L


Calculating wavelet coefficients for a continuous signal

Signal synthesis by discrete filter bank

Signal analysis by analysis filters h0[k], h1[k]Discrete wavelet transform

( ) ( ) ( ) [ ] ( ) ( ) [ ] ( ) ( )

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) [ ] ( )

( ) ( ) ( )

( ) [ ] ( )

( ) ( ) ( )1 1 1 1

0 0 0 0 00 0

1 1 1 1

0 1 1 1 10 1

Suppose continuous signal

Write as superposition of and

nn n

n ni j

x t V w t W

x t y n t n y n V

x t V w t W

x t y i y j

ϕ ϕ

ϕ ψ

∈ ∈

∈ ∈

∈ ∈

= − = ∈

∈ ∈

= +

∑ ∑

∑ ∑

Z Z

Z Z

( ) ( ) [ ] [ ] ( ) [ ] [ ]( ) [ ]00

0 1 10 0 1 1 2 2n

n i j

y n

y n g n i y j g n iϕ∈ ∈ ∈

⎛ ⎞= − + −⎜ ⎟

⎝ ⎠∑ ∑ ∑

Z Z Z


0h0g

1g

1h

0g

1g

0h

1h


( ) [ ]00y n

( ) [ ]10y n

( ) [ ]11y n

( ) [ ]20y n

( ) [ ]21y n

( ) [ ]10y n

( ) [ ]00y n

Sampling( )x t

( )Interpolation

tϕ

( )x t


Example: Daubechies wavelet, order 2

ϕ ψ

0h

0g

1h

1g


Example: Daubechies wavelet, order 9

ϕ ψ

0h

0g

1h

1g


Comparison JPEG vs. JPEG2000

Lenna, 256x256 RGBBaseline JPEG: 4572 bytes

Lenna, 256x256 RGBJPEG-2000: 4572 bytes


Comparison JPEG vs. JPEG2000

JPEG with optimized Huffman tables8268 bytes

JPEG20008192 bytes

multiresolution image processingee225b/fa12/lectures/multi... · bernd girod: ee368 digital image...

Documents