Gloria Menegaz 1

2D Wavelets

Hints on advanced Concepts

Gloria Menegaz 2

Advanced concepts

•  Wavelet packets

•  Laplacian pyramid

•  Overcomplete bases –  Discrete wavelet frames (DWF)

•  Algorithme à trous –  Discrete dyadic wavelet frames (DDWF)

•  Overview on edge sensitive wavelets –  Contourlets

Gloria Menegaz 3

Wavelet packets

Gloria Menegaz 4

Wavelet packets Both the approximation and the detail subbands are further decomposed

Gloria Menegaz 5

Packet tree

Gloria Menegaz 6

Wavelet Packets

time

frequ

ency

η

η /2

incr

easi

ng th

e sc

ale

η /4

Gloria Menegaz 7

Laplacian Pyramid

Low-pass Interpolation ↓2 ↑2

coarser version

prediction

- residual

residuals

Gloria Menegaz 8

Overcomplete bases

Gloria Menegaz 9

Translation Covariance

Translation DWT

DWT Translation

Signal

Signal

Wavelet coefficients

Wavelet coefficients

{ }{ } { }{ }),(),( yxfTDWTyxfDWTT ≠

Translation covariance

If translation covariance does not hold:

{ }{ } { }{ }),(),( yxfTDWTyxfDWTT =

NOT good for signal analysis

Gloria Menegaz 10

Rationale

•  In pattern recognition it is important to build representations that are translation invariant. This means that when the pattern is translated the descriptors should be translated but not modified in the value.

•  CWTs and windowed FT provide translation covariance, while sampling the translation parameter might destroy translation covariance unless some conditions are met.

•  Intuition: either the sampling step is very small compared to the translation or the translation is a multiple of the sampling step.

Gloria Menegaz 11

Translation invariant representations

Gloria Menegaz 12

Translation covariance

•  The signal descriptors should be covariant with translations –  Continuous WT and windowed FT are translation covariant.

–  Wavelet frames (DWF) are constructed by sampling continuous transforms over uniform time grids.

–  The sampling grid removes the translation covariance because the translation factor τ is a priori not equal to the translation interval

),(1)(),(

)(1)(),(

)()(

suWfdtsut

stfsuWf

ufdtsut

stfsuWf

tftf

s

τψτ

ψψ

τ

τ

τ

−=⎟⎠

⎞⎜⎝

⎛ −−=

∗=⎟⎠

⎞⎜⎝

⎛ −=

−=

∫

∫

Gloria Menegaz 13

Sampling and translation covariance

τ

If the translation does not correspond to a multiple of the sampling step, a different set of samples will be obtained by keeping the same sampling grid

Gloria Menegaz 14

Sampling and translation covariance

τ

Gloria Menegaz 15

Translation invariant representation

•  If the sampling interval is small enough than the samples of are approximately translated when f is shifted.

•  Translation covariance holds if namely it is a multiple of the sampling interval

0ja u ( )ja

f tψ∗

0jku aτ =

Wftau(u,aj)

Wf(u,aj)

u

u τ 0

ja u

Gloria Menegaz 16

Translation invariant representation

•  Uniformly sampling the translation parameter destroys covariance unless the translation is very small

•  Translation invariant representations can be obtained by sampling the scale parameter s but not the translation parameter u

Gloria Menegaz 17

Dyadic Wavelet Transform

•  Sampling scheme –  Dyadic scales –  Integer translations

–  If the frequency axis is completely covered by dilated dyadic wavelets, then it defines a complete and stable representation

•  The normalized dyadic wavelet transform operator has the same properties of a frame operator, thus both an analysis and a reconstruction wavelets can be identified

•  Special case: algorithme à trous

)(22

1)()2,( 2 ufdtuttfuWf jjjj ψψ ∗=⎟

⎠

⎞⎜⎝

⎛ −= ∫

⎟⎠

⎞⎜⎝

⎛ −=−= jj

ttt jj22

1)()( 22 ψψψ

Gloria Menegaz 18

Algorithme à trous

•  Similiar to a fast biorthogonal WT without subsampling

•  Fast dyadic transform –  The samples of the discrete signal a0[n] are considered as averages of some function

weighted by some scaling kernels φ(t-n)

–  For any filter x[n], we denote by xj[n] the filters obtained by inserting 2j-1 zeros between each sample of x[n] → create holes (trous, in French)

)2,()(),(][

)(),(][

)(),(][

2

22

0

jnWfnttfnd

nttfna

nttfna

jj

jj

=−=

−=

−=

>

≥

ψ

ϕ

ϕ

grid integer the over 0j for computed are tscoefficien eletdyadic wav The

denote we0jany For

][][ nxnx jj −=

Gloria Menegaz 19

Algorithme a trous

•  Proposition

The dyadic wavelet representation of a0 is defined as the set of wavelet coefficients up to the scale 2J plus the remaining low-pass frequency information aJ

–  Fast filterbank implementation

( )][~][~21][

][][

][][0

11

1

1

ngdnhana

ngand

nhanaj

jjjjj

jjj

jjj

∗+∗=

∗=

∗=

≥

++

+

+

any For

{ }1

,J j j Ja d

≤ ≤

⎡ ⎤⎢ ⎥⎣ ⎦

Gloria Menegaz 20

Analysis

~ hj ~ hj+1

~ gj ~ gj+1

aj aj+1

dj+1

aj+2

dj+2

2j-1”trous” No subsampling!!

2102

2101

210

000000][~00][~

][~

hhhnh

hhhnh

hhhnh

=

=

=

Gloria Menegaz 21

Synthesis

hj+1

gj+1

aj+2

dj+2

+ hj

gj

aj+1

dj+1

+ aj

Overcomplete wavelet representation: [aJ, {dj}1≤j≤J]

Gloria Menegaz 22

Algorithme a trous

g(z2)

h(z2)

a2f

d2f

g(z4)

h(z4)

a3f

d3f

g(z)

h(z) s(z) a1f

d1f

Gloria Menegaz 23

DWT vs DWF

•  DWT –  Non-redundant –  Signal il subsampled –  Not translation invariant –  Total number of coefficients:

NxNy

•  Compression

•  DWF –  Redundant (in general) –  Signal is not subsapled –  Filters are upsampled –  Translation invariant –  Total number of coefficients:

(3J+1)NxNy

•  Feature extraction

Gloria Menegaz 24

Discrete WT vs Dyadic WT

Original

HL LH HH LL

DWT

Dyadic WT

Gloria Menegaz 25

Example 1

Gloria Menegaz 26

Example 2

Gloria Menegaz 27

Rotation covariance

•  Oriented wavelets

–  In 2D, a dyadic WT is computed with several wavelets which have different spatial orientations

•  We denote

•  The WT in the direction k is defined as

–  One can prove that this is a complete and stable representation if there exist A>0 and B>0 such that

Kkk yx ≤≤1)},({ψ

⎟⎠

⎞⎜⎝

⎛= jj

kj

k yxyxj2,

221),(2 ψψ

),(),(),,()2,,( 22 vufvyuxyxfvufW kkjkjj ψψ ∗=−−=

{ } ( )2

2

1

ˆ( , ) 0,0 , 2 ,2K

j jx y x y

k jA Bω ω ψ ω ω

+∞

= =−∞

∀ ∈ − ≤ ≤∑ ∑R

Gloria Menegaz 28

Oriented wavelets

–  Then, there exists a reconstruction wavelet family such that

•  Gabor wavelets

–  In the Fourier plane the energy of the Gabor wavelet is mostly concentrated

f (x, y) = 12 j

W k f (w, y,2 j )k=1

K

∑j=−∞

+∞

∑ ∗ ψ2 jk (x, y)

ψk x, y( ) = g(x, y)e−iη(xcosαk + ysinα

k)

g(x, y) = 12π

e−x2+y2

2

!

"#

$

%&

→Gaussian → Gabor wavelets

ψ2 jk (ωx ,ω y ) = 2 j g(2 jωx −ηcosαk ,2

jω y −ηsinαk )

ωy

ωx

( )2 cos ,2 sinj jk kη α η α− − in a neighborhood proportional to 2-j.

j increases

Gloria Menegaz 29

Gabor wavelets

( )

( ) ( )

( cos sin ), ( , ) ,

( , ) Gaussian Gabor waveletsthen

, cos , sinx y x y

i x yk k kx y g x y eg x y

k g k k

η α αψ

ψ ω ω ω η α ω η α

− +=

→

= − −

( ),x yg ω ωkα

( )cos , sinx yg k kω η α ω η α− −

ωy

ωx

η2 j

Gloria Menegaz 30

Gabor wavelets

2ˆ ˆ( , ) 2 (2 cos ,2 sin )jk j j jgx y x k x kψ ω ω ω η α ω η α= − −

( ),x yg ω ω

η2 j−1

kα

ωy

ωx

Gloria Menegaz 31

Gabor wavelets

•  Other directional wavelet families –  Dyadic Frames of Directional Wavelets [Vandergheynst 2000] –  Curvelets [Donoho&Candes 1995] –  Steerable pyramids [Simoncelli-95] –  Contourlets [Do&Vetterli 2002]

( ) ( )

( )

2 2

1

cos sin2 2j k kj j

k ktg

η ηρ α α

θ α−

= + =

=

jρ

αk

2-j

ωy

ωx

Gloria Menegaz 32

Dyadic Frames of Directional Wavelets

Pierre Vandergheynst (LTS-EPFL) http://people.epfl.ch/pierre.vandergheynst

Gloria Menegaz 33

Dyadic Directional WF

Gloria Menegaz 34

Dyadic Directional Wavelet Frames

•  Directional selectivity at any desired angle at any scale –  Not only horizontal, vertical and diagonal as for DWT and DWF –  Rotation covariance for multiples of 2π/K

•  Recipe –  Build a family of isotropic wavelets such that the Fourier transform of the mother wavelet

expressed in polar coordinates is separable

–  Split each isotropic wavelet in a set of oriented wavelets by an angular window •  Express the angular part Θ(φ) as a sum of window functions centered at θk

( ) ( ) ( )

( )22

^,

yx ωωω

ϕωϕω

+=

ΘΓ=Ψ

ωy

ωx

ω

Gloria Menegaz 35

Partitions of the F-domain

2π ϕ

( )θ ϕ

Gloria Menegaz 36

Dyadic Directional Wavelet Frames

4 orientations (K=4)

Gloria Menegaz 37

Dyadic Directional Wavelet Frames

DDWF

Feature Images or Neural images

Orientation

Scale

Scaling function

Properties •  Overcomplete

• Translation covariance

•  Rotation covariance for given angles

Gloria Menegaz 38

Contourlets Brief overview

Minh Do, CM University Martin Vetterli, LCAV-EPFL

Gloria Menegaz 39

Contourlets

•  Goal –  Design an efficient linear expansion for 2D signals, which are smooth away from

discontinuities across smooth curves –  Efficiency means Sparseness

–  [Do&Vetterli] Piecewise smooth images with smooth contours –  Inspired to curvelets [Donoho&Candes]

Gloria Menegaz 40

Curvelets

•  Basic idea –  Curvelets can be interpreted as a grouping of nearby wavelet basis functions into linear

structures so that they can capture the smooth discontinuity curve more efficiently

–  More efficient in capturing the geometry -> more concise (sparse) representation

2-j

2-j

2-j

c2-j/2

wavelets curvelets

Parabolic scaling

Gloria Menegaz 41

M. Do and M. Vetterli, The Contourlet Transform: An Efficient Directional Multiresolution Image Representation, IEEE-TIP

Gloria Menegaz 42

Curvelets

Embedded grids of approximations in spatial domain. Upper line represents the coarser scale and the lower line the finer scale. Two directions (almost horizontal and almost vertical) are considered. Each subspace is spanned by a shift of a curvelet prototype function. The sampling interval matches with the support of the prototype function, for example width w and length l, so that the shifts would tile the R2 plan. The functions are designed to obey the key anisotropy scaling relation:

width ∝ length2

Close resemblance with complex cells’ (orientation selective RF)!

Gloria Menegaz 43

For a given rate, a better representation of edges is reached

Gloria Menegaz 44

Summary of useful relations

•  If f is real

( )

*

* ( )

* ( ) 1

ˆ ˆ( ) ( )ˆ ˆ( ) ( )ˆ ˆ ˆ( ) ( ) ( )

ˆ ˆˆ( ) ( ) ( ) [ ] ( 1) [ ]ˆ ˆˆ( ) ( ) ( ) [ ] ( 1) [1 ]

j

j

j

j n

j j j n

f f ef f ef f e f

t f f e t n f ng e f e f e g n f n

ω

ω π

ω

ω π

ω ω ω π

ωω πω ω

ω ω πω ω π

+

−

− + −

− − − + −

=

+ =

− = =

= + = → = − −

= + = → = − −

Gloria Menegaz 45

Conclusions

•  Multiresolution representations are the fixed point of vision sciences and signal processing

•  Different types of wavelet families are suitable to model different image features –  Smooth functions -> isotropic wavelets –  Contours and geometry -> Curvelets

•  Adaptive basis –  More flexible tool for image representation –  Could be related to the RF of highly specialized neurons

Gloria Menegaz 46

References

•  A Wavelet tour of Signal Processing, S. Mallat, Academic Press

•  Papers –  A theory for multiresolution signal decomposition, the wavelet representation, S. Mallat,

IEEE Trans. on PAMI, 1989 –  Dyadic Directional Wavelet Transforms: Design and Algorithms, P. Vandergheynst and

J.F. Gobbers, IEEE Trans. on IP, 2002 –  Contourlets, M. Do and M. Vetterli (Chapter)