WaveletsMarialuce Graziadei

1. A brief summary

2. Vanishing moments

3. 2D-wavelets

4. Compression

5. De-noising

1

1. A brief summary

• φ(t): scaling function.For φ the 2-scale relation hold

φ(t) =∞

∑

k=−∞

pkφ(2t − k) (t ∈ IR)

• ψ(t): mother wavelet. For ψ the 2-scale relation hold

ψ(t) =∞

∑

k=−∞

qkφ(2t − k) (t ∈ IR)

• The decomposition for φ reads

φ(2t−k) =∞

∑

m=−∞h2m−kφ(t−m)+g2m−kψ(t−m) (t ∈ IR)

2

2. Vanishing moments

Moments of a mother-wavelet

mµ =∫ ∞

−∞tµψ(t) dt .

Goal: to show that, if the mother-wavelet has successive mo-ments equal to zero (for a fixed b) the wavelet coefficients de-crease ’quickly’ when a decreases.

Assumptions:

• the function f (t) is (k−1) times continuously differentiable;

• f (k)(t) has jumps at most in a finite number of points.

Then f (t) can be expanded as

f (t + b) = f (b)+ f ′(b)t + ...+f (k−1)(b)(k − 1)!

tk−1 + t kr(t),

where r(t) is piecewise continuous and bounded, with at mosta finite number of jumps.

3

Wavelet coefficients

< f, ψa,b > =1

√a

∫ ∞

−∞f (t)ψ((t − b)/a) dt =

√a

∫ ∞

−∞f (b + at)ψ(t) dt

=√

a∫ ∞

−∞( f (b)+ f ′(b)(at))+ ...+

f (k−1)(b)(k − 1)!

(at)k−1)ψ(t) dt

+ ak+ 12

∫ ∞

−∞tkr(at)ψ(t) dt

If the first (k − 1) moments of the mother-wavelet are equal tozero

< f, ψa,b > = ak+ 12

∫ ∞

−∞tkr(at)ψ(t) dt

This means that

< f, ψa,b >= O(ak+ 12 ) (a → 0)

4

Example

The function

f (t) = sin (π |t −12|),

is not-differentiable in t = 12 .

The following table shows the wavelet coefficients, computedfor different values of a and for b = 0.5 and b = 0.6, using theMexican hat. The convergence factors (log2) are also reported.

a b = 0.5 k + 12 b = 0.6 k + 1

20.128 −6.6277 1.329 −4.0958 2.73890.064 −2.6678 1.4559 −0.6136 4.33820.032 −0.9724 1.4888 0.0303 2.17960.016 −0.3465 1.4987 0.0067 2.49860.008 −0.1226 1.5050 0.0012 2.49970.004 −0.0432 0.0002

For b = 0.5 the wavelet coefficients go to zero more slowlythan for b = 0.6!!!

5

3. 2D-wavelets

Notation:

f (x − k, y − l) = fk,l(x, l)

are the translations of f .

f (x, y) → f (2nx, 2n y)

are the dilatations of f .

The definition of a 2D Multi-Resolution Analysis (MRA) issimilar to a 1D-MRA.

If a sequence of subspaces (Vn) satisfies the following proper-ties

a) Vn ⊂ Vn+1 (n ∈ ZZ),

b)⋃

n∈ZZVn = L2(IR2),

c)⋂

n∈ZZVn = {0},

d) f (x, y) ∈ Vn ⇔ f (2x, 2y) ∈ Vn+1,

e) f (x, y) ∈ V0 ⇒ fk,l(x, y) ∈ V0.

then it is called a MRA of L2(IR2).

6

φ is a scaling function for the MRA

• it is continuous, with a compact support in IR2, with possiblejumps on the boundary of the support;

• the integer translations of φ, φk,l , form a Riesz-basis for V0.

Scaling functionsSufficient conditions for a compactly supported function φ tobe a scaling function for an MRA

1. There exists a sequence of numbers pk,l (only a finite num-ber differs from zero) such that

φ(x, y) =∑

k,l

pk,lφ(2x − k, 2y − l) ((x, y) ∈ IR2).

(2-scale relation).

2. Introduce the 2D-autocorrelation function of φ as

ρ(k, l) :=∫∫ ∞

−∞φ(x, y)φ(x + k, y + l) dx dy

and the 2D-Riesz-function as

Rφ(z1, z2) =∑

k,l

ρ(k, l) zk1 zk

2 ((z1, z2) ∈ C| 2)

The Riesz function Rφ(z1, z2) is positive for |z1| = |z2| = 1.

3. The translation of the function φ are such that

∑

k

φ(t − k) ≡ 1. (Partition of the unity).

7

If φ has these properties, the following hold

• The reference space V0 consists of functions f (x, y) that canbe expressed as

f (x, y) =∑

k,l

ak,lφk,l(x, y) ((x, y) ∈ IR2)

• Whatever is n, the space Vn consists of functions f such that

f (x, y) =∞

∑

k=−∞

ak,lφk,l(2nx, 2n y) ((x, y) ∈ IR2)

As in 1D-MRA, the goal is to build the detail space (Wn) suchthat

Vn+1 = Vn ⊕ Wn

The space Wn is built from a mother-wavelet ψ .

The functions ψ(2nx − k, 2n y − l) form a Riesz-basis for Wn.

8

In 2-D more than one wavelet is necessary to span W0.

Example: 2-D Haar wavelets

φ1(x): 1-D Haar scaling function;

ψ1(x): 1-D Haar wavelet.

The 2-D Haar scaling function is defined from the 1-D Haarscaling function as

φ(x, y)=φ1(x)φ1(y)

and, if one want to fill V0 to obtain V1:

ψ (1)(x, y) = ψ1(x)φ1(y) = φ(2x, y)− φ(2x − 1, y),

ψ (2)(x, y) = φ1(x)ψ1(y) = φ(x, 2y)− φ(x, 2y − 1),

ψ (3)(x, y) =ψ1(x)ψ1(y) = φ(2x, 2y)− φ(2x − 1, 2y)+ φ(2x − 1, 2y − 1)− φ(2x, 2y − 1).

These three functions belong to V1 and are orthogonal to V0, sothey belong to W0.

9

00.2

0.40.6

0.81 0

0.2

0.4

0.6

0.8

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

00.2

0.40.6

0.81 0

0.2

0.4

0.6

0.8

1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Figure 1: 2-D Haar scaling function φ and Haar wavelet ψ (1)

00.2

0.40.6

0.81 0

0.2

0.4

0.6

0.8

1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

00.2

0.40.6

0.81 0

0.2

0.4

0.6

0.8

1

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Figure 2: Haar wavelet ψ (2)and Haarwaveletψ (3)

10

The function ψ (1), ψ (2) and ψ (3) must satisfy the following

•

W0 = W 10 ⊕ W 2

0 ⊕ W 30

• the translation of ψ ( j) are a Riesz basis for W ( j)0

Then

φ(x, y) =∑

k,l

pk,lφ(2x − k, 2y − l) ((x, y) ∈ IR2).

ψ ( j)(x, y) =∑

k,l

q( j)k,l φ(2x − k, 2y − l) ((x, y) ∈ IR2).

These relations, taking into account that

V1 = V0 ⊕ W 10 ⊕ W 2

0 ⊕ W 30

allows to find the coefficients h jk , with ( j = 0, 1, 2, 3) such

that

φm,n(2x, 2y) =∑

k,l

h02k−m,2l−nφk,l(x, y)+

3∑

j=1

∑

k,l

h( j)2k−m,2l−nψ

( j)k,l (x, y).

11

Furthermore, for all functions f ∈ L. 2(IR2) there are three se-ries (a( j)

r,k,l), ( j = 1, 2, 3) such that

f (x, y) =3

∑

m=1

∞∑

r=−∞

∑

k,l

a(m)r,k,lψ(m)(2r x − k, 2r y − l).

Filterbanks

As in 1-D, one can decompose f ∈ V0 using a filterbank. fcan be written as

f (x, y) =∑

k,l

a0k,lφk,l(x, y).

But f = f−1+g−1+g−2+g−3 with f−1 ∈ V−1 and g−1, g−2, g−3 ∈W−1.

These functions can be represented as

f−1(x, y) =∑

k,l

a−1k,l φk,l(x/2, y/2);

g−1(x, y) =∑

k,l

d−1,k,lψ(1)k,l (x/2, y/2);

g−2(x, y) =∑

k,l

d−2,k,lψ(2)k,l (x/2, y/2);

g−3(x, y) =∑

k,l

d−3,k,lψ(3)k,l (x/2, y/2);

12

Then

a−1k,l =

∑

u,v

h02k−u,2l−va

0u,v,

d−m,k,l =∑

u,v

hm2k−u,2l−va

0u,v (m = 1, 2, 3).

The reconstruction coefficients can be computed in the sameway as in the 1-D case.

13

Filterbanks

-

-

-

-

-

H1(z)

H2(z)

a0k,l

-

-

H0(z)

H3(z) -

-

(d−1,k,l)

(a−1k,l )

(d−2,k,l)

(d−3,k,l)

?

?

?

?

-

-

-

-

Figure 3: Decomposition

14

-

-

(d−1,k,l)

(a−1k,l )

-

-

(d−2,k,l))

(d−3,k,l))

6

6

6

6

-

-

-

-

Q1(z)

Q2(z)

P(z)

Q3(z)

-

-

-

-

+ -(a0

k,l)

Figure 4: Reconstruction

15

Analysis of 2-D images

Discrete images: arrays f of M rows and N columns

f =

f1,M f2,M . . . fN,M

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

f1,2 f2,2 . . . fN,2

f1,1 f2,1 . . . fN,1

The 2-D wavelet transform of such an image can be performedin two steps.

Step 1. Perform a 1-D wavelet transform on each row of f,thereby producing a new image.

Step 2. Perform a 1-D wavelet transform on each column of ofthe matrix obtained with the Step 1.

A 1-level wavelet transform can be therefore symbolized asfollows:

f →

a1 | h1

− −v1 | d1

where the sub-images h1, d1, a1 and v1 each have M/2 rowsand N/2 columns.

16

Sub-image a1: it is created computing trends along rows of f,following by computing trends along columns, so it is an anaveraged, lower resolution version of f.

Sub-image h1: it is created computing trends along rows off, following by computing fluctuation along columns. Conse-quently it detects horizontal edges.

Sub-image v1: it is created like h1, but inverting the role ofrows and columns: it detects vertical edges.

Sub-image d1: it tends to emphasize diagonal features, becauseit is created from fluctuation along both rows and columns.

20 40 60 80 100 120

20

40

60

80

100

120

17

Approximation A1

50 100 150

20

40

60

80

100

120

140

160

Orizontal detail H1

50 100 150

20

40

60

80

100

120

140

160

Vertical detail H1

50 100 150

20

40

60

80

100

120

140

160

Diagonal detail D1

50 100 150

20

40

60

80

100

120

140

160

18

4. De-noising

• The transmission of a signal over some distance often im-plies the contamination of the signal itself by noise.

• The term noise refers to any undesirable change that has al-tered the value of the original signal.

• The simplest model for acquisition of noise by a signal is theadditive noise , which has the form

f = s + n,

where f is the contaminated signal, s is the original signaland n is the noise.

The most common types of noise are the following:

1.Random noise. The noise signal is highly oscillatory aboveand below an average mean value.

2.Pop noise. The noise is perceived as randomly occurring,isolated ’pops’. As a model for this type of noise we add a fewnon-zero values to the original signal at isolated locations.

3.Localized random noise. It appears as in type 1, but onlyover a (some) short segment(s) of the signal. This can occurwhen there is a short-lived disturbance in the environment dur-ing transmission of the signal.

19

De-noising procedure principles

1. Decompose. Choose a wavelet and a level N . Compute thewavelet decomposition of the signal at level N.

2. Threshold detail coefficients. For each level from 1 to N ,select a threshold and apply soft or hard thresholding to the de-tail coefficients.

3. Reconstruct.

How to choose a threshold?

The most frequently encountered noise in transmission is thegaussian noise. It can be characterised by a the mean µ and bythe standard deviation σ .

• Assume that µ = 0.

• The gaussian nature of the noise is preserved during the trans-formation ⇒ the wavelet coefficients are distributed ac-cording to a Gaussian curve having µ = 0 and standarddeviation σ .

• From the theory, if one chooses

T = 4.5σ,

the 99.99% of the wavelet coefficients will be eliminated.

Usually the finest detail consist almost entirely of noise. Itsstandard deviation can be assumed as a good estimate for σ .

20

Soft or hard thresholding?

Let T denote the threshold, x the wavelet transform values andH (x) the transform value after the thresholding.

Hard thresholding means

H (x) =

{

x if |x | ≥ T

0 if |x | ≤ T

Soft thresholding means

H (x) =

{

sign(x)(|x | − T ) if |x | ≥ T

0 if |x | ≤ T

21

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Figure 5: Original signal, hard thresholding and soft thresholding

1. Hard thresholding exaggerates small differences in trans-form values that have magnitude near the threshold value T

2. Soft thresholding has risk of oversmoothing

22

5. Compression

Compression means converting the signal(s) data into a newformat that requires less bit to be transmitted.

Lossless compressionIt is completely error free (ex: techniques that produce .zipfiles). The maximum compression ratio are 2 : 1.

Lossy compressionIt is used when inaccurancies can be accepted because quiteimperceptible. The compression ratio vary from 10 : 1 to100 : 1 when more complex techniques are used. Waveletsare applied in this field.

Compression procedure

• Perform wavelet transform of the signal up to level N ;

• Set equal to zero all values of the wavelet coefficients whichare insignificant, i.e. which are below some thresholdvalue;

• Transmit only the significant, non-zero values of the trans-form obtained from Step 2;

• Reconstruct the signal.

23

How to choose a threshold?

Suppose that L j are the transform coefficients. One can putthem in decreasing order

L1 ≥ L2 ≥ L3 ≥ ... ≥ L M

The energy of a signal is

E f =M

∑

j=1

L2j

Compute the cumulative energy profile

L21

E f,

L21 + L2

2

E f, ...,

L21 + L2

2 + L23 + ...+ L2

N

E f, ..., 1.

The threshold T is chosen according with the amount of energythat we want to retain. If the term

L21 + L2

2 + L23 + ...+ L2

N

E f

is such that a sufficient amount of energy is kept, then all thecoefficients smaller than L N can be put equal to zero.

24