introduction to multiresolution analysis (mra)bigdft.org/images/1/1b/day-2-rs-fk-mra.pdf · outline...
TRANSCRIPT
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation References
Introduction to Multiresolution Analysis (MRA)
R. SchneiderF. Kruger
TUB - Technical University of Berlin
November 22, 2007
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 1 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesOutline
Outline
Introduction and Example
Multiresolution Analysis
Discrete Wavelet Transform (DWT)
Finite Calculation
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 2 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
Introduction and Example
Multiresolution Analysis
Discrete Wavelet Transform (DWT)
Finite Calculation
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 3 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
goal approximation of functions (e.g. signals, images,orbitals)
idea coarse approximation (trend) + fine improvement(detail) with detail << trend
imagination building a house. start with big pieces and fill in withmiddle sized and at the end with little pieces
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 4 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15detail 1
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 5 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15detail 2
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 6 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2
0.25detail 3
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 7 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.2
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
0.2detail 4
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 8 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 14.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9step 6
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.1
−0.08
−0.06
−0.04
−0.02
0
0.02
0.04
0.06
0.08
0.1detail 5
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 9 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
Details
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−2
−1.5
−1
−0.5
0
0.5
1
1.5
2x 10
−3 detail 10
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 10 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
In the example a very simple set of basis functions is used:
ϕ(x) =1[0,1)(x) =
{1, x ∈ [0, 1)0, else
−1 −0.5 0 0.5 1 1.5 2−0.5
0
0.5
1
1.5Haar function
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 11 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
ϕjk(x) =2j/21[2−jk,2−j(k+1))(x) =
{2j/2, x ∈ [2−jk, 2−j(k + 1))0, else
=2j/2ϕ(2jx− k).
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−0.5
0
0.5
1
1.5
2
2.5Haar function, j = 2, k = 3
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 12 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
The details are given by the functions
ψjk(x) =2j/2ψ(2jx− k)
with
ψ(x) =
1, x ∈ [0, 0.5)−1, x ∈ [0.5, 1)0, else.
−1 −0.5 0 0.5 1 1.5 2−1.5
−1
−0.5
0
0.5
1
1.5Haar wavelet
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5Haar wavelet, j = 2, k = 3
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 13 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesIntroduction and Example
Refinement Equation
ϕ is called Haar function and ψ is called the Haar wavelet. Theysatisfy the so-called refinement equations :
ϕjk(x) =2−1/2(ϕj+1
2k (x) + ϕj+12k+1(x))
ψjk(x) =2−1/2(ϕj+1
2k (x)− ϕj+12k+1(x))
−1 −0.5 0 0.5 1 1.5 2−0.5
0
0.5
1
1.5Refinement for the Haar function
φ2−1/2 φ0
1
2−1/2 φ11
−1 −0.5 0 0.5 1 1.5 2−1.5
−1
−0.5
0
0.5
1
1.5Refinement for Haar wavelet
ψ2−1/2 φ0
1
2−1/2 φ11
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 14 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Introduction and Example
Multiresolution Analysis
Discrete Wavelet Transform (DWT)
Finite Calculation
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 15 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Multiresolution Analysis
Given a function
ϕ ∈ L2(R) = {f | ‖f‖2 =( ∫
R|f(x)|2dx
)1/2<∞}.
We consider the shifts and dilatations of ϕ :
ϕjk(x) = 2j/2ϕ(2jx− k), j, k ∈ Z.
We write
Vj = span{ϕjk | k ∈ Z}.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 16 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
If every f ∈ L2(R) can be arbitrarily accurately approximated byϕj
k’s, i.e.
∞⋃j≥j0
Vj = L2(R)
holds and ϕ fulfills a refinement equation
ϕ =∑k∈Z
hkϕ1k
then we say that ϕ or the V ′j s, respectively, build a multi resolutionanalysis (MRA).
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 17 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Orthonormal Wavelets
Because of the refinement equation it holds Vj ⊂ Vj+1 for everyj ∈ Z. Therefore there exists the orthogonal space Wj of Vj inVj+1. Therefore
Vj ⊥Wj , Vj ⊕Wj = Vj+1.
Wj is called the detail space or the wavelet space for Vj . Afunction ψ that satisfies
1.∫
R ψ(x)dx = 02. {ψ(· − k) | k ∈ Z} is an orthonormal basis of W0
is called (orthonormal) wavelet or mother wavelet for the functionϕ. ϕ is also called scaling function or generator function or fatherwavelet. The Haar wavelet is a wavelet for the Haar function, forexample.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 18 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
If the translations of ψ are not orthonormal we need biorthogonalwavelets. But we do not go into details for this case.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 19 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Multi Levels
Let J be the level at which we want to approximate, i.e. weproject into the space VJ . Then we have
VJ =VJ−1 ⊕WJ−1
=VJ−2 ⊕WJ−2 ⊕WJ−1
...
=V0 ⊕J−1⊕j=0
Wj .
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 20 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
If we go to infinity we have
L2(R) = V0 ⊕∞⊕
j=0
Wj .
We can imagine that we start at the very coarse level 0 andimprove the result by successively adding the finer becomingdetails.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 21 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Filters
Because of the fact that Vj ⊂ Vj+1 and Wj ⊂ Vj+1 in a MRA wehave the refinement equations
ϕjk =
∑l
hlϕj+12k+l
ψjk =
∑l
glϕj+12k+l.
(hl)l∈Z and (gl)l∈Z are called filters. If ϕ has compact support hhas finite length. If additionally ψ has compact support g has alsofinite length.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 22 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesMultiresolution Analysis
Reconstruction
If {ϕjk | k ∈ Z} and {ψj
k | k ∈ Z} are orthonormal bases, i.e.
〈ϕjk, ϕ
jl 〉 =
∫Rϕj
k(x)ϕjl (x)dx =δk,l
〈ψjk, ψ
jl 〉 =δk,l
we have the reconstruction
ϕj+1k =
∑l
hk−2lϕjl +
∑l
gk−2lψjl .
There is also a very simple correlation between g and h:
gk = (−1)1−kh1−k
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 23 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
Introduction and Example
Multiresolution Analysis
Discrete Wavelet Transform (DWT)
Finite Calculation
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 24 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
Orthogonal Projection
Assume that {ϕk | k ∈ Z} and {ψk | k ∈ Z} are orthonormal.Given f ∈ L2(R). We consider the orthogonal projections PJ ontoVJ and QJ onto WJ . Let
λjk =〈ϕj
k, f〉 =∫
Rϕj
k(x)f(x)dx and
µjk =〈ψj
k, f〉.
Then it holds
fJ = PJf =∑
k
λJkϕ
Jk ∈ VJ
QJf = (PJ+1 − PJ)f =∑
k
µJkψ
Jk ∈WJ .
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 25 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
We can easily get the coefficients in the coarser levels and thedetail spaces by using successively the synthese equation.
fJ =∑
k
λJkϕ
Jk
=∑
k
λJk (
∑l
hk−2lϕJ−1l +
∑l
gk−2lψJ−1l )
=∑
l
λJ−1l ϕJ−1
l +∑
l
µJ−1l ψJ−1
l
...
=∑
l
λ0l ϕ
0l +
J−1∑j=0
∑l
µjlψ
jl
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 26 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
The occurring coefficients are given by
λjl =
∑k
λj+1k hk−2l and (1)
µjl =
∑k
λj+1k gk−2l, j = 0, . . . , J − 1. (2)
(1) and (2) can be written as(λj
µj
)= Tλj+1.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 27 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
Discrete Inverse Wavelet Tansform (DIWT)
Given the coefficients λjk and µj
k we get the cofficients λj+1k back
by
λj+1k =
∑l
hk−2lλjl +
∑l
gk−2lµjl .
This describes again a linear transformation
λj+1 = T−1
(λj
µj
)= T T
(λj
µj
).
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 28 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
Note: Because of the D(I)WT it is not really necessary to knowexplicitly the functions ϕ and ψ. It is sufficient to know the filtersh and g.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 29 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesDiscrete Wavelet Transform (DWT)
Schemes
DWT DIWT
λJ λ0 µ0
↓ ↘ ↓ ↙λJ−1 µJ−1 λ1 µ1
↓ ↘ ↓ ↙λJ−2 µJ−2 λ2 µ2
↓ ↘ ↓ ↙...
......
...↓ ↘ ↓ ↙λ1 µ1 λJ−1 µJ−1
↓ ↘ ↓ ↙λ0 µ0 λJ
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 30 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesFinite Calculation
Introduction and Example
Multiresolution Analysis
Discrete Wavelet Transform (DWT)
Finite Calculation
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 31 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesFinite Calculation
In the previous chapter we still had infinite many coefficientsλj
k, µjk. For practical calculations we have to make their size finite.
There are mainly two ways to do this.
1. Zeropadding: You consider only a finite region and assumethat all coefficients out of this region are zero.
2. Periodizing: You assume that your data is periodic and youcalculate only on one period.
Then the D(I)WT is a finite transform if the filters are finite. Thenumber of arithmetic operations for one transformation is O(N) ifN is the size of the input data. To transform on J levels you haveO(J ·N) arithmetic operations.
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 32 / 33
Outline Introduction and Example Multiresolution Analysis Discrete Wavelet Transform (DWT) Finite Calculation ReferencesReferences
References
S. Mallat, A Wavelet Tour of Signal Processing, 2nd. ed., AcademicPress, 1999
R. Schneider F. Kruger Introduction to Multiresolution Analysis (MRA) November 22, 2007 33 / 33