diego milone -...

Wavelet packetsDiego Milone

Análisis y Procesamiento Avanzado de Señales

Doctorado en Ingeniería, FICH-UNLMaestría en Ingeniería Biomédica, FI-UNER

Maestría en Computación Aplicada a la Ciencia y la Ingeniería, FICH-UNL

30 de octubre de 2015

Introduction WP definitions Splitting algorithm More basis...

Outline

Introduction

Wavelet packetsDefinition of basic wavelet packetsGeneral wavelet packetsWavelet packet treesSupport and localization

The splitting algorithmSplitting the treeFinite signals and computational complexity

More basis...Some (fixed) wavelet packets basesLocal cosine bases


Linear expansions in signal processing

We write a signal f belonging to a space S as

f =∑i

αiψi

where the set {ψi}i∈Z is complete in S.

[3]


Linear expansions in signal processing

We write a signal f belonging to a space S as

f =∑i

αiψi


For the orthogonal bases case, there exists a complete set{ψi}i∈Z such that

〈ψi, ψj〉 = δi,j

and then

f =∑i

〈ψi, f〉ψi

[3]⇑


Linear expansions in signal processingWe write a signal f belonging to a space S as

f =∑i

αiψi


For the orthogonal bases case, there exists a complete set{ψi}i∈Z such that

〈ψi, ψj〉 = δi,j

and then

f =∑i

〈ψi, f〉ψi


Linear expansions in signal processingIn the bi-orthogonal case, there exists a dual set {ψi}i∈Z suchthat ⟨

ψi, ψj

⟩= δi,j

and we have

f =∑i

⟨ψ, f

⟩ψi =

∑i

〈ψ, f〉 ψi


Multirate filter banks and structured basisAssume a sequence x[n] in `2(Z), the space of finite energysequences.

Structured basis are obtained when a finite set gi[n] is used togenerate the infinite basis, for example

ψi+kN [n] = gi[n− kN ].


Multirate filter banks and structured basisAssume a sequence x[n] in `2(Z), the space of finite energysequences.

Structured basis are obtained when a finite set gi[n] is used togenerate the infinite basis, for example

ψi+kN [n] = gi[n− kN ].

Expansion in such basis can be computed using multirate filterbanks, with filters hi[n] = gi[−n] (a filter bank to obtain theexpansion coefficients) followed by subsampling.


Multirate filter banks and structured basisThe synthesis filter bank consist of the upsampling followed byinterpolation with impulse response gi[n]... For N = 2 we have:


Multirate filter banks and structured basisThe synthesis filter bank consist of the upsampling followed byinterpolation with impulse response gi[n]... For N = 2 we have:

Because gi[n] and their translates by integer multiples of Nform an orthonormal basis, we have

S0 ⊕ S1 ⊕ · · · ⊕ SN−1 = `2(Z)


Wavelet basesInstead of (Fourier) modulation, a very different family oforthonormal bases is obtained by choosing a single prototypebut using shifting and scaling...

ψm,n(t) = 2−m/2ψ

(t− 2n

2m

)


Wavelet basesInstead of (Fourier) modulation, a very different family oforthonormal bases is obtained by choosing a single prototypebut using shifting and scaling

ψm,n(t) = 2−m/2ψ

(t− 2n

2m

)


Completing the wavelet tree...[5]


Completing the wavelet tree...


Alternative tiling of the time-frequency plane


Alternative tiling of the time-frequency plane

...?


Wavelet packet decomposition


Wavelet packet decomposition

DWT WP


Outline

Introduction





Definition of basic wavelet packets

Consider two finite trigonometric sums:

√2m0(ξ) =

2N−1∑0

h[k]e−ikξ

√2m1(ξ) =

2N−1∑0

g[k]e−ikξ

[1]



Consider two finite trigonometric sums:

√2m0(ξ) =

2N−1∑0

h[k]e−ikξ

√2m1(ξ) =

2N−1∑0

g[k]e−ikξ

[1]⇑


Definition of basic wavelet packetsConsider two finite trigonometric sums:

√2m0(ξ) =

2N−1∑0

h[k]e−ikξ

√2m1(ξ) =

2N−1∑0

g[k]e−ikξ



√2m0(ξ) =

2N−1∑0

h[k]e−ikξ

√2m1(ξ) =

2N−1∑0

g[k]e−ikξ

...satisfying the following conditions:

g[k] = (−1)k+1h[2N −1−k]

or m1(ξ) = ej(2N−1)ξm0(ξ+π)

m0(0) = 1; m0(ξ) 6= 0 on [−π/3, π/3]

|m0(ξ)|2 + |m0(ξ + π)|2 = 1.



√2m0(ξ) =

2N−1∑0

h[k]e−ikξ

√2m1(ξ) =

2N−1∑0

g[k]e−ikξ

...satisfying the following conditions:

g[k] = (−1)k+1h[2N −1−k] or m1(ξ) = ej(2N−1)ξm0(ξ+π)

m0(0) = 1; m0(ξ) 6= 0 on [−π/3, π/3]

|m0(ξ)|2 + |m0(ξ + π)|2 = 1.



The wavelet packets wn(x) are defined by induction onn = 0, 1, 2, 3, ... using the two identities

w2n(x) =√

22N−1∑

0

h[k]wn(2x− k)

w2n+1(x) =√

2

2N−1∑0

g[k]wn(2x− k)

and by the condition w0(x) ∈ L1(R),∫∞−∞w0(x)dx = 1.


Definition of basic wavelet packetsThe wavelet packets wn(x) are defined by induction onn = 0, 1, 2, 3, ... using the two identities

w2n(x) =√

2

2N−1∑0

h[k]wn(2x− k)

w2n+1(x) =√

2

2N−1∑0

g[k]wn(2x− k)

and by the condition w0(x) ∈ L1(R),∫∞−∞w0(x)dx = 1.


With n = 0 we have

w0(x) =√

2

2N−1∑0

h[k]w0(2x− k)

thus, ϕ(x) = w0(x) appears as a fixed point of the operator Tdefined by

T [f(x)] =√

2

2N−1∑0

h[k]f(2x− k)

example...


Example: interatively building ϕ(x)

A possible choice for h[k] is given by

√2h[0] = (1 +

√3)/4,

√2h[1] = (3 +

√3)/4

√2h[2] = (3−

√3)/4,

√2h[3] = (1−

√3)/4

Then we have...



...and the sequence fj converges uniformly to the fixed point ϕ(x).


⇒ ϕ(x) is the “father” wavelet in the construction oforthonormal wavelet bases.



Then, using w2n+1(x) =√

2∑2N−1

0 g[k]wn(2x− k) with n = 0we obtain ψ = w1.

⇒ ψ(x) is the “mother” wavelet in the construction oforthonormal wavelet bases.



Then, using w2n+1(x) =√

2∑2N−1

0 g[k]wn(2x− k) with n = 0we obtain ψ = w1.

⇒ ψ(x) is the “mother” wavelet in the construction oforthonormal wavelet bases.

Next, with n = 1 we obtain w2(x) and w3(x), and by repeatingthe process we generate all of the wavelet packets.


An important result from this construction is that the doblesequence (on n and k)

wn(x− k); n ∈ N0; k ∈ Z

is an orthonormal basis for L2(R).



wn(x− k); n ∈ N0; k ∈ Z


That is, the subsequence derived from here by taking2j ≤ n < 2j+1 is an orthonormal basis for the orthogonalcomplement Wj of Vj in Vj+1.



wn(x− k); n ∈ N0; k ∈ Z



Recall: in the multiresolution wavelet analysis 2j/2ψ(2jx− k)generates Wj (and 2j/2ϕ(2jx− k) generates Vj).



wn(x− k); n ∈ N0; k ∈ Z



Recall: in the multiresolution wavelet analysis 2j/2ψ(2jx− k)generates Wj (and 2j/2ϕ(2jx− k) generates Vj).

Thus, this construction of wavelet packets appears as a changeof orthonormal basis inside each Wj .


General wavelet packets

The general wavelet packets are the functions

2j/2wn(2jx− k), n ∈ N j, k ∈ Z




2j/2wn(2jx− k), n ∈ N j, k ∈ Z

if j = 0 ...




2j/2wn(2jx− k), n ∈ N j, k ∈ Z

if j = 0 we have the previous basic wavelet packets definition




2j/2wn(2jx− k), n ∈ N j, k ∈ Z


if n = 1 ...




2j/2wn(2jx− k), n ∈ N j, k ∈ Z


if n = 1 we have an standard orthogonal wavelet basis


But.... which of these sets of wavelet packets makes uporthogonal bases for L2(R)?



Let E be a set of pairs (j, n) such that the correspondingfrequency intervals 2jn ≤ ξ < 2j(n+ 1) constitute a partition of[0,∞), up to a contable set. Then the subsequence

2j/2wn(2jx− k), n ∈ N j, k ∈ Z

is an orthogonal basis for L2(R).



Let E be a set of pairs (j, n) such that the correspondingfrequency intervals 2jn ≤ ξ < 2j(n+ 1) constitute a partition of[0,∞), up to a contable set. Then the subsequence

2j/2wn(2jx− k), n ∈ N j, k ∈ Z

is an orthogonal basis for L2(R).

(...filter different frequency bands and then cut these bands intemporal windows to study their energy variations...)


Trees and spaces

Any node of the binary tree is labeled by (j, p), where:j − L is the depth of the node in the tree, andp is the number of the node (at the same depth)

[4]


Admissible trees

We call admissible tree any binary tree where each node haseither 0 or 2 children.

Basic examples...


Alternative tiling of the time-frequency plane[R]


Number of WP basesThe number of different wavelet packet orthogonal bases of VLis equal to the number of different admissible binary trees...


Number of WP basesThe number of different wavelet packet orthogonal bases of VLis equal to the number of different admissible binary trees...

The number BJ of wavelet packet bases in a full wavelet packetbinary tree of depth J satisfies

22J−1 ≤ BJ ≤ 2

542J−1

Homework: Proof [4] 8.1.1


Trees and Heisenberg boxes

The wavelet packet tree and the corresponding Heisenbergboxes of a wavelet packet basis.


Trees and Heisenberg boxes

The wavelet packet tree and the corresponding Heisenbergboxes of a wavelet packet basis.

More examples...?• The Heisenberg boxes from the WP tree...• The WP tree from the Heisenberg boxes...


Time support

Suppose that for some N ∈ N, the scaling function w0(x) issupported in [0, 2N − 1] and that the scaling filter h[k] satisfiesh[k] = 0 for k < 0 and k ≥ 2N .

[2]


Time supportSuppose that for some N ∈ N, the scaling function w0(x) issupported in [0, 2N − 1] and that the scaling filter h[k] satisfiesh[k] = 0 for k < 0 and k ≥ 2N .


Time supportSuppose that for some N ∈ N, the scaling function w0(x) issupported in [0, 2N − 1] and that the scaling filter h[k] satisfiesh[k] = 0 for k < 0 and k ≥ 2N .

Then for each n ∈ N, wn(x) is supported in [0, 2N − 1].


Nominal frequency

|wn(ξ)| for n = 0, 1, . . . , 7 corresponding to the Daubechies12 filter.

The location of the dominant peak is referred to as the nominalfrequency of wn(x).


Frequency order

Frequency ordered wavelet packets computed with theDaubechies5 filter, at the depth j − L = 3 of the wavelet packettree, with L = 0.

[4]


Frequency order

Frequency ordered wavelet packets computed with a Haar filter,at the depth j − L = 3 of the wavelet packet tree, with L = 0.


Outline

Introduction





Decomposition

d2pj+1[k] = dpj ∗ h[−2k] d2p+1j+1 [k] = dpj ∗ g[−2k]

Homework: What is the frequency localization of signals for each node aftersubsampling?

[4]


Decomposition

d2pj+1[k] = dpj ∗ h[−2k] d2p+1j+1 [k] = dpj ∗ g[−2k]

Homework: What is the frequency localization of signals for each node aftersubsampling?


Reconstruction

dpj [k] = d2pj+1 ∗ h[k] + d2p+1j+1 [k] ∗ g[k]

Homework: What is the frequency localization of signals for each node afterinterpolation?


Finite signals

d2pj+1[k] = dpj ∗ h[−2k]

d2p+1j+1 [k] = dpj ∗ g[−2k]


Standard or circular convolutions?


Coefficients in the full tree



For each node p in the scale j the signal dpj has 2−j samples.




At any depth j − L of the tree, the wavelet packet signals{dpj}0≤p<2j−L include a total of N coefficients.




At any depth j − L of the tree, the wavelet packet signals{dpj}0≤p<2j−L include a total of N coefficients.

Since the maximum depth is log2N , there are at most Nlog2Ncoefficients in a full wavelet packet tree.

Basic examples...


Computational complexityIn a full wavelet packet tree of depth log2N all coefficients arecomputed by iterating

d2pj+1[k] = dpj ∗ h[−2k]

d2p+1j+1 [k] = dpj ∗ g[−2k]

for L ≤ j < 0.

Homework: How many wavelet packet bases are included in this wavelet packetstree?


Computational complexityIn a full wavelet packet tree of depth log2N all coefficients arecomputed by iterating

d2pj+1[k] = dpj ∗ h[−2k]

d2p+1j+1 [k] = dpj ∗ g[−2k]

for L ≤ j < 0.If h and g have K non-zero coefficients, this requires about

KNlog2N

additions and multiplications.

Homework: How many wavelet packet bases are included in this wavelet packetstree?


Computational complexity

And for the reconstruction algorithm...


• the computational complexity in the worst case?• the computational complexity in the simplest case?• and for the reconstruction of a simple (dyadic) wavelet

tree?





What is...• the computational complexity in the worst case?

• the computational complexity in the simplest case?• and for the reconstruction of a simple (dyadic) wavelet

tree?





What is...• the computational complexity in the worst case?KNlog2N

• the computational complexity in the simplest case?• and for the reconstruction of a simple (dyadic) wavelet

tree?





What is...• the computational complexity in the worst case?• the computational complexity in the simplest case?

• and for the reconstruction of a simple (dyadic) wavelettree?





What is...• the computational complexity in the worst case?• the computational complexity in the simplest case?

2KN

• and for the reconstruction of a simple (dyadic) wavelettree?





What is...• the computational complexity in the worst case?• the computational complexity in the simplest case?• and for the reconstruction of a simple (dyadic) wavelet

tree?





What is...• the computational complexity in the worst case?• the computational complexity in the simplest case?• and for the reconstruction of a simple (dyadic) wavelet

tree?2KN


Outline

Introduction





sinc wavelet packetsinc wavelet packets are computed with perfect discretelow-pass and high-pass filters

|h(ξ)| ={ √

2 if ξ ∈ [−π/2 + 2kπ, π/2 + 2kπ]0 otherwise

|g(ξ)| ={ √

2 if ξ ∈ [π/2 + 2kπ, 3π/2 + 2kπ]0 otherwise

therefore...


sinc wavelet packetsinc wavelet packets are computed with perfect discretelow-pass and high-pass filters

|h(ξ)| ={ √

2 if ξ ∈ [−π/2 + 2kπ, π/2 + 2kπ]0 otherwise

|g(ξ)| ={ √

2 if ξ ∈ [π/2 + 2kπ, 3π/2 + 2kπ]0 otherwise

therefore...

ψpj (t) = 2−j/2+1 sin(2−jπt/2)

2−jπtcos(2−jπ(k + 1/2)(t− τj,p))


Walsh wavelet packetWalsh wavelet packets are generated by the Haar conjugatemirror filter

h[n] =

{1√2

if n = 0, 1

0 otherwise


(pseudo) local cosine wavelet packet

{ψkJ(t− 2Jn)

}n∈Z,0≤k<2J−L


mel wavelet packet0 -

2 5 0

2 5 0 -

5 0 0

5 0 0 -

6 2 5

6 2 5 -

7 5 0

7 5 0 -

8 7 5

8 7 5 -

1 0 0 0

1 0

0 0 - 1

1 2 5

1 1 2 5

- 1 2 5

0 1 2

5 0 - 1

3 7 5

1 3 7 5

- 1 5 0

0 1 5

0 0 - 1

6 2 5

1 6 2 5

- 1 7 5

0 1 7

5 0 - 1

8 7 5

1 8 7 5

- 2 0 0

0 2 0

0 0 - 2

2 5 0

2 2 5 0

- 2 5 0

0

2 5 0 0

- 3 0 0

0

3 0 0 0

- 3 5 0

0

3 5 0 0

- 4 0 0

0

4 0 0 0

- 5 0 0

0

5 0 0 0

- 6 0 0

0

6 0 0 0

- 7 0 0

0

7 0 0 0

- 8 0 0

0


Block transforms

Let g = 1[0,1] and the interval [ap, ap+1] covered by

gp(t) = 1[ap,ap+1](t) = g

(t− ap`p

)with `p = ap+1 − ap.

If {ek}k∈Z is an orthonormal basis of L2[0, 1], then{gp,k(t) = gp(t)

1√`pek

(t− ap`p

)}(p,k)∈Z

is a block orthonormal basis of L2(R).


Discrete block bases

Given a discrete rectangular window

gp[n] = 1[ap,ap+1−1](n)

If {ek,`}0≤k<` is an orthonormal basis of C` for any ` > 0, thenthe family {

gp,k[n] = gp[n]ek,`p [n− ap]}0≤k<`p,p∈Z

is a block orthonormal basis of `2(Z).


Discrete block cosine transform

Given the discrete windows gp[n] = 1[ap,ap−1](n) to divide Z, theblock basis

{gp,k[n] = gp[n]λk

√2

`pcos

[kπ

`p

(n+

1

2− ap

)]}0≤k<N,p∈Z

is an orthonormal basis of `2(Z).

(The standard for JPEG image compression)


Lapped orthogonal transforms


Lapped orthogonal bases

If {ek}k∈N is an orthonormal basis of L2[0, 1], and ek(t) is anextension over R, symmetric with repect to 0 andantisymmetric with respecto to 1,

then

{gp,k(t) = gp(t)

1√`pek

(t− ap`p

)}k∈N,p∈Z

is an orthonormal basis of L2(R).


Lapped orthogonal bases

If {ek}k∈N is an orthonormal basis of L2[0, 1], and ek(t) is anextension over R, symmetric with repect to 0 andantisymmetric with respecto to 1, then

{gp,k(t) = gp(t)

1√`pek

(t− ap`p

)}k∈N,p∈Z



Local cosine basis

The family of local cosine functions

{gp,k(t) = gp(t)

√2

`pcos

[π

(k +

1

2

)t− ap`p

]}k∈N,p∈Z



Local cosine vectors in time-frequency


Local cosine trees

Admissible tree of (dyadic) local cosine


It can be automatically found the wavelet packet basis that“best fits” or is “best adapted to” a given signal?

DWT WP


It can be automatically found the wavelet packet basis that“best fits” or is “best adapted to” a given signal?

• Single tree algorithm• Double tree algorithm (Frame-adaptive WP)• Wavelet packet and local cosine best basis• Basis pursuit• Matching pursuit• Best orthogonal basis• Method of frames• ...• ...• ...


Bibliography

[1] Meyer Y., Wavelets, algorithms and applications, SIAM,1993.

[2] Walnut D., An introduction to wavelet analysis,Birkhauser, 2002.

[3] Ramchandran K., Vetterli M., Herley C., “Wavelets,subband coding, and best bases,” Proceedings of the IEEE,vol.84, no.4, 1996.

[4] Mallat S., A wavelet tour of signal processing, AcademicPress, 1999.

[5] Addison P., The Illustrated Wavelet Transform Handbook,IOP, 2002.

[6] Hess-Nielsen N., Wickerhauser M.V., Wavelets andtime-frequency analysis, Proceedings of the IEEE, vol.84,no.4, 1996.

diego milone -...

Documents