1

Wavelets and compression

Dr Mike Spann

2

Contents

Scale and image compression Signal (image) approximation/prediction

– simple wavelet construction Statistical dependencies in wavelet

coefficients – why wavelet compression works

State-of-the-art wavelet compression algorithms

3

Image at different scales

4

Correlation between features at different scales

5

Wavelet construction – a simplified approach

Traditional approaches to wavelets have used a filterbank interpretation

Fourier techniques required to get synthesis (reconstruction) filters from analysis filters

Not easy to generalize

6

3 steps

Split

Predict (P step)

Update (U step)

Wavelet construction – lifting

7

Example – the Haar wavelet

S step

Splits the signal into odd and even samples

ns 1ne

1no

even samples

odd samples

8

For the Haar wavelet, the prediction for the odd sample is the previous even sample :

lnln ss 2,12,ˆ

lns ,

l

Example – the Haar wavelet

P step

Predict the odd samples from the even samples

9

lnlnln ssd 2,12,,1

Example – the Haar wavelet

lnd ,1

l

lns ,

l

Detail signal :

10

2/,12,,1 lnlnln dss

The signal average is maintained :

12

0

12

0,,1

1

2/1n n

l llnln ss

Example – the Haar wavelet

U step

Update the even samples to produce the next coarser scale approximation

11

…..

-1

Summary of the Haar wavelet decomposition

lnlnln ssd 2,12,,1

2/,12,,1 lnlnln dss

Can be computed ‘in place’ :

lns 2, 12, lns12, lns

lnd ,1lns 2,1,1 lnd

…..-1

lnd ,11,1 lnd lns ,1

1/21/2

P step

U step

12

lnlnln dss ,12,12,

2/,1,12, lnlnln dss

Then merge even and odd samples

Mergelns 2,

12, lnslns ,

Inverse Haar wavelet

transform Simply run the forward Haar wavelet

transform backwards!

13

General lifting stage of wavelet decomposition

-

Split P U

+

js

1js

1jd

1je

1jo

14

Multi-level wavelet decomposition

lift lift lift…ns

1ns

2nd

0s

1nd 0d

We can produce a multi-level decomposition by cascading lifting stages

15

General lifting stage of inverse wavelet synthesis

-

MergePU

+

js

1je

1jo

1js

1jd

16

lift …...lift lift0s 1s 2s 1nsns

0d

1d 2d 1nd

We can produce a multi-level inverse wavelet synthesis by cascading lifting stages

Multi-level inverse wavelet synthesis

17

Advantages of the lifting implementation

Inverse transform Inverse transform is trivial – just run the code

backwards No need for Fourier techniques

Generality The design of the transform is performed without

reference to particular forms for the predict and update operators

Can even include non-linearities (for integer wavelets)

18

Example 2 – the linear spline wavelet

A more sophisticated wavelet – uses slightly more complex P and U operators

Uses linear prediction to determine odd samples from even samples

19

Linear prediction at odd samples Original signal

Detail signal (prediction error at odd samples)

The linear spline wavelet

P-step – linear prediction

20

22,2,12,,1 2/1 lnlnlnln sssd

22,2,12, 2/1ˆ lnlnln sss

The linear spline wavelet

The prediction for the odd samples is based on the two even samples either side :

21

)(4/1 ,11,12,,1 lnlnlnln ddss

The linear spline wavelet

The U step – use current and previous detail signal sample

22

Preserves signal average and first-order moment (signal position) :

The linear spline wavelet

12

0,

12

0,1 2/1

1 nn

lln

lln ss

12

0

12

0,,1

1

2/1n n

l llnln lsls

23

1/41/41/41/4

-1/2-1/2-1/2-1/2

lns 2, 12, lns 22, lns

lnd ,1lns 2, 22, lns

lns ,1 1,1 lnslnd ,1

P step

U step

The linear spline wavelet

Can still implement ‘in place’

24

Summary of linear spline wavelet decomposition

22,2,12,,1 2/1 lnlnlnln sssd

)(4/1 ,11,12,,1 lnlnlnln ddss

Computing the inverse is trivial :

)(4/1 ,11,1,12, lnlnlnln ddss 22,2,,112, 2/1 lnlnlnln ssds

The even and odd samples are then merged as before

25

Wavelet decomposition applied to a 2D image

detail

detail

approx

lift

.

.

.

.

.lift

approx

26

approx

detaildetailapprox

approx

detailapproxdetail

lift lift lift lift

1,1 nni1

1,1 nnd 21,1 nnd 3

1,1 nnd

Wavelet decomposition applied to a 2D image

27

Why is wavelet-based compression effective?

Allows for intra-scale prediction (like many other compression methods) – equivalently the wavelet transform is a decorrelating transform just like the DCT as used by JPEG

Allows for inter-scale (coarse-fine scale) prediction

28

1 level Haar

1 level linear spline 2 level Haar

Original

Why is wavelet-based compression effective?

29

0

5000

10000

15000

20000

25000

-255 -205 -155 -105 -55 -5 45 95 145 195 245

Original

Haar wavelet

Why is wavelet-based compression effective? Wavelet coefficient histogram

30

  Entropy

Original image 7.22

1-level Haar wavelet 5.96

1-level linear spline wavelet 5.53

2-level Haar wavelet 5.02

2-level linear spline wavelet 4.57

Why is wavelet-based compression effective? Coefficient entropies

31

X

)(XP

Why is wavelet-based compression effective?

Wavelet coefficient dependencies

32

))(|( SXPSXP

))(|( LXPLXP

Why is wavelet-based compression effective?

Lets define sets S (small) and L (large) wavelet coefficients

The following two probabilities describe interscale dependancies

33

2

#))(|(

N

SSXPSXP

2

#))(|(

N

LLXPLXP

Why is wavelet-based compression effective?

Without interscale dependancies

34

0.886 0.529 0.781 0.219

))(|( SXPSXP 2

#

N

S))(|( LXPLXP

2

#

N

L

Why is wavelet-based compression effective?

Measured dependancies from Lena

35

X

8

1

)(8

1

nnXcc

X1

X8 TcLX

TcSX

n

n

if

if

Why is wavelet-based compression effective?

Intra-scale dependencies

36

0.912 0.623 0.781 0.219

)|( SXSXP n 2

#

N

S )|( LXLXP n 2

#

N

L

Why is wavelet-based compression effective?

Measured dependancies from Lena

37

Why is wavelet-based compression effective?

Have to use a causal neighbourhood for spatial prediction

38

We will look at 3 state of the art algorithms

Set partitioning in hierarchical sets (SPIHT)

Significance linked connected components analysis (SLCCA)

Embedded block coding with optimal truncation (EBCOT) which is the basis of JPEG2000

Example image compression algorithms

39

lsb

msb 1 1 0 0 0 0 0 0 0 0 0 0 0 0 …

x x 1 1 0 0 0 0 0 0 0 0 0 0 …

x x x x 1 1 1 0 0 0 0 0 0 …

x x x x x x x 1 1 1 1 1 1 0 …

x x x x x x x x x x x x x 1 …

x x x x x x x x x x x x x x …

Coeff. number 1 2 3 4 5 6 7 8 9 10 11 12 13 14…….

5

4

3

2

1

0

The SPIHT algorithm Coefficients transmitted in partial order

0

40

2 components to the algorithm  Sorting pass

Sorting information is transmitted on the basis of the most significant bit-plane

Refinement pass Bits in bit-planes lower than the most

significant bit plane are transmitted

The SPIHT algorithm

41

N= msb of (max(abs(wavelet coefficient)))

for (bit-plane-counter)=N downto 1

transmit significance/insignificance wrt bit-plane counter

transmit refinement bits of all coefficients that

are already significant

The SPIHT algorithm

42

The SPIHT algorithm Insignificant coefficients (with respect to current bitplane counter)

organised into zerotrees

43

The SPIHT algorithm

Groups of coefficients made into zerotrees by set paritioning

44

….1100101011100101100011………01011100010111011011101101….

bitstream

The SPIHT algorithm

SPIHT produces an embedded bitstream

45

The SLCCA algorithm

Bit-plane encode

significant coefficients

Wavelet transform

Quantise coefficients

Cluster andtransmit

significance map

46

The SLCCA algorithm

The significance map is grouped into clusters

47

Seed

Significant coeff

Insignificant coeff

The SLCCA algorithm

Clusters grown out from a seed

48

Significance link

The SLCCA algorithm

Significance link symbol

49

Image compression results

Evaluation  Mean squared error Human visual-based metrics Subjective evaluation

50

21

0,2

),(ˆ),((1

N

cr

crIcrIN

mse

msedBPSNR

2

10

255log10)(

Usually expressed as peak-signal-to-noise (in dB)

Image compression results

Mean-squared error 

51

25

27

29

31

33

35

37

39

41

43

0.2 0.4 0.6 0.8 1 1.2

bit-rate (bits/pixel)

PSNR(dB)

SPIHT

SLCCA

JPEG

Image compression results

52

25

27

29

31

33

35

37

39

41

43

0.2 0.4 0.6 0.8 1 1.2

bit-rate (bits/pixel)

PSNR(dB)

Haar

Linear spline

Daubechies 9-7

Image compression results

53

SPIHT 0.2 bits/pixel JPEG 0.2 bits/pixel

Image compression results

54

SPIHT JPEG

Image compression results

55

EBCOT, JPEG2000 JPEG2000, based on embedded block coding

and optimal truncation is the state-of-the-art compression standard

Wavelet-based It addresses the key issue of scalability

SPIHT is distortion scalable as we have already seen

JPEG2000 introduces both resolution and spatial scalability also

An excellent reference to JPEG2000 and compression in general is “JPEG2000” by D.Taubman and M. Marcellin

56

Resolution scalability is the ability to extract from the bitstream the sub-bands representing any resolution level

….1100101011100101100011………01011100010111011011101101….bitstream

EBCOT, JPEG2000

57

Spatial scalability is the ability to extract from the bitstream the sub-bands representing specific regions in the image Very useful if we want to selectively

decompress certain regions of massive images

….1100101011100101100011………01011100010111011011101101….bitstream

EBCOT, JPEG2000

58

Introduction to EBCOT JPEG2000 is able to implement this

general scalability by implementing the EBCOT paradigm

In EBCOT, the unit of compression is the codeblock which is a partition of a wavelet sub-band

Typically, following the wavelet transform,each sub-band is partitioned into small blocks (typically 32x32)

59

Introduction to EBCOT Codeblocks – partitions of wavelet

sub-bands

codeblock

60

Introduction to EBCOT A simple bit stream organisation could

comprise concatenated code block bit streams

……0CB 1CB 2CB0L 1L 2L

Length of next code-block stream

61

Introduction to EBCOT This simple bit stream structure is

resolution and spatially scalable but not distortion scalable

Complete scalability is obtained by introducing quality layers Each code block bitstream is individually

(optimally) truncated in each quality layer Loss of parent-child redundancy more than

compensated by ability to individually optimise separate code block bitstreams

62

Introduction to EBCOT Each code block bit stream

partitioned into a set of quality layers

0Q0QL

1QL 1Q …

00CB 0

1CB 02CB0

0L 01L 0

2L …

10CB 1

1CB 12CB1

0L 11L 1

2L …

63

EBCOT advantages Multiple scalability

Distortion, spatial and resolution scalability Efficient compression

This results from independent optimal truncation of each code block bit stream

Local processing Independent processing of each code block

allows for efficient parallel implementations as well as hardware implementations

64

EBCOT advantages Error resilience

Again this results from independent code block processing which limits the influence of errors

65

Performance comparison A performance comparison with other

wavelet-based coders is not straightforward as it would depend on the target bit rates which the bit streams were truncated for With SPIHT, we simply truncate the bit stream

when the target bit rate has been reached However, we only have distortion scalability

with SPIHT Even so, we still get favourable PSNR (dB)

results when comparing EBCOT (JPEG200) with SPIHT

66

Performance comparison We can understand this more fully

by looking at graphs of distortion (D) against rate (R) (bitstream length)

R

D

R-D curve for continuously modulated quantisation step size

Truncation points

67

Performance comparison Truncating the bit stream to some

arbitrary rate will yield sub-optimal performance

R

D

68

Performance comparison

25

27

29

31

33

35

37

39

41

43

0.0625 0.125 0.25 0.5 1

bit-rate (bits/pixel)

PSNR(dB)

Spiht

EBCOT/JPEG2000

69

Performance comparison Comparable PSNR (dB) results

between EBCOT and SPIHT even though: Results for EBCOT are for 5 quality layers

(5 optimal bit rates) Intermediate bit rates sub-optimal

We have resolution, spatial, distortion scalability in EBCOT but only distortion scalability in SPIHT