1
Wavelets and compression
Dr Mike Spann
2
Contents
Scale and image compression Signal (image) approximation/prediction
– simple wavelet construction Statistical dependencies in wavelet
coefficients – why wavelet compression works
State-of-the-art wavelet compression algorithms
3
Image at different scales
4
Correlation between features at different scales
5
Wavelet construction – a simplified approach
Traditional approaches to wavelets have used a filterbank interpretation
Fourier techniques required to get synthesis (reconstruction) filters from analysis filters
Not easy to generalize
6
3 steps
Split
Predict (P step)
Update (U step)
Wavelet construction – lifting
7
Example – the Haar wavelet
S step
Splits the signal into odd and even samples
ns 1ne
1no
even samples
odd samples
8
For the Haar wavelet, the prediction for the odd sample is the previous even sample :
lnln ss 2,12,ˆ
lns ,
l
Example – the Haar wavelet
P step
Predict the odd samples from the even samples
9
lnlnln ssd 2,12,,1
Example – the Haar wavelet
lnd ,1
l
lns ,
l
Detail signal :
10
2/,12,,1 lnlnln dss
The signal average is maintained :
12
0
12
0,,1
1
2/1n n
l llnln ss
Example – the Haar wavelet
U step
Update the even samples to produce the next coarser scale approximation
11
…..
-1
Summary of the Haar wavelet decomposition
lnlnln ssd 2,12,,1
2/,12,,1 lnlnln dss
Can be computed ‘in place’ :
lns 2, 12, lns12, lns
lnd ,1lns 2,1,1 lnd
…..-1
lnd ,11,1 lnd lns ,1
1/21/2
P step
U step
12
lnlnln dss ,12,12,
2/,1,12, lnlnln dss
Then merge even and odd samples
Mergelns 2,
12, lnslns ,
Inverse Haar wavelet
transform Simply run the forward Haar wavelet
transform backwards!
13
General lifting stage of wavelet decomposition
-
Split P U
+
js
1js
1jd
1je
1jo
14
Multi-level wavelet decomposition
lift lift lift…ns
1ns
2nd
0s
1nd 0d
We can produce a multi-level decomposition by cascading lifting stages
15
General lifting stage of inverse wavelet synthesis
-
MergePU
+
js
1je
1jo
1js
1jd
16
lift …...lift lift0s 1s 2s 1nsns
0d
1d 2d 1nd
We can produce a multi-level inverse wavelet synthesis by cascading lifting stages
Multi-level inverse wavelet synthesis
17
Advantages of the lifting implementation
Inverse transform Inverse transform is trivial – just run the code
backwards No need for Fourier techniques
Generality The design of the transform is performed without
reference to particular forms for the predict and update operators
Can even include non-linearities (for integer wavelets)
18
Example 2 – the linear spline wavelet
A more sophisticated wavelet – uses slightly more complex P and U operators
Uses linear prediction to determine odd samples from even samples
19
Linear prediction at odd samples Original signal
Detail signal (prediction error at odd samples)
The linear spline wavelet
P-step – linear prediction
20
22,2,12,,1 2/1 lnlnlnln sssd
22,2,12, 2/1ˆ lnlnln sss
The linear spline wavelet
The prediction for the odd samples is based on the two even samples either side :
21
)(4/1 ,11,12,,1 lnlnlnln ddss
The linear spline wavelet
The U step – use current and previous detail signal sample
22
Preserves signal average and first-order moment (signal position) :
The linear spline wavelet
12
0,
12
0,1 2/1
1 nn
lln
lln ss
12
0
12
0,,1
1
2/1n n
l llnln lsls
23
1/41/41/41/4
-1/2-1/2-1/2-1/2
lns 2, 12, lns 22, lns
lnd ,1lns 2, 22, lns
lns ,1 1,1 lnslnd ,1
P step
U step
The linear spline wavelet
Can still implement ‘in place’
24
Summary of linear spline wavelet decomposition
22,2,12,,1 2/1 lnlnlnln sssd
)(4/1 ,11,12,,1 lnlnlnln ddss
Computing the inverse is trivial :
)(4/1 ,11,1,12, lnlnlnln ddss 22,2,,112, 2/1 lnlnlnln ssds
The even and odd samples are then merged as before
25
Wavelet decomposition applied to a 2D image
detail
detail
approx
lift
.
.
.
.
.lift
approx
26
approx
detaildetailapprox
approx
detailapproxdetail
lift lift lift lift
1,1 nni1
1,1 nnd 21,1 nnd 3
1,1 nnd
Wavelet decomposition applied to a 2D image
27
Why is wavelet-based compression effective?
Allows for intra-scale prediction (like many other compression methods) – equivalently the wavelet transform is a decorrelating transform just like the DCT as used by JPEG
Allows for inter-scale (coarse-fine scale) prediction
28
1 level Haar
1 level linear spline 2 level Haar
Original
Why is wavelet-based compression effective?
29
0
5000
10000
15000
20000
25000
-255 -205 -155 -105 -55 -5 45 95 145 195 245
Original
Haar wavelet
Why is wavelet-based compression effective? Wavelet coefficient histogram
30
Entropy
Original image 7.22
1-level Haar wavelet 5.96
1-level linear spline wavelet 5.53
2-level Haar wavelet 5.02
2-level linear spline wavelet 4.57
Why is wavelet-based compression effective? Coefficient entropies
31
X
)(XP
Why is wavelet-based compression effective?
Wavelet coefficient dependencies
32
))(|( SXPSXP
))(|( LXPLXP
Why is wavelet-based compression effective?
Lets define sets S (small) and L (large) wavelet coefficients
The following two probabilities describe interscale dependancies
33
2
#))(|(
N
SSXPSXP
2
#))(|(
N
LLXPLXP
Why is wavelet-based compression effective?
Without interscale dependancies
34
0.886 0.529 0.781 0.219
))(|( SXPSXP 2
#
N
S))(|( LXPLXP
2
#
N
L
Why is wavelet-based compression effective?
Measured dependancies from Lena
35
X
8
1
)(8
1
nnXcc
X1
X8 TcLX
TcSX
n
n
if
if
Why is wavelet-based compression effective?
Intra-scale dependencies
36
0.912 0.623 0.781 0.219
)|( SXSXP n 2
#
N
S )|( LXLXP n 2
#
N
L
Why is wavelet-based compression effective?
Measured dependancies from Lena
37
Why is wavelet-based compression effective?
Have to use a causal neighbourhood for spatial prediction
38
We will look at 3 state of the art algorithms
Set partitioning in hierarchical sets (SPIHT)
Significance linked connected components analysis (SLCCA)
Embedded block coding with optimal truncation (EBCOT) which is the basis of JPEG2000
Example image compression algorithms
39
lsb
msb 1 1 0 0 0 0 0 0 0 0 0 0 0 0 …
x x 1 1 0 0 0 0 0 0 0 0 0 0 …
x x x x 1 1 1 0 0 0 0 0 0 …
x x x x x x x 1 1 1 1 1 1 0 …
x x x x x x x x x x x x x 1 …
x x x x x x x x x x x x x x …
Coeff. number 1 2 3 4 5 6 7 8 9 10 11 12 13 14…….
5
4
3
2
1
0
The SPIHT algorithm Coefficients transmitted in partial order
0
40
2 components to the algorithm Sorting pass
Sorting information is transmitted on the basis of the most significant bit-plane
Refinement pass Bits in bit-planes lower than the most
significant bit plane are transmitted
The SPIHT algorithm
41
N= msb of (max(abs(wavelet coefficient)))
for (bit-plane-counter)=N downto 1
transmit significance/insignificance wrt bit-plane counter
transmit refinement bits of all coefficients that
are already significant
The SPIHT algorithm
42
The SPIHT algorithm Insignificant coefficients (with respect to current bitplane counter)
organised into zerotrees
43
The SPIHT algorithm
Groups of coefficients made into zerotrees by set paritioning
44
….1100101011100101100011………01011100010111011011101101….
bitstream
The SPIHT algorithm
SPIHT produces an embedded bitstream
45
The SLCCA algorithm
Bit-plane encode
significant coefficients
Wavelet transform
Quantise coefficients
Cluster andtransmit
significance map
46
The SLCCA algorithm
The significance map is grouped into clusters
47
Seed
Significant coeff
Insignificant coeff
The SLCCA algorithm
Clusters grown out from a seed
48
Significance link
The SLCCA algorithm
Significance link symbol
49
Image compression results
Evaluation Mean squared error Human visual-based metrics Subjective evaluation
50
21
0,2
),(ˆ),((1
N
cr
crIcrIN
mse
msedBPSNR
2
10
255log10)(
Usually expressed as peak-signal-to-noise (in dB)
Image compression results
Mean-squared error
51
25
27
29
31
33
35
37
39
41
43
0.2 0.4 0.6 0.8 1 1.2
bit-rate (bits/pixel)
PSNR(dB)
SPIHT
SLCCA
JPEG
Image compression results
52
25
27
29
31
33
35
37
39
41
43
0.2 0.4 0.6 0.8 1 1.2
bit-rate (bits/pixel)
PSNR(dB)
Haar
Linear spline
Daubechies 9-7
Image compression results
53
SPIHT 0.2 bits/pixel JPEG 0.2 bits/pixel
Image compression results
54
SPIHT JPEG
Image compression results
55
EBCOT, JPEG2000 JPEG2000, based on embedded block coding
and optimal truncation is the state-of-the-art compression standard
Wavelet-based It addresses the key issue of scalability
SPIHT is distortion scalable as we have already seen
JPEG2000 introduces both resolution and spatial scalability also
An excellent reference to JPEG2000 and compression in general is “JPEG2000” by D.Taubman and M. Marcellin
56
Resolution scalability is the ability to extract from the bitstream the sub-bands representing any resolution level
….1100101011100101100011………01011100010111011011101101….bitstream
EBCOT, JPEG2000
57
Spatial scalability is the ability to extract from the bitstream the sub-bands representing specific regions in the image Very useful if we want to selectively
decompress certain regions of massive images
….1100101011100101100011………01011100010111011011101101….bitstream
EBCOT, JPEG2000
58
Introduction to EBCOT JPEG2000 is able to implement this
general scalability by implementing the EBCOT paradigm
In EBCOT, the unit of compression is the codeblock which is a partition of a wavelet sub-band
Typically, following the wavelet transform,each sub-band is partitioned into small blocks (typically 32x32)
59
Introduction to EBCOT Codeblocks – partitions of wavelet
sub-bands
codeblock
60
Introduction to EBCOT A simple bit stream organisation could
comprise concatenated code block bit streams
……0CB 1CB 2CB0L 1L 2L
Length of next code-block stream
61
Introduction to EBCOT This simple bit stream structure is
resolution and spatially scalable but not distortion scalable
Complete scalability is obtained by introducing quality layers Each code block bitstream is individually
(optimally) truncated in each quality layer Loss of parent-child redundancy more than
compensated by ability to individually optimise separate code block bitstreams
62
Introduction to EBCOT Each code block bit stream
partitioned into a set of quality layers
0Q0QL
1QL 1Q …
00CB 0
1CB 02CB0
0L 01L 0
2L …
10CB 1
1CB 12CB1
0L 11L 1
2L …
63
EBCOT advantages Multiple scalability
Distortion, spatial and resolution scalability Efficient compression
This results from independent optimal truncation of each code block bit stream
Local processing Independent processing of each code block
allows for efficient parallel implementations as well as hardware implementations
64
EBCOT advantages Error resilience
Again this results from independent code block processing which limits the influence of errors
65
Performance comparison A performance comparison with other
wavelet-based coders is not straightforward as it would depend on the target bit rates which the bit streams were truncated for With SPIHT, we simply truncate the bit stream
when the target bit rate has been reached However, we only have distortion scalability
with SPIHT Even so, we still get favourable PSNR (dB)
results when comparing EBCOT (JPEG200) with SPIHT
66
Performance comparison We can understand this more fully
by looking at graphs of distortion (D) against rate (R) (bitstream length)
R
D
R-D curve for continuously modulated quantisation step size
Truncation points
67
Performance comparison Truncating the bit stream to some
arbitrary rate will yield sub-optimal performance
R
D
68
Performance comparison
25
27
29
31
33
35
37
39
41
43
0.0625 0.125 0.25 0.5 1
bit-rate (bits/pixel)
PSNR(dB)
Spiht
EBCOT/JPEG2000
69
Performance comparison Comparable PSNR (dB) results
between EBCOT and SPIHT even though: Results for EBCOT are for 5 quality layers
(5 optimal bit rates) Intermediate bit rates sub-optimal
We have resolution, spatial, distortion scalability in EBCOT but only distortion scalability in SPIHT