block transform coding - uc santa barbaramanj/ece178-fall2009/e178-l14.ppt.pdf · block transform...
TRANSCRIPT
Image Compression-II 1
Block Transform Coding
Section 8.2.8
Image Compression-II 2
Transform coding
Construct n X n subimages
Forward transform Quantizer Symbol
encoder N X N images
Compressed image
Compressed image Symbol
decoder Inverse transform
Merge n X n subimages
Uncompressed image
Blocking artifact: boundaries between subimages become visible
Image Compression-II 3
Transform Selection
DFT Discrete Cosine Transform (DCT) Wavelet transform Karhunen-Loeve Transform (KLT) ….
T (u,v) = g(x, y)r(x, y,u,v)
y=0
n−1
∑x=0
n−1
∑ − − − −(8.2.10)
g(x, y) = T (u,v)s(x, y,u,v) − − − −8.2.11
v=0
N −1
∑u=0
N −1
∑
Image Compression-II 4
Transform Kernels
Separable if
E.g. DFT:
E.g. Walsh-Hadamard transform (see page 568, text)
E.g. DCT
r(x, y,u,v) = r1(x,u)r2 ( y,v) − − − −(8.2.12)
r(x, y,u,v) = exp(− j2π (ux + vy) / n)
Image Compression-II 5
Approximations using DFT, Hadamard and DCT, and the scaled error images
Image Compression-II 6
Discrete Cosine Transform
Ahmed, Natarajan, and Rao, IEEE T-Computers, pp. 90-93, 1974.
Image Compression-II 7
1-D Case: Extended 2N Point Sequence
Consider 1- D first; Let x(n) be a N point sequence 0 n N -1.
x(n)2 - N point
y(n)
pointY(u)
pointC(u)
≤ ≤
↔−
↔−
= + − − =≤ ≤ −
− − ≤ ≤ −
↔DFT N N
y n x n x N nx n n Nx N n N n N
2
2 10 1
2 1 2 1( ) ( ) ( )
( ),( ),
0 1 2 3 4
x(n)
0 1 2 3 4 5 6 7 8
y(n)
Image Compression-II 8
DCT & DFT
Y (u) = y(n)exp − j 2π2N
un
n=0
2 N −1
∑
= x(n)exp − j 2π2N
un
n=0
N −1
∑ + x(2N −1− n)exp − j 2π2N
un
n=N
2 N −1
∑
= x(n)exp − j 2π2N
un
n=0
N −1
∑ + x(m)exp − j 2π2N
u(2N −1− m)
m=0
N −1
∑
= exp j π2N
u
x(n)exp − j π
2Nu − j 2π
2Nun
n=0
N −1
∑
+ exp j π2N
u
x(n)exp j π
2Nu + j 2π
2Nun
n=0
N −1
∑
= exp j π2N
u
2x(n)cos
π2N
u(2n +1)
n=0
N −1
∑ .
Image Compression-II 9
DCT
The N-point DCTof x(n), C(u), is given by
C(u) =exp − j
π2N
u
Y (u), 0 ≤ u ≤ N-1
0 otherwise.
The unitary DCT transformations are:
F(u) = α(u) f (n)cos π2N
(2n +1)u
,n=0
N −1
∑ 0 ≤ u ≤ N −1, where
α(0) = 1N
, α(u) = 2N
for 1 ≤ k ≤ N −1.
The inverse transformation is
f (n) = α(u)F(u)cos π2N
(2n +1)u
,n=0
N −1
∑ 0 ≤ u ≤ N −1.
Image Compression-II 10
Discrete Cosine Transform—in 2-D
C(u,v) = α(u)α(v)x=0
N −1
∑ f (x, y)cos (2x+1)uπ2N
y=0
N −1
∑ cos (2y+1)vπ2N
for u, v = 0,1,2,…,N −1, where
α(u) =1N for u = 0
2N for u= 1,2,..,N-1.
and
f (x, y) =u=0
N −1
∑ α(u)α(v)C(u,v)cos (2x+1)uπ2N
v=0
N −1
∑ cos (2y+1)vπ2N
for x, y = 0,1,2,…,N −1
Image Compression-II 11
DCT Summary
T (u,v) = f (x, y)r(x, y,u,v) − − − −8.2.10y=0
N −1
∑x=0
N −1
∑
r(x, y,u,v) = s(x, y,u,v)
= α(u)α(v)cos (2x+1)uπ2 N
cos (2 y+1)vπ
2 N
..........8.2.18
Image Compression-II 12
DCT Basis functions
Image Compression-II 13
Implicit Periodicity-DFT vs DCT (Fig 8.32)
Image Compression-II 14
Why DCT?
Blocking artifacts less pronounced in DCT than in DFT. Good approximation to the Karhunen-Loeve Transform (KLT) but
with basis vectors fixed. DCT is used in JPEG image compression standard.
Image Compression-II 15
Sub-image size selection (fig 8.26)
Image Compression-II 16
Different sub-image sizes
Image Compression-II 17
Bit Allocation/Threshold Coding
• # of coefficients to keep
• How to quantize them Threshold coding
Zonal coding Threshold coding
For each subimage i • Arrange the transform coefficients in decreasing order of magnitude • Keep only the top X% of the coefficients and discard rest. • Code the retained coefficient using variable length code.
Image Compression-II 18
Zonal Coding Zonal coding: " " " " " " " " " i th coefficient
in each sub image
(1) Compute the variance of each of the transform coeff; use the subimages to compute this. (2) Keep X% of their coeff. which have maximum variance. (3) Variable length coding (proportional to variance)
Bit allocation: In general, let the number of bits allocated be made proportional to the variance of the coefficients. Suppose the total number of bits per block is B. Let the number of retained coefficients be M. Let v(i) be variance of the i-th coefficient. Then
b(i) = B
M + 12 log2 v(i) − 1
2 M log2 v(i)i=1
M
∑
Image Compression-II 19
Zonal Mask & bit allocation: an example
1 1 1 1 1 0 0 0
1 1 1 1 0 0 0 0
1 1 1 0 0 0 0 0
1 1 0 0 0 0 0 0
1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
8 7 6 4 3 0 0 0
7 6 5 4 0 0 0 0
6 5 4 0 0 0 0 0
4 4 0 0 0 0 0 0
3 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Image Compression-II 20
Typical Masks (Fig 8.36)
Image Compression-II 21
Image Approximations
Image Compression-II 22
The JPEG standard
The following is mostly from Tekalp’s book, Digital Video Processing by M. Tekalp (Prentice Hall).
For the new JPEG-2000 check out the web site www.jpeg.org.
See Example 8.17 in the text
Image Compression-II 23
JPEG (contd.)
JPEG is a lossy compression standard using DCT. Activities started in 1986 and the ISO in 1992. Four modes of operation: Sequential (basline),
hierarchical, progressive, and lossless. Arbitrary image sizes; DCT mode 8-12 bits/
sample. Luminance and chrominance channels are separately encoded.
We will only discuss the baseline method.
Image Compression-II 24
JPEG-baseline.
DCT: The image is divided into 8x8 blocks. Each pixel is level shifted by 2n-1 where 2n is the maximum number of gray levels in the image. Thus for 8 bit images, you subtract 128. Then the 2-D DCT of each block is computed. For the baseline system, the input and output data precision is restricted to 8 bits and the DCT values are restricted to 11 bits.
Quantization: the DCT coefficients are threshold coded using a quantization matrix, and then reordered using zig-zag scanning to form a 1-D sequence.
The non-zero AC coefficients are Huffman coded. The DC coefficients of each block are DPCM coded relative to the DC coefficient of the previous block.
Image Compression-II 25
JPEG -color image
RGB to Y-Cr-Cb space – Y = 0.3R + 0.6G + 0.1B – Cr = 0.5 (B-Y) + 0.5 – Cb = (1/1.6) (R – Y) + 0.5
Chrominance samples are sub-sampled by 2 in both directions.
Y1 Y2 Y3 Y4
Y5 Y6 Y7 Y8
Y9 Y10 Y11 Y12
Y13 Y14 Y15 Y16
Cr1 Cr2 Cr3 Cr4
Cb1 Cb2 Cb3 Cb4
Non-Interleaved Scan 1:Y1,Y2,…,Y16 Scan 2: Cr1, Cr2, Cr3, Cr4 Scan 3: Cb1, Cb2, Cb3, Cb4
Interleaved: Y1, Y2, Y3, Y4, Cr1, Cb1,Y5,Y6,Y7,Y8,Cr2,Cb2,…
Image Compression-II 26
JPEG – quantization matrices
Check out the matlab workspace (dctex.mat) Quantization table for the luminance channel. Quantization table for the chrominance channel. JPEG baseline method
– Consider the 8x8 image (matlab: array s.) – Level shifted (s-128=sd). – 2d-DCT: dct2(sd)= dcts – After dividing by quantiization matrix qmat: dctshat. – Zigzag scan as in threshold coding. [20, 5, -3, -1, -2,-3, 1, 1, -1, -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, 0, 0, 0, 0, 0,
1, 1, 0, 1, EOB].
Image Compression-II 27
An 8x8 sub-image (s)
s = (8x8block)
183 160 94 153 194 163 132 165
183 153 116 176 187 166 130 169
179 168 171 182 179 170 131 167
177 177 179 177 179 165 131 167
178 178 179 176 182 164 130 171
179 180 180 179 183 164 130 171
179 179 180 182 183 170 129 173
180 179 181 179 181 170 130 169
sd =(level shifted) 55 32 -34 25 66 35 4 37
55 25 -12 48 59 38 2 41
51 40 43 54 51 42 3 39
49 49 51 49 51 37 3 39
50 50 51 48 54 36 2 43
51 52 52 51 55 36 2 43
51 51 52 54 55 42 1 45
52 51 53 51 53 42 2 41
Image Compression-II 28
2D DCT (dcts) and the quantization matrix (qmat)
dcts=
312 56 -27 17 79 -60 26 -26
-38 -28 13 45 31 -1 -24 -10
-20 -18 10 33 21 -6 -16 -9
-11 -7 9 15 10 -11 -13 1
-6 1 6 5 -4 -7 -5 5
3 3 0 -2 -7 -4 1 2
3 5 0 -4 -8 -1 2 4
3 1 -1 -2 -3 -1 4 1
qmat=
16 11 10 16 24 40 51 61
12 12 14 19 26 58 60 55
14 13 16 24 40 57 69 56
14 17 22 29 51 87 80 62
18 22 37 56 68 109 103 77
24 35 55 64 81 104 113 92
49 64 78 87 103 121 120 101
72 92 95 98 112 100 103 99
Image Compression-II 29
Division by qmat (dctshat)=dcts/qmat
dcthat=
20 5 -3 1 3 -2 1 0
-3 -2 1 2 1 0 0 0
-1 -1 1 1 1 0 0 0
-1 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
dcts=
312 56 -27 17 79 -60 26 -26
-38 -28 13 45 31 -1 -24 -10
-20 -18 10 33 21 -6 -16 -9
-11 -7 9 15 10 -11 -13 1
-6 1 6 5 -4 -7 -5 5
3 3 0 -2 -7 -4 1 2
3 5 0 -4 -8 -1 2 4
3 1 -1 -2 -3 -1 4 1
Image Compression-II 30
Zig-zag scan of dcthat
Zigzag scan as in threshold coding.
[20, 5, -3, -1, -2,-3, 1, 1, -1, -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, 0, 0, 0, 0, 0,1, 1, 0, 1, EOB].
dcthat=
20 5 -3 1 3 -2 1 0
-3 -2 1 2 1 0 0 0
-1 -1 1 1 1 0 0 0
-1 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Image Compression-II 31
Threshold coding -revisited
Zig-zag scanning of the coefficients.
The coefficients along the zig-zag scan lines are mapped into [run,level] where the level is the value of non-zero coefficient, and run is the number of zero coeff. preceding it. The DC coefficients are usually coded separately from the rest.
Image Compression-II 32
JPEG – baseline method example
The DC coefficient is DPCM coded (difference between the DC coefficient of the previous block and the current block.)
The AC coef. are mapped to run-level pairs. (0,5), (0,-3), (0, -1), (0,-2),(0,-3),(0,1),(0,1),(0,-1),(0,-1), (2,1), (0,2), (0,3),
(0,-2),(0,1),(0,1),(6,1),(0,1),(1,1),EOB. These are then Huffman coded (codes are specified in the JPEG scheme.) The decoder follows an inverse sequence of operations. The received
coefficients are first multiplied by the same quantization matrix. (recddctshat=dctshat.*qmat.) Compute the inverse 2-D dct. (recdsd=idct2(recddctshat); add 128 back.
(recds=recdsd+128.)
Zigzag scan as in threshold coding. [20, 5, -3, -1, -2,-3, 1, 1, -1, -1, 0, 0, 1, 2, 3, -2, 1, 1, 0, 0, 0, 0, 0, 0,1, 1, 0, 1, EOB].
Image Compression-II 33
Decoder
Recddcthat=dcthat*qmat
320 55 -30 16 72 -80 51 0
-36 -24 14 38 26 0 0 0
-14 -13 16 24 40 0 0 0
-14 0 0 29 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Recdsd=
67 12 -9 20 69 43 -8 42
58 25 15 30 65 40 -4 47
46 41 44 40 59 38 0 49
41 52 59 43 57 42 3 42
44 54 58 40 58 47 3 33
49 52 53 40 61 47 1 33
53 50 53 46 63 41 0 45
55 50 56 53 64 34 -1 57
Image Compression-II 34
Received signal
Reconstructed S=
195 140 119 148 197 171 120 170
186 153 143 158 193 168 124 175
174 169 172 168 187 166 128 177
169 180 187 171 185 170 131 170
172 182 186 168 186 175 131 161
177 180 181 168 189 175 129 161
181 178 181 174 191 169 128 173
183 178 184 181 192 162 127 185
s = (8x8block)
183 160 94 153 194 163 132 165
183 153 116 176 187 166 130 169
179 168 171 182 179 170 131 167
177 177 179 177 179 165 131 167
178 178 179 176 182 164 130 171
179 180 180 179 183 164 130 171
179 179 180 182 183 170 129 173
180 179 181 179 181 170 130 169
Image Compression-II 35
Example
Image Compression-II 36
Wavelet Compression
Image Compression-II 37
Image Compression-II 38
Image Compression-II 39
Image Compression: Summary
Data redundancy Self-information and Entropy Error-free compression
– Huffman coding, Arithmetic coding, LZW coding, Run-length encoding – Predictive coding
Lossy coding techniques – Predictive coing (Lossy) – Transform coding
DCT, DFT, KLT, …
JPEG image compression standard