redundant slice optimal allocation for h.264 multiple description coding
DESCRIPTION
Redundant Slice Optimal Allocation for H.264 Multiple Description Coding. Tammam Tillo, Macro Grangetto, and Gabriella Olmo. CSVT, Jan 2008. Outline. Introduction Multiple description coding Redundant slice Proposed Algorithm Optimal Allocation Problem Algorithm Implementation - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/1.jpg)
Tammam Tillo, Macro Grangetto, and Gabriella Olmo
CSVT, Jan 2008
1
![Page 2: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/2.jpg)
Introduction◦ Multiple description coding◦ Redundant slice
Proposed Algorithm Optimal Allocation Problem Algorithm Implementation Experimental Results
2
![Page 3: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/3.jpg)
Multiple Description Coding (MDC)◦ Single source Multiple Descriptions
◦ D0 < D1, D2
◦ Rsingle_description = R1 + R2 – Rredundancy
Encoder 1
Encoder 2
Central decoder
Side decoder 1
Side decoder 2
Channel I
Channel 2
R1
R2
D1
D2
D0S
S’1
S’2
S’3
Introduced by MD coding 3
I I2i+1
2i
Example
![Page 4: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/4.jpg)
Redundant slice in H.264◦ A slice can be encoded and transmitted twice Called primary slice and redundant slice
◦ QPp < QPr ◦ The redundant slice is used at the decoder side if
the primary slice is lost
QPr
QPp Primary slice
redundant slice
Encoder Decoder
4
![Page 5: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/5.jpg)
MDC with redundant slices
◦ Simple post and pre-processing + H.264 decoder
I P P I
Primary Slice
Redundant Slice
Source
I P P I Description 1
Description 2
5
![Page 6: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/6.jpg)
Redundant slice◦ QPr > QPp
◦ Not for reference◦ Interlaced with primary slices◦ Drift problem◦ QPr drift redundancy◦ QPr drift (under loss rate p) Dtotal = (1-p)dprimary_slice + p(1-p)(dredundant_slice+dpropagation) + p2dall_loss Rtotal = Rprimary_slice(QPp) + Rredundant_slice(QPr)
6
![Page 7: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/7.jpg)
Impact of loss of a primary slice k on the total distortion of the current GOP with size N ◦
N
ijkmkrkt jddd
1.,, ][Distortion caused by
loss of primary slice k
Distortion of redundant slice k at frame i
Propagated distortion cause by redundant slice k after frame i
...
i i+1
Loss occurs
k
N
7
![Page 8: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/8.jpg)
Decay of propagated distortion [22]◦
dm,k = dr,k -dp,k dr,k when p,k << r,k
◦ f[n]: Power transfer function Representation of distortion attenuation Reasons
Spatial filter such as de-blocking filter Spatial interpolation such as fractional pel ME Intra blocks such as random intra replacement
iN
nkm
N
ijkm nfdjd
1,
1. ][][
]1[,, idd kmkm
...
i i+1
krd , kmd , ]1[, fd km ][, iNfd km
[22] N. Farber, K. Stuhlmuller, and B. Girod, “Analysis of error propagation in hybrid video coding with application to error resilience,” in ICIP, 1999. 8
![Page 9: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/9.jpg)
Evaluation of dm,k ◦
◦ Aligned quantizer
◦
deefeeEd km )(}{ 22,
e: Difference between the primary and redundant reconstruction (drift)f(e): pdf of e
n
jkpkp
kr
kp
kr
kp jejeeef1
,,,
,
,
, )]()([)()(
: Delta function
r
pe
Primary reconstruction
Redundant reconstruction
Source x
Source x
e
f(e)
)( prp je
p
)( prp je e = 0,p, 2p,…f(e) = p/r at p|e
9
![Page 10: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/10.jpg)
Evaluation of dm,k ◦
◦ p,k << r,k dm,k dr,k
2
,
,2
,
1
2
,
3,
1,,
2
,
,2,
112
)]()([)(
kr
kpkr
n
jkr
kp
n
jkpkp
kr
kpkm
jc
dejejeedeefed
Assumption:r,k/p,k = 2n+1dr,k = r,k
2/12
10
![Page 11: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/11.jpg)
Evaluation of dm,k ◦ Not aligned quantizer
◦
r
p e
Reconstructed level
Reconstructed level
Source x
Source x
e
f(e)
r
r1
12
1)(
2,
2/
2/ ,
22,
,
,
kr
krkm
kr
kr
deedeefed
e can be any numberf(e) = 1/r
11
![Page 12: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/12.jpg)
Accuracy of evaluated dm,k
Real distortion – evaluated dm,kReal distortion
12
![Page 13: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/13.jpg)
Total GOP distortion dt,k ◦
Expected GOP distortion caused by loss of slice k at loss rate p◦
Expected GOP distortion caused by random loss of a slice◦
ikr
iN
nkr
iN
nkrkr
iN
nkmkrkt dnfdnfddnfddd .
0.
1.,
1.,, ][][][
ktkp
kktkpk
dppdp
dpdppdpd
,,
,02
,,
)1()1(
)1()1(
N
i
N
iiirip
kk DpDpdD
1 1,,)1(
f[0] = 1
iN
ni nf
0
][
low loss rate
Loss of primary sliceUsing primary slice Loss of primary and redundant slice
ik
krir dD ,,
ik
kpip dD ,,
Expected frame distortion
13
![Page 14: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/14.jpg)
Optimization problem◦
◦ Lagrangian approach: min J = D + R ◦
Dmin
GOP1
,, )( subject to RRRN
iirip
0)1(
0)1(
,
,
.
,
,
.
ir
iri
ir
ip
ip
ip
R
Dpp
R
J
R
Dp
R
J
ir
iri
ip
ip
R
Dp
R
D
,
,
,
,
1,
1,1
,
,
1,
1,
1,
1,
r
r
iir
ir
r
ri
p
p
p
p
R
D
R
D
R
Dp
R
D
R
D
Since Dp,i/ Rp,i is independent of i
14
![Page 15: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/15.jpg)
Optimal conditions◦
Redundant data < primary data i > i+1
1,
1,1
,
,
1,
1,
r
r
iir
ir
r
ri
p
p
R
D
R
D
R
Dp
R
D
15
![Page 16: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/16.jpg)
Encoder
Decoder
QPp QPr,i
Post processing
Channel 1
Channel 2
Channel 1
Channel 2
Preprocessing
Merge
H.264 Standard decoding
Redundant slice
16
![Page 17: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/17.jpg)
Determination of RD parameters◦
◦ H.264 reference software: QPp = QPr,1 + 3log(p1)
QPr,i = QPr,1 + 3log(1/i)
◦ In [22], assuming , deducing
Taking
3
12
285.0QP
RD1,
1,1
,
,
1,
1,
r
r
iir
ir
r
ri
p
p
R
D
R
D
R
Dp
R
D
-
nenf ][
yxyxuuyxtv ddHt ),(),(4
1][
2
22
22
1
1][ uv t
t
Variance of drift v[x,y,t]
Power spectral density of u[x,
y]= v[x,y,0]
Linear system (decoder)
)1()1( )1( ee iNi
Small for serious packet loss17
![Page 18: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/18.jpg)
(QPi) = QPr,i – QPp ◦ i QPr,i◦ p QPr,i◦ QPr,i
18
![Page 19: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/19.jpg)
Parameters◦ H.264 reference software JM9.4◦ P-slice with 5 reference pictures◦ 5 slices for a CIF picture and 3 slices for a QCIF
picture◦ Bernoulli model for PLR = p ◦ foreman and coastguard at CIF and QCIF format
are used
19
![Page 20: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/20.jpg)
Selection of ◦ Not sensitive to
20
![Page 21: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/21.jpg)
RD comparisons under different p and N (=0.4)◦ Shorter GOP size for larger PLR
p=0.01p=0.05
p=0.1
N=11
N=21N=45
21
![Page 22: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/22.jpg)
Redundancy (=0.4)◦
i irip
i ir
RR
R
)( ,,
,
p
p
R
D
1,
1,
r
r
R
D
22
![Page 23: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/23.jpg)
Comparisons with other MDC◦ Four descriptions by subsampling CIF to QCIF◦ Two descriptions by separating odd and even
rows Other MDC schemes:N = 21 with P and B picturesExtra computation is needed at the decoder side
23
![Page 24: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/24.jpg)
Central decoder versus side decoder◦ p redundancy (central decoder) and (side
decoder)
No loss
24
![Page 25: Redundant Slice Optimal Allocation for H.264 Multiple Description Coding](https://reader036.vdocuments.mx/reader036/viewer/2022062518/5681477a550346895db4b154/html5/thumbnails/25.jpg)
Central decoder versus side decoder (p=0.05)
25