institute of electronics, national chiao tung university scalable extension of h.264/avc student:...

48
Institute of Electronics, National Chiao Tung University Scalable Extension of H.264/AVC Student: Hung-Chih Lin Advisor: Prof. Hsueh- Ming Hang

Post on 22-Dec-2015

219 views

Category:

Documents


0 download

TRANSCRIPT

Institute of Electronics, National Chiao Tung University

Scalable Extension of H.264/AVC

Student: Hung-Chih Lin

Advisor: Prof. Hsueh-Ming Hang

2660/03/30 hclin 2

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

References• [1] Reichel, J., Hanke, K., Popescu, B.: Scalable

Video Coding V1.0. ISO/IEC JTC1/SC29/WG11, N6372 (2004)

• [2] H. Schwarz, D. Marpe, and T. Wiegand, “Scalable Extension of H.264/AVC”, ISO/IEC JTC1/WG11 Doc. M10569/S03, Mar. 2004.

• [3] I. Daubechies and W. Sweldens, “Factoring wavelet transforms into lifting steps”, J. Fourier Anal. Appl. 4(3), pp. 245-267, 1998.

• [4] J. Reichel, H. Schwarz, and M.Wien, "Joint Scalable Video Model JSVM-2," 17th JVT meeting, JVT-Q202, Nice, France.

• [5] Tabatabai, A., Visharam, Z., Suzuki, T.: Compariosn of MCTF and closed-loop hierarchical B pictures. ISO/IEC JTC/SC29/WG11 and ITU-T SG16 Q.6, JVT-P059 (2005)

2660/03/30 hclin 3

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Outline

• Overview• MCTF in JSVM• Scalability Concepts • JSVM Reference Software

2660/03/30 hclin 4

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Outline

• Overview– Motivation– Scalable Video Coding

• MCTF in JSVM • Scalability Concepts• JSVM Reference Software

2660/03/30 hclin 5

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Motivation

• To support clients with diverse capabilities in complexity, bandwidth, power, and display resolution.

Ethernet

Ethernet

Server

Wireless

Point-to-PointTransmission

Broadcasting

Router

Wireless

512 kbps

32 kbps

128 kbps

256 kbps

64 kbps

3 Mbps

1.5 Mbps

384 kbps

64 kbps

Bandwidth

Time

2660/03/30 hclin 6

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Scalable Video Coding

• Approaches– wavelet-based

• 2D+t structure• t+2D structure

– AVC-based• Layered coding concept

2660/03/30 hclin 7

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Approaches

A wavelet-based approach with 2D+t structure

Video

Entropy Coding

Motion Coding

Texture Coding

Spatio-Temporal "Transform"

2D Spatial DWT

Temporal TransformMCTF based

2D SpatialDecomposition

LL

Bitstream

In band TemporalTransform MCTF based

2D SpatialDecomposition

HF

2660/03/30 hclin 8

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Approaches

A wavelet-based approach with t+2D structure

Video

Entropy Coding

Motion Coding

Texture Coding

Spatio-Temporal "Transform"

5/3 based MCTF 2D Spatial DWT

Bitstream

2660/03/30 hclin 9

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Approaches

Spatio-Temporal "Transform"

Video

Entropy Coding

Motion Coding

Layered AVC TextureCoding

2D SpatialDecimation

QCIF

Bitstream

CIF

AVC 4x4 integer transform

AVC 4x4 integer transform

2D SpatialInterpolation

5/3 MCTF

5/3 MCTF

An AVC/H.264-based structure

2660/03/30 hclin 10

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Scalabilities

• Temporal– fps

• Spatial– resolution

• SNR/Rate– quality

scheme Temporal

Spatial SNR/Rate

wavelet-based MCTF wavelet transform

(multi-resolution)

zero-tree coding

AVC-based MCTF Layered coding

CABAC (CGS)

Bit-plane coding (FGS)

2660/03/30 hclin 11

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Outline

• Overview• MCTF in JSVM

– Why MCTF ?– Base layer structure– Inter layer prediction– Adaptive Prediction/Update Steps – Progressive MCTF

• Scalability Concepts• JSVM Reference Software

2660/03/30 hclin 12

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Why MCTF?

• MCTF = Motion-Compensated Temporal Filtering

• A temporal sub-band coding– 2-channel filter bank in temporal

direction• Performs the wavelet decomposition /

reconstruction along the motion trajectory

• Implementation technique– Lifting scheme (the main reason) : Any

bi-orthogonal wavelet filters can be factorized by prediction and update steps

2660/03/30 hclin 13

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Lifting scheme

• Attraction– An in-place implementation like

FFT.– Easy to build non-linear WT.– Insure PR.– All operations within one lifting

step can be done entirely parallel.

• Computational complexity– ~40% of original one (depend on

the wavelet filter)

2660/03/30 hclin 14

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Lifting scheme

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

2660/03/30 hclin 15

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps• 2-channel Filter

Bank

• Bi-orthogonal

2H0(z) 2 y[n]x[n] F0(z)

2H1(z) 2 F1(z)

2 2 y[n]x[n]

2 2

1H z

1G z

H z

G z

2660/03/30 hclin 16

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps• PR condition

• Define

1 1

1 1

2

0

H z H z G z G z

H z H z G z G z

H z H zz

G z G z

M

1 2z z M M I

2660/03/30 hclin 17

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps• Type 1 polyphase representation

• Define

2 1 2

2 1 2

e o

e o

H z H z z H z

G z G z z G z

e e

o o

H z G zz

H z G z

P

2 11

12

zz z

z

z z

P M

P P I

2660/03/30 hclin 18

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps• Noble identities

H(zL) L H(z) L

H(z) MH(zM) M

2660/03/30 hclin 19

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps• We want and are FIR.

• By Euclidean algorithm, we can get

zP zP

det z cz P det 1z assumption P

1

1 0 01

1 0 1/0 1

mi

i i

Ks zz

t z K

P

1

11

1 0 1 1/ 0

1 00 1

mi

ii

t z Kz

s z K

P

2660/03/30 hclin 20

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Factoring Wavelet Transforms into Lifting Steps

2660/03/30 hclin 21

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Wavelet filters

• 2-2 Filter Bank (Haar)

• 5-3 Filter Bank

2,2 2,2

11 1 1 1

2L H

2,2 2,20 0

11

2P U

5,3 5,33

1 11 2 6 2 1 1 2 1

2 2L H

5,3 5,30 0 2

1 11 1 1 1

2 2P U

2660/03/30 hclin 22

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Lifting scheme

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

],[2

1,

]2,[2,

khkh

ksks

Haar

Haar

xxU

xxP

]1,[],[4

1,

]22,[]2,[2

12,

3/5

3/5

khkhkh

ksksks

xxxU

xxxP

2660/03/30 hclin 23

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

MCTF

H H H H

H 2 H 2

H 3

H H H H 1

H 2 H 2

H 3

L H 4

15Hz Video Sequence

7.5Hz Video Sequence

30Hz Video Sequence

3.25Hz Video Sequence

2660/03/30 hclin 24

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

MCTF

(a) Without M.C. (b) With M.C.

2660/03/30 hclin 25

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Lifting scheme

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

2

2

P USk

hk

lk

z-1

S2k+1

S2k

U P

2

2

Sk

S2k

S2k+1

z

Fh

Fl

Fh-1

Fl-1

Lifting Scheme(Analysis Filterbank)

(a)

Inverse Lifting Scheme(Synthesis Filterbank)

(b)

0 0

0 0

5/3 0 0 1

5/3 0 0 1

, 2 1 ,2 2

1,2 , 2

21

,2 1 ,2 2 ,2 2 22

1,2 , , 1

4

Haar P P

Haar U U

P P P

U U U

P s k s k r

U s k h k r

P s k s k r s k r

U s k h k r h k r

x x m

x x m

x x m x

x x m x

2660/03/30 hclin 26

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Base layer Structure

• Compatible with AVC Main profile– Dyadic hierarchical B pictures– Only prediction step is performed.

(UMCTF)

I0/P0 B1B2B3 I0/P0 I0/P0B3 B3 B3B3 B3 B3 B3B2 B2 B2B1

0 1221 8 167 9 153 5 11 136 10 144display order

group of pictures (GOP) group of pictures (GOP)

I0/P0 B1B2B3 I0/P0 I0/P0B3 B3 B3B3 B3 B3 B3B2 B2 B2B1

0 1221 8 167 9 153 5 11 136 10 144display order

group of pictures (GOP) group of pictures (GOP)

2660/03/30 hclin 27

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Base layer Structure

• Non-dyadic decomposition is available– Temporal scalability

H1

L0

H2

L1

L0

H1

L0

L2

H3

L1

L0

H1

L0

H2

L1

L0

H1

L0

L2

H3

L1

L0

H1

L0

H2

L1

L0

H1

L0

L2

L3

L1

L0

H1H1

L0L0

H2H2

L1L1

L0L0

H1H1

L0L0

L2L2

H3H3

L1L1

L0L0

H1H1

L0L0

H2H2

L1L1

L0L0

H1H1

L0L0

L2L2

H3H3

L1L1

L0L0

H1H1

L0L0

H2H2

L1L1

L0L0

H1H1

L0L0

L2L2

L3L3

L1L1

L0L0

Level 0: full resolution

Level 1: 1/2 of the full resolution

Level 2: 1/4 of the full resolution

Level 3: 1/12 of the full resolution

2660/03/30 hclin 28

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Inter Layer Prediction

• Remove the redundancy among the different layers– Residues– Motion vectors

2660/03/30 hclin 29

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Inter Layer Prediction

Video Bitstream

2D Decimation(by 2)

MultiplexMCTF

Motion CodingMotion

TextureSpatial Transform -

SNR ScalableEntropy Coding

2D Decimation(by 4)

MCTF

Motion CodingMotion

Texture

MCTF

Motion CodingMotion

Texture

Prediction

Prediction

Prediction

Interpolation

Interpolation

Spatial Transform -SNR Scalable

Entropy Coding

Spatial Transform -SNR Scalable

Entropy Coding

2660/03/30 hclin 30

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Adaptive Prediction/Update Steps• Goal

– Control the encoding delay

• Method– GOP is partitioned into sub-groups

• Restrictions : no across the partition boundary– Backward prediction steps– Backward and forward update steps

2660/03/30 hclin 31

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Adaptive Prediction/Update Steps

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

H

0

H H H H H H H

L L L L L L L L

H H H H

L L L L

H H

L L

H

L

80 G 80 G30 C 30 C

1 0 2 4 6 5 7 3 9 8 10 12 14 13 15 11coding order

prediction

update

prediction

update

prediction

update

prediction

update

GOP border partition bordersub-partition

borderGOP border

sub-partition border

2660/03/30 hclin 32

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Progressive MCTF

• Prediction steps and update steps are interlaced.

• Process the pictures in the reverse display order.

2660/03/30 hclin 33

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Progressive MCTF

1 2 3 4 5 6 7 8

H

0

H H H

L L L L

H H

L L

H

L

prediction

update

prediction

update

prediction

update

GOP border GOP border

L´L´

12457813

6914 3

101115

1216

18

17

Numbering of prediction and update steps

2660/03/30 hclin 34

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Outline

• Overview• OMCTF in JSVM • Scalability Concepts

– Three Scalabilities– Slice Types– Combined scalability

• JSVM Reference Software

2660/03/30 hclin 35

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Temporal Scalability

L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0

H2L2 H2

L2 H2L2

H3

L3

H3

L1 H1L1 H1

L1 H1L1 H1

L1 H1L1 H1 {MP}1

{MP}2

{MP}3Temporal Enhancement Layer (Layer 1)

Temporal Base Layer (Layer 0)

Temporal Enhancement Layer (Layer 2)

Temporal Enhancement Layer (Layer 3)

L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0

H2H2L2L2 H2H2

L2L2 H2H2L2L2

H3H3

L3L3

H3H3

L1L1 H1H1L1L1 H1H1

L1L1 H1H1L1L1 H1H1

L1L1 H1H1L1L1 H1H1 {MP}1{MP}1

{MP}2{MP}2

{MP}3{MP}3Temporal Enhancement Layer (Layer 1)

Temporal Base Layer (Layer 0)

Temporal Enhancement Layer (Layer 2)

Temporal Enhancement Layer (Layer 3)

2660/03/30 hclin 36

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Spatial Scalability

L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*

L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1

L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0

Spatial Base Layer (Layer 0)

Spatial Enhancement Layer (Layer 1)

reconstructedsequence

reconstructedand upsam pledsequence

H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1

reconstructedsequence

temporalsubbandpictures

Spatial upsampling

Base Layer Prediction

Reconstruction

L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0* L0*

L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1

L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0 L0

Spatial Base Layer (Layer 0)

Spatial Enhancement Layer (Layer 1)

reconstructedsequence

reconstructedand upsam pledsequence

H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1H1 H1 H1 H1 H1 L1 H1 H1 H1 H1 H1 H1

reconstructedsequence

temporalsubbandpictures

Spatial upsampling

Base Layer Prediction

Reconstruction

Interpolation filter: {1,-5,20,20,-5,1}

2660/03/30 hclin 37

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

SNR Scalability

Transform,Scal. / Quant.

EntropyCoding

Inv. Scaling,Inv. Transform

MC / IntraPrediction

+

++

-

-

Transform,Scal. / Quant.

EntropyCoding

Inv. Scaling,Inv. Transform

+ -

Transform,Scal. / Quant.

EntropyCoding

SNR Base Layer (Layer 0)

SNR Enhancement Layer (Layer 1)

SNR Enhancement Layer (Layer N-1)

General

Decomposition

of a Group of

Pictures using

Motion-Comp.

Temporal

Filtering

(MCTF) ...

Group of Pictures

.

.

.

Tem

pora

l su

bban

d P

ictu

res

.

.

.

Transform,Scal. / Quant.

EntropyCoding

Inv. Scaling,Inv. Transform

MC / IntraPrediction

+

++

-

-

Transform,Scal. / Quant.

EntropyCoding

Inv. Scaling,Inv. Transform

+ -

Transform,Scal. / Quant.

EntropyCoding

SNR Base Layer (Layer 0)

SNR Enhancement Layer (Layer 1)

SNR Enhancement Layer (Layer N-1)

General

Decomposition

of a Group of

Pictures using

Motion-Comp.

Temporal

Filtering

(MCTF) ...

Group of Pictures

.

.

.

Tem

pora

l su

bban

d P

ictu

res

.

.

.

2660/03/30 hclin 38

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Slice Types

SliceType

Supported macroblock modes

INTRA_4x4 INTRA_16x16 INTRA_PCM INTRA_BASE RESIDUAL motion-compensated modes

M X(1) X

I X X X

P X X X X

B X X X X

IE X X X X

PE X X X X X

BE X X X X X

E X

H X X X X(2)

HE X X X X X(2)

(1) For M slices, the intra mode is called INTRA and it is not identical to the INTRA_4x4 mode.(2) The residual mode (RESIDUAL) is not indicated by the syntax element mb_type, instead the macroblocks that

are coded in residual mode are specified by the corresponding prediction data array.

2660/03/30 hclin 39

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Slice TypesSlice Type Usage

M Coding of prediction data arrays

I

Coding of base-layer (SNR, spatial) representations of low-pass picturesP

B

IE

Coding of enhancement-layer (SNR, spatial) representations of low-pass picturesPE

BE

ECoding of SNR enhancement-layer representations of high-pass picturesCoding of enhancement-layer (SNR, spatial) representations of low-pass pictures

H Coding of base-layer (SNR, spatial) representations of high-pass pictures

HE Coding of spatial enhancement-layer representations of high-pass pictures

2660/03/30 hclin 40

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Combined Scalability

H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0

I B P B P B

H20 H1

0 L20 H1

0 H20 H1

0

Spatial upsampling

H21 H1

1 L21 H1

1 H21 H1

1

H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1

{MP}1,2

{MP}0

Layer 0: QCIF, 7.5 Hz, 64 kbit/s

Layer 1: QCIF, 15 Hz, 128 kbit/s

Layer 2: CIF, 15 Hz, 256 kbit/s

Layer 3: CIF, 15 Hz, 512 kbit/s

Layer 4: CIF, 30 Hz, 1024 kbit/s

Layer 5: CIF, 30 Hz, 2048 kbit/s

H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0

I B P B P BI B P B P B

H20 H1

0 L20 H1

0 H20 H1

0H20 H1

0 L20 H1

0 H20 H1

0

Spatial upsampling

H21 H1

1 L21 H1

1 H21 H1

1H21 H1

1 L21 H1

1 H21 H1

1

H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1

{MP}1,2{MP}1,2

{MP}0{MP}0

Layer 0: QCIF, 7.5 Hz, 64 kbit/s

Layer 1: QCIF, 15 Hz, 128 kbit/s

Layer 2: CIF, 15 Hz, 256 kbit/s

Layer 3: CIF, 15 Hz, 512 kbit/s

Layer 4: CIF, 30 Hz, 1024 kbit/s

Layer 5: CIF, 30 Hz, 2048 kbit/s

2660/03/30 hclin 41

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Outline

• Overview• OMCTF in JSVM • Scalability Concepts• JSVM Reference Software

– Tools– UMCTF at Decoder

2660/03/30 hclin 42

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Tools

• Converter – Spatial domain

• Upsample– Interpolation FIR filter

• Downsample– Apply an anti-aliasing FIR filter proir to

2D downsampling

– Temporal domain

• PSNR

2660/03/30 hclin 43

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

UMCTF at Decoder

• Update step– Improve coding efficiency– Increase significantly complexity

of the decoder operation• Additional M.C. operations• Picture buffer management• M.V. needs intensive branch

instructions

2660/03/30 hclin 44

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

UMCTF at Decoder

• UMCTF => update step at decoder side is omitted– The visual quality and PSNR of the

decoded video is not degraded– UMCTF → purely predictive

structure– Reduce the complexity of decoder

by 50%

2660/03/30 hclin 45

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Normal Mode

STOCKHOLM

34.00000

34.50000

35.00000

35.50000

36.00000

1 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Stockholm_DEC_U_ON

Stockholm_DEC_U_OFF

CREW

35.50000

36.00000

36.50000

37.00000

37.50000

38.00000

38.50000

39.00000

39.50000

40.00000

1 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Crew_Dec_U_ON

Crew_Dec_U_OFF

2660/03/30 hclin 46

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

High Quality (Qp = 0)

40

45

50

55

60

65

70

75

80

85

901 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Stockholm_QP0_UpdateDec_ON Stockholm_QP0_UpdateDec_OFF

50

55

60

65

70

75

80

1 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Crew _QP0_UpdateDec_ON Crew _QP0_UpdateDec_OFF

2660/03/30 hclin 47

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Qp = 24

35

36

37

38

39

40

41

42

431 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Stockholm_QP24_UpdateDec_ON Stockholm_QP24_UpdateDec_OFF

39

40

41

42

43

44

45

1 4 7 10

13

16

19

22

25

28

31

34

37

40

43

46

49

52

55

58

61

64

Crew _QP24_UpdateDec_ON Crew _QP24_UpdateDec_OFF

2660/03/30 hclin 48

Institu

te of E

lectron

ics, Natio

nal C

hia

o T

ung

Un

iversity

Thank you !!