Overview of the H. 264/AVC video coding standard

History and Naming

Goal of “The Standard”

• Deploy over existing & future networks in these areas with substantially low bitrate:– Broadcast

– Interactive or serial storage

– Conversational services

– Video-on-demand or multimedia streaming services

– Multimedia messaging services

• Flexibility and Customizability:

Comparison: PSNR

• H.264 improves the coding efficiency over other coding methods

Design Feature Highlights

• Enhancements of coding efficiency– Better prediction

• Variable block-size motion compensation with small block sizes• Quarter-sample-accurate motion compensation• Multiple reference picture motion compensation• Weighted prediction

– Other factors• Hierarchical block transform• Short word-length transform

• Enhancements of robustness and flexibility for operation over a variety of networks

Structure of H.264 Video Encoder

Network Abstraction Layer

• Facilitates the ability to map H.264 VCL data to transport layers such as RTP/IP, H.32x, MPEG-2.– NAL Units– VCL and non-VCL units– Parameter sets– Access units– Coded video sequence

NAL: Parameter Sets

VCL: Revolution or Evolution?• There is no single coding element in the VCL that

provides the majority of the significant improvement in compression efficiency in relation to prior video coding standards.

• It is rather a plurality of smaller improvements that add up to the significant gain.

• The basic source-coding algorithm is a hybrid of inter-picture prediction to exploit temporal statistical dependencies and transform coding of the prediction residual to exploit spatial statistical dependencies.

VCL: Elements

• Pictures, Frames, Fields• Macroblocks (MBs)(16x16)

• Slices, Slices Groups

VCL: Coding the Slices

• Each slice can be coded using different coding types as follows:– I slice: A slice in which all macroblocks of the slice are coded using

intra prediction.

– P slice: In addition to the coding types of the I slice, some macroblocks of the P slice can also be coded using inter prediction with at most one motion-compensated prediction signal per prediction block.

– B slice: In addition to the coding types available in a P slice, some macroblocks of the B slice can also be coded

– SP slice (switching P slice): coded such that efficient switching between different pre-coded pictures becomes possible.

– SI slice (switching I slice): allows an exact match of a macroblock in an

SP slice for random access and error recovery purposes.

Peek into Encoder

Adaptive Frame/Field Coding

• 3 choices:– To combine the two fields together and to code them

as one single coded frame (frame mode). – To not combine the two fields and to code them as

separate coded fields (field mode). – To combine the two fields together and compress

them as a single frame, but when coding the frame to split the pairs of two vertically adjacent macroblocks into either pairs of two field macroblocks or frame macroblocks before coding them.

Adaptive Frame/Field Coding

• PAFF (Picture-Adaptive Frame/Field)– The choice between first 2– >16% better than field only in general

• MBAFF (MacroBlock-Adaptive Frame/Field )– Non-moving: frame mode, moving: field mode– Use a pair of MBs (16x32 luma) – >14% better than PAFF in general

Intra-Frame Prediction

Constrained intra coding mode: prediction only from intra-coded neighboring MBs

3 modes: Intra_4x4, Intra_16x16 (with chroma prediction) and I_PCM

Inter-Frame Prediction• Full sample, half sample, one-quarter

sample– Half: apply one-dimensional 6-tap FIR filter– One-quarter: average full and half

• Results:– Accurate motion representation– More flexibility in prediction filtering

• Different degrees of low pass filtering could be chosen

• better preserving hi-frequency content.

Transform, Scaling and Quantization• 4x4 Integer transform vs. 8x8 DCT

– No mismatch in inverse transform– Less computations, short processing wordlength– Less noise

•52 quantization values; step size: 12%• one step means 12% bit rate change

• 6 steps: factor of 2

Entropy Coding• CAVLC (Context-Adaptive Variable Length

Coding)– VLC tables for various syntax elements are switched depending

on already transmitted syntax elements. – Since the VLC tables are designed to match the corresponding

conditioned statistics, performance is improved in comparison to schemes using a single VLC table.

• CABAC (Context-Adaptive Binary Arithmetic Coding)– allows the assignment of a noninteger number of bits to each

symbol of an alphabet– permits adaptation to nonstationary symbol statistics– uses only shifts and table look-ups – Compared to CAVLC, typically provides a reduction in bit rate

5%–15%

In-Loop Deblocking Filter

• Goal: reduce blockiness and maintain sharpness

Profiles and Levels

• Profiles: Facilitate interoperability between various applications.– Defines a set of coding tools or algorithms that can be used in

generating a conforming bitstream– Incl. Baseline, Main, Extended, High(4)

• Levels: place constraints on certain key parameters of the bitstream– Correspond to processing power and memory capability of a

codec– Set limts for data bitrate, framesize, pic buffer size. Eg. QCIF,

ITU-R601, HDTV etc.– Universal (15) used in different profile

Profiles: Coding Features

Computation Consumption• Use large amount of computation to achieve

high quality, reliability and compression ratio– Intra/inter frame prediction: ~60%– Integer transform: ~10%– Error resilience: ~20%– De-blocking filter: ~10%

• Encoding: ~10x of MPEG-2’s CPU usage• For current home PC users, realtime software

decoding (HDTV quality) could be achieved, but not encoding

• Recommend software:– QuickTime Alternative (www.free-codecs.com)– X264 (www.video-lan.org/x264.html)

Licensing

• Standard: no free lunch

• Lessons from MPEG-4 Visual– Was not friendly at the beginning…

• H.264– Codec manufactures pay for the consumer

use by/between end users– Cheap for free television provider– Free internet broadcast till 2010.12.31

Applications• HD DVD Video (DVDForum); Blu-ray Disk

(BDA) (High Profiles: mandatory)

• DVB in Europe, DMB in Korea, ISDB-T in Japan

• Direct broadcast satellite TV (multi-region)

• 3GPP (optional feature)

• IETF: RFC3984 for RTP

• ITU-T: H.32x (in production)

• VoD