error control and concealment for video communication cmpt820 summer 2008 michael jia
Post on 19-Dec-2015
235 views
TRANSCRIPT
“Error Control and Concealment for Video Communication : A
Review”
YAO WANG, Member, IEEEQIN-FAN ZHU, Member, IEEE
Reference
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
2 Types of Transmission Errors
Random Bit Errors
Bit inversion, bit insertion, bit deletion
Erasure Errors
Packet loss, burst errors, system failures
More destructive
Common usage of VLC makes them no differences in video
streaming
Introduction
Lossless recovery
FEC (Forward Error Correction)
ECC (Error Control Coding)
ARQ (Automatic Retransmission Request)
Not necessary for video transmission, human eyes can
tolerate a certain degree of distortion
Focus on signal-reconstruction and error-concealment
techniques
Introduction
3 groups of error-concealment techniques Forward error concealment
Encoder plays the primary role Error concealment by post-processing
Decoder fulfills the task Interactive error concealment
Main concerns Effectiveness Required delay Bit-rate overhead Processing complexity
Introduction
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
At transport layer
Adding header information (sequence number)
H.223
FEC (Forward Error Correction)
H.223, H.261
More reliable
Need more bandwidth
Error Detection
At decoder Based on characteristics of natural video signals
Pixel value differences of neighboring lines (compare to a threshold) Differences between boundary pixels in a block and its four neighbor
blocks Obvious false value of quantization step size or DCT coefficients Insert synchronization code word at the end of line of blocks
No additional bits or very few Rely on smoothness of signal
Error Detection
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
Layered Coding with Transport Prioritization Most popular and effective (MPEG-2)
Error Concealment at Coder (1)
Layered Coding Base layer – most important layer, with acceptable quality
Transport prioritization Deliver base layer with higher degree of error protection High priority channel Re-transmission and/or FEC
No explicit bit-rate overhead Complicate structure and coding overhead H.264 AVC/SVC
Redundant pictures Data Partitioning
Error Concealment at Coder (1)
Multiple-Description Coding (MDC)
Several parallel channels Independent error events Small probability of “all channels down”
Multiple “Descriptions” Several coded bit streams Transmitted over separate channels Any one will work
Error Concealment at Coder (2)
Joint Source and Channel Coding
Source-channel interaction at a lower level
Given channel error characteristics, design quantizer and entropy
coder for to minimize the effect of errors
For general sources, noisy channel coarse quantizer is better
For image signals, noisy channel
fewer bits to high-frequency coefficients
more bits to low-frequency coefficients
Error Concealment at Coder (3)
Robust Waveform Coding
Intentionally keep some redundancy in source-coding
Layered coding and MDC both belong to this category
Adding auxiliary information in waveform coder
MPEG-2: sending motion vectors for microblocks in I-frames
Restricting prediction domain
H.263/H.264: prediction is confined within each slice
Error Concealment at Coder (4)
Robust Entropy Coding
Add redundancy in entropy-coding
To help detect bit errors and prevent error propagation
Self-Synchronizing entropy coding
Add a synchronization code word
H.261, H.263, MPEG-4
Error-Resilient entropy coding (EREC)
MPEG-4 uses RVLC (reversible VLC)
Error Concealment at Coder (5)
Transport-Level Control
Add redundancies at transport level
Prioritized transport for layered coding
Robust packetization
Spatial block interleaving
Dual transmission of important information
H.264 AVC/SVC
NAL unit syntax structure
Parameter Sets
Error Concealment at Coder (7)
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
Preview
Perform error concealment at the decoder
Can be used in conjunction with the auxiliary information provided
by the source coder
Low frequency components dominate images of natural scenes
Color values of adjacent pixels vary smoothly except sharp edges
Human eyes tolerate more distortion to high-frequency
components
Error Concealment at Decoder
Motion-Compensated Temporal Prediction
Replace damaged MB with the motion compensated block
Very effective when all the motion information in the base layer
Widely used (MPEG-2)
What if motion information or coding mode is lost?
Will discuss in next slide
Error Concealment at Decoder (1)
Recovery of Motion Vectors and Coding Modes What if we lost motion vectors or coding modes?
Interpolated from spatially and temporally adjacent blocks Estimation of coding modes
Simple: treat as ‘intracoded’ More sophisticate: MPEG-2 (See tables)
Error Concealment at Decoder (2)
Estimation of motion vectors
Set to zeros (works well for low motion video)
Use MV of the corresponding block in the previous frame
Use the average of MVs from spatially adjacent blocks
Use the median of MVs from spatially adjacent blocks
Select one of the above methods depending on least boundary
matching error
Error Concealment at Decoder (2)
Maximally Smooth Recovery
A constrained energy minimization approach
Minimize a measure of spatial and temporal variation between
adjacent pixels in this block and its spatially and temporally
neighboring blocks
Measure differences in 3 domains
Spatial – adjacent blocks
Temporal – prediction block in previous frame
Frequency – received coefficients for this block
Error Concealment at Decoder (3)
Spatial- and Frequency-Domain Interpolation
A coefficient in a damaged block is likely to be close to the
corresponding coefficients in spatially adjacent blocks
Interpolate from four neighbor blocks
Maybe not accurate (8-pixel is too far)
Interpolate from four 1-pixel-wide boundaries
2 pixels in 2 nearest boundaries
4 pixels in all 4 boundaries
See picture in next slide
Error Concealment at Decoder (4)
Spatial- and Frequency-Domain Interpolation Interpolate from four 1-pixel-wide boundaries
Error Concealment at Decoder (4)
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
Preview
If a backward channel is available, can achieve better performance
by cooperation
Based on the feedback
At source coder – coding parameters can be adapted
At transport level – adjust the portion of bandwidth used for error control
Decoding delay issue
Interactive Error Concealment
Selective Encoding for Error Concealment
Simple – code next frame in intramode
Error stopped in about one round-trip time
Will cause bit-rate increase
Send identity info back, perform error concealment at the same
time
Continue encode without using the affected area
Perform same error concealment procedure (need a prediction frame
buffer)
See picture in next slide
Interactive Error Concealment (1)
H.263 uses more prediction frame buffers (reference picture selection mode)
Interactive Error Concealment (1)
Adaptive Transport for Error Concealment
Retransmission is unacceptable for real-time video applications?
Not always
For one-way video broadcast, we may tolerate a few seconds delay
For multipoint video conferencing, use MCU (multipoint control unit)
If retransmission is controlled properly, end-to-end quality can be
improved
Both H.323 and H.324 defined such mechanisms
Interactive Error Concealment (2)
Retransmission Without Waiting
Wait for the retransmission data
Not good, may freeze the display
Cause transit delay and accumulation delay
Without waiting
Request the retransmission
Conceal the error
Track the affected pixels
Correct them upon the arrival of the retransmission data
Interactive Error Concealment (3)
Prioritized, Multi-copy Retransmission
Effective in very lossy channels
Video streaming via PSTN
Are you kidding me?
Send multiple copies of a lost packet
Use layered coding
# of retransmission trials and # of copies are proportional to the
importance of the layer
Interactive Error Concealment (4)
Introduction
Error Detection
Error Concealment at Coder
Error Concealment at Decoder
Interactive Error Concealment
Conclusion
Outline
Conclusion
Real-time video communication doesn’t require lossless
delivery; signal-reconstruction and error-concealment
techniques are more effective.
Add redundancy when encoding or delivering
Estimate missing information when decoding
Inform sender what is lost
Burstiness has a significant impact on the choice of algorithms