ENCODER COMPLEXITY REDUCTION WITH SELECTIVE MOTION MERGE IN HEVC

ENCODER COMPLEXITY REDUCTION WITH SELECTIVE MOTION MERGE IN HEVCby Abhishek Hassan Thungaraj

Supervisor- Dr. K. R. RaoOutlineBasics of a Digital VideoNeed for compressionAreas for compressionVideo codecs Introduction to HEVCFeatures of HEVCPresent and Proposed algorithmsExperimental conditions and ResultsConclusions

Digital VideoWhat makes a video? Group of Pixels ------> ImagesSeries of Image at certain speed ------> Video

What are its features?Types 2-D, 3-D, HDRframes per secondResolutionVideo size

Need for compressionInternet and its ability to stream videoVoIP Online streaming, online video games, video conference, internet TVIntroduction of faster hand-held devicesBandwidth limitations

What to compress?Discard what cannot be seenPixel Representation- Eye perceives Intensity better than Color

Discard RedundanciesSpatial Temporal

Areas of compressionSpatial redundancies Large homogenous regions

Areas of compressionTemporal redundanciesBetween every second there are 30 frames!Implies adjacent frames are almost identical

Who exploits them? Video Codecs

High Efficiency Video Coding-HEVCLatest video coding standard by Joint Collaborative Team on Video Coding (JCT-VC) in January, 2013

Can address all applications of H.264/MPEG-4 AVC [8]

50% bitrate reduction over H.264/AVC with same quality

Supports parallel processing architectures

Encoder block[8]

Partitioning in HEVC

Coding Unit (CU)

Coding Tree Unit (CTU)

Prediction Unit and Transform Unit

Partitioning in a Video frame

Inter-Prediction in HEVC Motion EstimationMotion Compensation

Result of Motion EstimationMode information indicating the mode

Reference indices indicating the Reference Picture

Motion Vector i.e. horizontal and vertical displacement values which directs to the reference PU in the reference picture

Motion Vector in a video frame

Motion Information after Motion Estimation

Motion Information after Motion Merge

Motion Merge in HEVCObjects in images have homogenous motion

Effectively cluster of PUs could have same Motion Information !

Solution Indicate such cluster of PUs to follow one base PU thus reducing redundancy in motion information

Spatial Merge Mode in HEVCEvaluation of 4 spatial candidates among 5 candidates at different positions in the order A1 - B1 - B0 - A0 - B2

If a candidate has identical motion information - mark it as the true candidate

Encode all five candidates

Limitations and MotivationsThe candidates must have motion information and itself cannot be in a merge mode

Hence all candidate PUs must obtain motion information from their root PU which increases the time overhead

Limits the maximum size of a merging block to neighboring areas

Proposed algorithmStep 1: Evaluate if the current CU is greater than the threshold size. If yes proceed to step 2 else go to step 7

Step 2: Check whether the candidate PU is in merge mode.If yes proceed to step 3 else go to step 4

Step 3: Locate the base PU of the candidate PU and term it as the candidate PU

Proposed algorithm (cont.)Step4: Check if the Motion Information matches with PUIf Yes, Proceed to Step5 else go to step 6

Step 5: Select the PU as the base PU and terminate future evaluations.

Step 6: Select the next candidate PU and go to step 1

Step 7: Select all of its descending PUs to follow merge mode and select the top left PU as the base PU

Test conditions Source code: HEVC Reference software HM 13.0 [38]

Platform: Windows 7 64-bit OS on 16 GB RAM at 3.70 GHz on Intel Xenon E5-1620 v2 processor

Profile: random access profile

GOP length: 8

CTB size: 64x64 with minimum CU size of 8x8.

Quantization Parameters: 22, 27, 32, 37

Test Sequences

Metric -Encoding timeIndicated the time taken by the encoder in terms of seconds.

Indicates the fastness of the codec and its underlying algorithm

Depends on availability of resources

Encoding time gain (1)Encoding time gain (2)Metric-BitrateRate of the bitstream generated by the codec

Measured in kilo bits per sec (kbps)

Indicates the compression performance of the codec and the underlying algorithm

Bitrate vs. QP (1)Bitrate vs. QP (2)Metric-Peak Signal to Noise Ration (PSNR)Useful signal among the total signal

Measured in terms of decibels (dB)

Indicates the quality of the encoded data generated by the codec and the underlying algorithmPSNR vs. QP (1)PSNR vs. QP (2)Bjontegaard Delta metrics (BD-metrics)Useful for comparing two codecs or two different algorithms used in a codec

Makes an Rate-Distortion (R-D) comparison using the generated bitstream and its effectiveness

BD-PSNR (dB): +ve value indicates an improvement BD-bitrate (%): -ve value indicates an improvement

BD-PSNR vs. QP (1)BD-PSNR vs. QP (2)BD-bitrate vs. QP (1)BD-bitrate vs. QP (2)Quality and bitrate comparisonThe quality of the original and the proposed algorithm in terms of PSNR in decibels (dB)

The data size of the original and the proposed algorithm in terms of bitrate in kilo bits per sec (kbps)

Provides a bird view of gains against losses of the proposed algorithm over original algorithmPSNR vs. Bitrate (1)PSNR vs. Bitrate (2)Summary of Results Reduction in encoding time by 13% - 24 %

Reduction in bitrate by 2% - 7%

Slight drop in PSNR of 2% - 6%

Positive value of BD-PSNR ranging from 0.29 to 0.56 dB

Dip in BD-bitrate ranging from -31% to -65%Conclusions Reduction in complexity has lead to reduction of encoding time making the codec faster

Reduction in bitrate as a result of larger merge blocks making it easier to transmit the codec data

Slight drop in quality as a tradeoff

BD metrics suggests proposed algorithm as an improvement over existing algorithm as the gains are greater than losses

Future workSingle Processor used can be made much faster using Parallel Processors like GPUs

Integrating with improved algorithms of Intra/Inter prediction produces faster and better compression

Associating with Scalable HEVC (SHVC) provides wide range of applications

Can be extended to Intra frames and temporal merging

AcronymsAVC - Advanced Video CodingAMVP Advanced Motion Vector PredictionBD - Bjontegaard DeltaCABAC Context Adaptive Binary Arithmetic CodingCB Coding BlockCBF Coding Block FlagCFM CBF Fast ModeCTU Coding Tree UnitCTB Coding Tree BlockCU Coding UnitDCT Discrete Cosine TransformDST Discrete Sine Transform

AcronymsHDTV - High Definition Tele VisionHDR - High Dynamic RangeHDRI - High Dynamic Range ImagingHEVC High Efficiency Video CodingHM HEVC Test ModelHVS Human Visual SystemISO International Standards OrganizationITU International Telecommunications UnionJCT-VC - Joint Collaborative Team on Video CodingMB MacroblockMC Motion CompensationME Motion Estimation

AcronymsMPEG Moving Picture Experts GroupNAL Network Abstraction LayerPB Prediction BlockPSNR Peak Signal to Noise RatioPU Prediction UnitQP Quantization ParameterRDOQ Rate Distortion Optimization QuantizationRGB Red Green Blue RMD Rough Mode DecisionSATD Sum of Absolute Transform DifferencesSD Standard DefinitionSSIM Structural Similarity

AcronymsTB Transform BlockTU Transform UnitURQ Uniform Reconstruction QuantizationVCEG Video Coding Experts GroupVPS Video Parameter SetWQVGA Wide Quarter Video Graphics ArrayWVGA Wide Video Graphics Array

References

References

References

References

References

References

References

References

References

Thank you