A performance Comparison of Fractional-Pel Interpolation filters in HEVC and H.264/AVC

Guided by:

DR.K.R.RAOSubmitted by:

Lohith Subramanya

1000928742

EE-5359 PROJECT INTERIM PRESENTATION

OBJECTIVE

The objective of this project is to compare and analyze the fractional-pel interpolation filters in HEVC [1] and H.264/AVC [17] based on their frequency responses, complexity, coding performance and performance gain. BD-PSNR [33] and BD-Bit Rate [33] are the metrics used for comparison.

INTRODUCTIONThe fractional-pel interpolation filters (6-tap FIR [24] and Average) adopted in H.264/AVC [17] improve motion compensation greatly.

Similarly, the DCT - based fractional-pel interpolation filters (7-tap and 8-tap) are adopted in the HEVC [1] standard. This project involves the differences in performance between these two types of filters.

H.264 [7]

It is an industry standard for video compression, the process of converting digital video into a format that takes up less capacity when it is stored or transmitted.The encoder converts video into a compressed format and the decoder converts compressed video back into an uncompressed format.

H.264 Block Diagram [23]

HEVC [1]

High Efficiency Video Coding (HEVC) [1] is the current joint video coding standardization project of the ITU-T Video Coding Experts Group (VCEG) (ITU-T Q.6/SG 16) and ISO/IEC Moving Picture Experts Group (MPEG) (ISO/IEC JTC 1/SC 29/WG 11).

HEVC Block Diagram [6]

Why use Interpolation?

Motion-compensated prediction (MCP) [8] is the key to the success of the modern video coding standards, as it removes the temporal redundancy in video signals and reduces the size of bitstreams significantly. With MCP, the pixels to be coded are predicted from the temporally neighboring ones, and only the prediction errors and the motion vectors (MV) [8] are transmitted.

Interpolation(Continued..)

However, due to the finite sampling rate, the actual position of the prediction in the neighboring frames may be out of the sampling grid, where the intensity is unknown. So, the intensities of the positions in between the integer pixels, called sub-positions, must be interpolated and the resolution of MV [8] is increased accordingly.

N-Tap FIR Filter [27]

Magnitude Response of Half-Pel Interpolation Filters [10]

Solid graph: DCTIF 8-Tap FilterDashed graph: H.264/AVC FilterDotted graph: DCTIF 6-Tap Filter

Filter Co-efficients for half-pel and quarter-pel pixels

Filter Weights of Pixels in HEVC

Representation of integer and fractional-pels [20]

 The interpolation filters used in H.264 [17] are 6 tap FIR filter for half-pel interpolation and the average filter for quarter-pel interpolation. Similarly, in HEVC [3], an 8-tap DCTIF is used for half-pel interpolation and a 7-tap DCTIF is used for quarter-pel interpolation.

The comparison of the modified filter coefficients based on frequency response that are obtained from Steps 1-4 can be further assessed for the required parametric results mentioned in “A comparison of Fractional-Pel Interpolation Filters in HEVC and H.264/AVC” [10]

Results

Waterfall_cif.yuv

Frame Height: 352Frame Width: 288Frame Rate: 25fpsNo. of frames encoded: 25

HEVC H.264

Encoding Time(seconds) 30.731 26.268

Bitrate(kbits/sec) 232.5279 233.7748

Y-PSNR 35.5360 34.589

U-PSNR 36.8714 35.170

V-PSNR 37.9408 37.442

Average PSNR 36.2535 35.2182

Results

Results

Bus_qcif.yuv

Frame Height: 176Frame Width: 144Frame Rate: 25fpsNo. of frames encoded: 25

HEVC H.264

Encoding Time(seconds) 8.740 8.200

Bitrate(kbits/sec) 320.3269 344.89

Y-PSNR 33.8993 31.8140

U-PSNR 38.2684 37.5162

V-PSNR 38.8129 40.0570

Average PSNR 36.2596 34.8071

Results

Results

Coastguard.yuv

Frame Height: 352Frame Width: 288Frame Rate: 15fpsNo. of frames encoded: 15

HEVC H.264

Encoding Time 29.002s 27.419s

Bitrate(kbits/sec) 543.0624 577.6629

Y-PSNR 33.9717 33.5862

U-PSNR 43.8025 44.6524

V-PSNR 44.6284 43.2639

Average PSNR 36.2576 35.4288

Results

Results

Stefan_cif.yuv

Frame Height: 352Frame Width: 288Frame Rate: 30fpsNo. of frames encoded: 30

HEVC H.264

Encoding Time 30.746s 29.143s

Bitrate(kbits/sec) 748.1192 769.14

Y-PSNR 35.4565 34.7083

U-PSNR 38.8193 37.9758

V-PSNR 38.7977 38.0275

Average PSNR 36.2945 35.5312

Results

List of ACRONYMSAIF: Adaptive Interpolation Filter ALF: Adaptive Loop Filter APEC: Adaptive Prediction Error Coding AVC: Advanced Video CodingAQMS: Adaptive Quantization Matrix SelectionCABAC: Context Adaptive Binary Arithmetic CodingCAVAC: Context Adaptive Variable Arithmetic CodingCSVT: Circuits and Systems for Video TechnologyDCT: Discrete Cosine TransformDCTIF: Discrete Cosine Transform Interpolation FilterDMVD: Decoder-side Motion Vector Deviation DSP: Digital Signal ProcessingEMS: Extended Macro-block SizeFIR: Finite Impulse ResponseHEVC: High Efficiency Video CodingHP: High ProfileIBDI: Internal Bit Depth IncreasingITU-T: International Telecommunication Union – Telecommunication Standardization Sector

List of ACRONYMS(Continued..)JCT-VC: Joint Collaborative Team on Video CodingJPEG: Joint Photographic Experts GroupKLT: Karhunen - Loeve TransformLTS: Larger Transform SizeMCP: Motion Compensated PredictionMP: Main ProfileMPEG: Moving Picture Experts GroupMV: Motion VectorsRDO: Rate Distortion OptimizationSOC: System On ChipSVN: Sub-VersionUVLC: Universal Variable Length CodingVCEG: Video Coding Experts GroupVCIP: Visual Communications and Image Processing

References1. Fraunhofer Heinrich Hertz Institute - http://hevc.hhi.fraunhofer.de/2. Open Patents and Standards Platform - http://www.iplytics.com/en/tag/hevc/3. HEVC Review Site-

http://telcogroup.ru/files/materials-pdf/High_Efficiency_Video_Coding_H265.pdf4. Overview of HEVC -

http://iphome.hhi.de/wiegand/assets/pdfs/2012_12_IEEE-HEVC-Overview.pdf5. HEVC Blog:

http://www.extremetech.com/computing/162027-h-265-benchmarked-does-the-next-generation-video-codec-live-up-to-expectations

6. Altera Technologies: http://www.altera.com/technology/system-design/articles/2013/tv-studio-system.html

7. I.Richardson, “ Real time implementation of H.264 Video Coding”, 2008 IEEE International SOC Conference, PP: 390, Sept. 2008

8. H.265 Blog http://www.h265.net/2010/07/adaptive-interpolation-filter-for-video-coding.html

9. CNET Blog on HEVC http://news.cnet.com/8301-11386_3-57566116-76/hevc-video-standard-finished-high-end-improvements-coming/

10.H.Lv, et al, “ A comparison of fractional-pel interpolation in HEVC and H.264/AVC”, 2012 IEEE Conference on Visual Communications and Image Processing (VCIP), PP: 1-6, Nov 2012

11.G.J.Sullivan, et al, “ Overview of the HEVC Standard”, 2012 IEEE Transactions on Circuits and Systems for Video Technology(CSVT), Vol: 22, No: 12, PP: 1649-1668, Sept 2012

12.B.Lee, et al, “Performance Comparison of various interpolation methods for color filter arrays”, 2001 IEEE Symposium on Industrial Electronics, Vol: 1, PP: 232-236, June 2001

13.V.Yu and J.Ostermann, “Locally Adaptive Non-Separable Interpolation Filter for H.264/AVC”, 2006 IEEE International Conference on Image Processing, PP: 33-36, Oct 2006

14.Video Test Sequences: http://trace.eas.asu.edu/yuv/ 15.Tortoise SVN Downloadable Software Link: http://tortoisesvn.net/downloads.html

16.HM 13 Software Link: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-13.0+RExt-6.0rc1/

17. H.264 Advanced Video Coding Blog https://www.vcodex.com/h264.html18. G.J.Sullivan, et al, “ Standardized Extensions of HEVC”, 2013 IEEE Journal of Selected Topics

in Signal Processing, Vol : 7, No: 6, PP: 1001-1016, Dec. 201319. K.R.Rao, D.N.Kim and J.J.Hwang, “Video coding standards”, Springer Publications, Jan. 2014:

http://www.springer.com/physics/book/978-94-007-6741-620. SPIE Digital Library Article on HEVC: http://

electronicimaging.spiedigitallibrary.org/article.aspx?articleid=1730243 21. Karhunen-Loeve Transform: http://

en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem22. Sharp 8Kx4K TV:

http://www.sound-news.net/index.php/the-novosti/hifi-av-novosti/item/552-sharp-8kx4k-tv

23. Institute of Computer and Communication Engineering – Article on HEVC: http://research.ncku.edu.tw/re/articles/e/20071102/2.html 24. FIR Filter: http://en.wikipedia.org/wiki/Finite_impulse_response25. JCT-VC Document Management System: http://phenix.int-evry.fr/jct/26. T.Wiegand, et al, “Overview of the H.264/AVC Video Coding Standard”, 2003 IEEE Transactions on

Circuits and Systems for Video Technology, Vol: 13, No: 7, PP: 560-576, July 200327. Iowegian International DSP Site: http://www.dspguru.com/dsp/faqs/fir/basics 28. N-Tap FIR Filter:

http://www.analog.com/static/imported-files/seminars_webcasts/MixedSignal_Sect6.pdf 29. I.Richardson, “ The H.264 Advanced Video Compression Standard”, Wiley Publications, Aug. 2010:

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470516925.html 30. HM 13 Software Reference Manual: http://

mpeg.chiariglione.org/standards/mpeg-h/high-efficiency-video-coding/high-efficiency-video-coding-hevc-encoder-description

31. JPEG: http://www.jpeg.org/ 32. JM 18.6 Software Repository: http://iphome.hhi.de/suehring/tml/download/ 33. BD-Metrics:

http://www.mathworks.com/matlabcentral/fileexchange/27798-bjontegaard-metric/content/bjontegaard.m

34. Special issue on emerging research and standards in next generation video coding, IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol: 22, PP: 1646-1909, Dec. 2012

35. Special issue on emerging research and standards in next generation video coding, IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Vol: 23, PP: 2009-2142, Dec. 2013