interim report on performance comaprison of hevc,h.264 and vp9 a project under the guidance of dr....
TRANSCRIPT
INTERIM REPORT ON
PERFORMANCE COMAPRISON OF HEVC,H.264 and VP9
A PROJECT UNDER THE GUIDANCE OF
DR. K. R. RAO
COURSE: EE5359 - MULTIMEDIA PROCESSING,
SPRING 2015 By
DEEPIKA SREENIVASULU PAGALA
OBJECTIVE:• The objective of this project is to study video coding
standards HEVC [1][34][35][36], H.264 [2] [34] and VP9 [3][4] and understand various techniques in video coding such as prediction, transform, quantization and coding.
• A performance comparison of these video codecs based on various metrics such as computational time, PSNR [25], SSIM [5][20][31], BD-Bitrate [6] and BD-PSNR [7] will be carried out.
• The HM 16.0 [26][33], JM 18.6 [27][32] and VPX encoder [28] from The WebM Project test models for HEVC, H.264 and VP9 respectively will be used for this purpose.
EVOLUTION OF VIDEO CODING STANDARDS [8]:
GROWING DEMAND FOR VIDEO [30]:• Video exceeds half of internet traffic and will grow
to 86 percent by 2016 [30]. • Increase in applications, content, fidelity, etc. -Need
higher coding efficiency! [30].• Ultra-HD 4K broadcast expected for Japan in 2014.
London Olympics Opening and Closing Ceremonies shot in Ultra-HD 8K. - Need higher throughput! [30].
• 25x increase in mobile data traffic over next five years. Video is a “must have” on portable devices. - Need lower power! [30].
FUNDAMENTAL CONCEPTS IN VIDEO CODING [10]:
Color Spaces • RGB color space – Each pixel is represented by three
numbers indicating the relative proportions of red, green and blue colors.
• YCrCb color space – Y is the luminance component and Cr
,Cb - two color difference or chrominance components. • As the human visual system is less sensitive to color
than the luminance component, YCrCb has advantages over RGB space.
H.264/MPEG-4 [2][9]:
• H.264/Advanced Video Coding (AVC) is video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group [2].
• Most popular video standard- 80% of video is encoded on internet with H.264/AVC [9].
• Developed for multimedia applications [9].• Adopted advanced coding techniques such as
multiple-reference frame prediction, and context-based adaptive binary arithmetic coding (CABAC).
ENCODER IN H.264 [11]:
DECODER IN H.264 [11]:
FEATURES OF H.264:• Prediction [12]:
Fig 4: Intra prediction in H.264 [12] Fig 5: Inter prediction in H.264 [12]
FEATURES OF H.264(cotd..):
• Transform and Quantization [13]: A block of residual samples is transformed using a 4x4 or 8x8 integer
transform, an approximate form of the Discrete Cosine Transform (DCT) . The transform outputs a set of coefficients, each of which is a weighting value for a standard basis pattern. The output of the transform, a block of transform coefficients, is quantized, i.e. each coefficient is divided by an integer value .
• Entropy Coding [15]: H.264 uses CABAC(Context Adaptive Binary Arithmetic Coding ) or CAVLC(Context Adaptive Variable Length Coding) for entropy coding[14].
HEVC/H.265 [1]:• High Efficiency Video Coding (HEVC) is an international
standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Visual Coding Experts Group).
• High Efficiency Video Coding (HEVC) is the latest Video Coding format .
• It challenges the state-of-the-art H.264/AVC Video Coding standard which is in current use in the industry by being able to reduce the bit rate by 50% and retaining the same video quality [1].
Encoder in H.265 [19][20]:
Fig 6 : Block Diagram of HEVC Encoder [17]
DECODER IN H.265 [19][21]:
Fig 7 : Block Diagram of HEVC Decoder [21]
FEATURES OF HEVC:• Partitioning [19]:
Fig 8: Picture, Slice, Coding Tree Unit (CTU), Coding Unit (CU) [19]
FEATURES OF HEVC(cotd..):• Prediction [1] :- Intra prediction: Each CU is predicted from neighboring image data in the same
picture, using DC prediction, planar prediction or directional prediction.- Inter Prediction :Each PU is predicted from image data in one or two reference
pictures, using motion compensated prediction.
• Transform and Quantization [19]:- Any residual data remaining after prediction, is transformed using a block
transform based on the Discrete Cosine Transform (DCT). The transformed data is quantized. One or more block transforms of size 32x32, 16x16, 8x8 and 4x4 are applied to residual data in each CU.
• Entropy Coding:- HEVC uses Context Adaptive Binary Arithmetic Coding (CABAC) [14] for Entropy
coding.
Vp9 [3][4]:• VP9 is an open and royalty free video compression
standard being developed by Google • VP9 is a successor to VP8. • One of the goals of VP9 is to reduce the bit rate by
50% compared to VP8 while having the same video quality [22].
• VP9 has many design improvements compared to VP8.
• VP9 supports the use of superblocks of 64x64 pixels.
ENCODER INVP9 [24]:
Fig 9: Encoder block diagram for VP9 [24]
DECODER IN VP9 [24]:
Fig 10: Decoder block diagram for VP9 [24]
FEATURES OF VP9:
• Prediction block sizes [4] [23]:
Fig 11: Example partitioning of a 64x64 Super-block [4] [23]
FEATURES OF VP9(cotd..)
• Prediction [4]:- Intra Prediction : VP9 supports a set of 10 Intra prediction modes for block
sizes ranging from 4x4 up to 32x32- Inter Prediction : VP9 supports a set of 4 inter prediction modes for block
sizes ranging from 4x4 up to 64x64 pixels
• Transform and Quantization [4]:- The residuals after subtraction of predicted pixel values are subjected to
transformation and quantization . Transform blocks can be 32x32, 16x16, 8x8 or 4x4 pixels.
• Entropy coding [4]:- VP9 uses 8-bit arithmetic coding engine from VP8 known as bool-coder
COMPARISON METRICS:
• Peak Signal to Noise Ratio (PSNR) [25]• Structural Similarity Index (SSIM) [5][20][31]• BD-Bitrate and BD-PSNR [6]• Computational time as a measure of
implementation complexity
PROFILES USED FOR COMPARISON:
• The HM 16.2 [26][33] ,JM 18.6 [27] [32] and VPX encoder [28] from The WebM Project test models for HEVC , H.264 and VP9 respectively will be used for comparison in this project.
TEST SEQUENCES :
Fig.12 akiyo_qcif.yuv(176x144) [38]
TEST SEQUENCES (Cotd..):
Fig.13 waterfall_cif.yuv(352x288) [38]
TEST SEQUENCES (Cotd..):
Fig.14 BasketballDrill_832x480.yuv [39]
TEST SEQUENCES (Cotd..):
Fig 15 :Jockey_1920x1080.yuv [29]
CONFIGURATION OF HM 16.2Main all-intra profile settings :• IntraPeriod : 1 # Period of I-Frame ( -1 = only first) • GOPSize : 1 # GOP Size (number of B slice = GOPSize-1) • QP : 22 # Quantization parameter(0-51) (22, 27, 32 or 37 is used at a time)
Command line parameters for using HM 16.2 encoder: • TAppEncoder [-h] [-c config.cfg] [--parameter=value] Options: • -h Prints parameter usage • -c Defines configuration file to use. Multiple configuration files may be used with
repeated –c options. • --parameter=value Assigns value to a given parameter.
• Sample command line parameters for HM 16.2 encoder: • C:\HEVC\bin\vc10\Win32\Release>TAppEncoder.exe -c encoder_intra_main.cfg -wdt
2560 -hgt 1600 -fr 30 -f 20 -i C:\HEVC\bin\vc10\Win32\testsequences >> C:\HEVC\bin\vc10\Win32\final_results\ PeopleOnStreet_2560x1600_30_crop_qp37.txt
CONFIGURATION OF JM 18.6 High all-intra profile settings :• FramesToBeEncoded : 20 # Number of frames to be coded • FrameRate : 50.0 # Frame Rate per second (0.1-100.0) • ProfileIDC : 100 # Profile IDC (66=baseline, 77=main, 88=extended;
FREXT Profiles: 100=High, 110=High 10, 122=High 4:2:2, 244=High 4:4:4, 44=CAVLC 4:4:4 Intra, 118=Multiview High Profile, 128=Stereo High Profile)
• IntraProfile : 1 # Activate Intra Profile for FRExt (0: false, 1: true) • LevelIDC : 40 # Level IDC (e.g. 20 = level 2.0) • IntraPeriod : 1 # Period of I-pictures (0=only first) • IDRPeriod : 1 # Period of IDR pictures (0=only first) • QPISlice : 22 # Quant. param for I Slices (0-51) (22, 27, 32 or 37 is used
at a time)
CONFIGURATION OF JM 18.6 (COTD..)
Command line parameters for using JM 18.6 encoder:• Options: • -h Prints parameter usage. • -d Use <defenc.cfg> as default file for parameter initializations. If not
used then file defaults to “encoder.cfg” in local directory. • -f Read <curencM.cfg> for resetting selected encoder parameters.
Multiple files could be used that set different parameters.• -p Set parameter <EncParamM> to <EncValueM>. The entry for
<EncParamM> is case insensitive.Sample command line parameters for JM 18.6 encoder: • C:\jm 18.6\JM\bin>lencod.exe –f encoder.cfg -p InputFile=" C:\jm 18.6\
JM\bin\ Jockey_1920x1080.yuv" –p SourceWidth=832 -p SourceHeight=480 >> C:\jm 18.6\JM\bin\final_results\ jockey_37.txt
RESULTS :
RESULTS(COTD..)
RESULTS(COTD..)
RESULTS(COTD..)
RESULTS(COTD..)
LIST OF ACRONYMS AND ABBREVIATIONS:
• ADST: Asymmetric Discrete Sine Transform.• AHG: Ad Hoc Group.• AVC: Advanced Video Coding. • BD-BR: Bjontegaard Delta Bitrate. • BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. • CABAC: Context Adaptive Binary Arithmetic Coding. • CAVLC: Context Adaptive Variable Length Coding.• CTB: Coding Tree Block. • CTU: Coding Tree Unit. • CU: Coding Unit. • DBF: De-blocking Filter. • DCT: Discrete Cosine Transform. • DST :Discrete Sine Transform. • DPB :Decoded Picture Buffer.• DVD: Digital Video Disk.• HD: High Definition.• HDR :High Dynamic Range.• HEVC: High Efficiency Video Coding. • HM: HEVC Test Model. • ICME: International Conference on Multimedia and Expo.• IEC: International Electro-technical Commission.• ISCAS : International Symposium on Circuits and Systems.• ISO: International Organization for Standardization. • ITU-T: International Telecommunication Union- Telecommunication
Standardization Sector. • JCT: Joint Collaborative Team.
• JCT-VC: Joint Collaborative Team on Video Coding. • JM: H.264 Test Model. • JCT-VC: Joint Collaborative Team on Video Coding. • JPEG: Joint Photographic Experts Group. • KTA: Key Technical Areas (H.264 based exploration software of VCEG)• MC: Motion Compensation. • ME: Motion Estimation. • MPEG: Moving Picture Experts Group. • MSE: Mean Square Error. • MVC : Multiview Video Coding..• NGOV: Next Generation open Video• PB: Prediction Block. • PCS : Picture Coding Symposium• PSNR: Peak Signal to Noise Ratio. • PU: Prediction Unit• QP: Quantization Parameter• RD: Rate Distortion • SAO: Sample Adaptive Offset. • SCC: Screen Content Coding. • SSIM: Structural Similarity Index. • TB: Transform Block. • TU: Transform Unit. • VCEG: Visual Coding Experts Group.• WCG: Wide Color Gamut.
REFERENCES:• [1] G. J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE
Transactions on circuits and systems for video technology, vol. 22, no.12, pp. 1649 – 1668, Dec 2012.
• [2] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050 available on http://ip.hhi.de/imagecom_G1/assets/pdfs/JVT-G050.pdf
• [3] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/MPEG-AVC Encoders”, IEEE PCS 2013, pp. 394-397, San José, CA, USA, Dec 8-11, 2013
• [4] D. Mukherjee et al, “The latest open-source video codec VP9–An overview and preliminary results”, Google Inc., United States
• [5] Z. Wang et al, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004
• [6] G. Bjøntegaard, “Calculation of average PSNR differences between RD-curves”, ITU-T Q.6/SG16 VCEG 13th Meeting, Document VCEG-M33, Austin, USA, Apr. 2001
REFERENCES(Cotd..):• [7] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International
Conference on Multimedia and Expo (ICME), pp. 685-690, July. 2010
• [8] N. Ling, “High efficiency video coding and its 3D extension: A research perspective,” Keynote Speech, ICIEA, pp 2150-2155, Singapore, July 2012 -
• http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6361087
• [9] V. sze , M. Budagavi , " Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial) " , IEEE ISCAS Tutorial 2014 , Melbourne , Australia , June.2014 - filehttp://www.rle.mit.edu/eems/wp-content/uploads/2014/06/H.265-HEVC-Tutorial-2014-ISCAS.pdf
• [10] I. E. G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002
• [11] A. Puri , X.Chen and A.Luthara , “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004.
• [12] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html
• [13] N. Ahmed , T. Natarajan and K. R. Rao, “Discrete Cosine Transform”, IEEE Transactions on Computers, Vol. C-23, pp. 90-93, Jan. 1974.
REFERENCES(Cotd..):• [14] D. Marpe, H. Schwarz, and T. Wiegand, “Context-based adaptive binary arithmetic coding
in the H.264/AVC video compression standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 620–636, Jul. 2003.
• [15] J . Ostermann, et al, " Video coding with H.264/AVC tools, performance , and complexity", IEEE Circuits and Systems Magazine , Vol.4 , pp.7-28, Aug.2004.
• [16] J. Ohm , et al, "Comparison of the Coding Efficiency of Video Coding Standards - including High Efficiency Video Coding (HEVC) ", IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, Issue: 12 , pp. 1669 -1684 , Dec.2012.
• [17] G. J. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013.
• [18] HEVC white paper - http://www.ateme.com/an-introduction-to-uhdtv-and-hevc
• [19] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html
• [20] W. Malpica and A. Bovik , "Range image quality assessment by structural similarity", IEEE ICASSP 2009, 19-24 Apr. 2009.
REFERENCES(Cotd..):• [21] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC
J0292r1, July 2012.
• [22] "VP-Next Overview and Progress Update" (PDF). WebM Project (Google). Retrieved 2012-12-29. Available on: http://downloads.webmproject.org/ngov2012/pdf/04-ngov-project-update.pdf
• [23] M. P. Sharabayko et al, "Intra Compression Efficiency in VP9 and HEVC" Applied Mathematical Sciences, Vol. 7, no. 137, pp.6803 – 6824, Hikari Ltd, 2013
• [24] J. Padia, “Complexity reduction for VP6 to H.264 transcoder using motion vector reuse,” M.S. Thesis, EE Dept., UTA, Arlington, TX, 2010. Available on:
• http://www.uta.edu/faculty/krrao/dip/Courses/EE5359/index_tem.html
• [25] White paper on PSNR-NI - http://www.ni.com/white-paper/13306/en/
• [26] Access to HM Reference Software: http://hevc.hhi.fraunhofer.de
• [27] Access to JM 18.6 Reference Software: http://iphome.hhi.de/suehring/tml/
REFERENCES(Cotd..):• [28] Chromium® open-source browser project, VP9 source code, Online: • http://git.chromium.org/gitweb/?p=webm/
libvpx.git;a=tree;f=vp9;hb=aaf61dfbcab414bfacc3171501be17d191ff8506
• [29] http://ultravideo.cs.tut.fi/#testsequences - Video test sequences (4kx2k)
• [30] Cisco Visual Networking Index - http://www.cisco.com/c/en/us/solutions/service-provider/visual-networking-index-vni/index.html
• [31] J. Wang et al, "Fractal image coding using SSIM", IEEE 18th International Conference on Image Processing, pp.241-244, Brussels, Belgium , 11-14 sept. 2011.
• [32] H.264/AVC Software Reference Manual: • http://iphome.hhi.de/suehring/tml/JM%20Reference%20Software%20Manual%20(JVT-AE010).pdf
• [33] HEVC Software Reference Manual :• https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-9.2-dev/doc/software-
manual.pdf
REFERENCES(Cotd..):• [34] K. R. Rao, D. N. Kim and J. J. Hwang,• “VideoCodingStandards:AVSChina,H.264/MPEG-4 Part10, HEVC, VP6, DIRAC and VC-1 ”, Springer, 2014.
• [35] V. Sze , M. Budagavi ,and G. J.Sullivan "High Efficiency Video Coding (HEVC) : Algorithms and Architectures", Springer, 2014.
• [36] M. Wien, "High Efficiency Video Coding : Coding Tools and Specification" , Springer , 2014.
• [37] I. E. Richardson , "Coding Video : A practical guide to HEVC and beyond " , Wiley , 11 May 2015.
• [38] https://media.xiph.org/video/derf/ - test sequences
• [39] ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ - test sequences
• [40] G. Correa , et al , " Fast HEVC Encoding Decisions Using Data Mining " , IEEE Transactions on Circuits and Systems for Video Technology , Vol . 25 , No. 4 , pp. 660 - 673, April 2015.
• [41] D. K. Kwon and M. Budagavi , " Combined scalable and multiview extension of High Efficiency Video Coding (HEVC) " , IEEE Picture Coding Symposium , pp. 414 - 417 , Dec . 2013 .
• [42] Encoding Time Evaluation Intel VTune Amplilfier XE Software profiler Available http://software.intel.com
