Interim Presentation onTopic Scalable video coding extension of HEVC (S-

HEVC)

A PROJECT UNDER THE GUIDANCE OF DR K R RAO COURSE EE5359 - MULTIMEDIA PROCESSING SPRING 2015

Submitted By

Aanal Desai

UT ARLINGTON ID 1001103728

EMAIL ID aanaldesaimavsutaedu

DEPARTMENT OF ELECTRICAL ENGINEERING UNIVERSITY OF TEXAS ARLINGTON

List of Acronymsbull AVC ndash Advanced Video Coding bull AMVP ndash Advanced motion vector predictionbull BL ndash Base Layer bull BO ndash Band Offsetbull CABAC ndash Context Adaptive Binary Arithmetic Codingbull CTB ndash Coding Tree Blockbull CTU ndash Coding Tree Unit bull CU ndash Coding Unitbull CIF ndash Common Intermediate Formatbull DASH ndash Dynamic Adaptive Streaming over HTTP bull DC ndash Direct Currentbull DCT ndash Discrete Cosine Transformbull Diff ndash Differencebull DPB ndash Decoded Picture Bufferbull DST ndash Discrete Sine Transformbull EL ndash Enhancement Layer bull ED ndash Entropy Decoderbull EO ndash Edge Offsetbull FPS ndash Frames per secondbull Filt ndash Filter bull FIR ndash Finite Impulse Response

bull GOP ndash Group of pictures bull HD ndash High Definition bull HDTV ndash High Definition Television bull HEVC ndash High Efficiency Video Coding bull HLS ndash High Level Syntax bull HTTP ndash Hyper Text Transfer Protocol bull ILR ndash Inter Layer Reference bull IEC ndash International Electro-technical Commission bull IP ndash Intra Predictionbull IQ ndash Inverse Quantizationbull IT ndash Inverse Transformbull ITU-T ndash International Telecommunication Union-Telecommunications standardization sector bull ISO ndash International Standardization Organization bull JCTVC ndash Joint Collaborative Team on Video Codingbull JPEG- Joint Picture Experts Groupbull LCU ndash Largest Coding Unit bull LM ndash Linear Mode bull LP ndash Loop Filtering bull MANE ndash Media Aware Network Elementsbull Mbps ndash Megabits per second bull MC ndash Motion compensation

bull MPD ndash Media Presentation Descriptionbull MPEG ndash Moving Picture Experts Group bull MV ndash Motion Vectorbull PB ndash Prediction Blockbull PDA ndash Personal Digital Assistant bull PSNR ndash Peak Signal to Noise Ratio bull PU ndash Prediction Unit bull QCIF ndash Quarter Common Intermediate Formatbull QP ndash Quantization Parameter bull ROI ndash Region Of Interest bull SAO ndash Sample Adaptive Offsetbull SHVC ndash Scalable High Efficiency Video Codingbull SNR ndash Signal to Noise Ratiobull SVC ndash Scalable Video Coding bull SPIE ndash Society of Photo-Optical Instrumentation Engineers bull TU ndash Transform Unit bull TB ndash Transform Block bull VCEG ndash Video Coding Experts Group bull VCL ndash Video Coding Layerbull VGA ndash Video Graphics Array bull UHD ndash Ultra High Definition bull URL ndash Uniform Resource Locatorbull 4CIF ndash 4x CIF

Overviewbull An increasing demand for video streaming to mobile devices such as

smartphones tablet computers or notebooks and their broad variety of screen sizes and computing capabilities stimulate the need for a scalable extension

bull Modern video transmission systems using the Internet and mobile networks are typically characterized by a wide range of connection qualities which are a result of the used adaptive resource sharing mechanisms In such diverse environments with varying connection qualities and different receiving devices a flexible adaptation of once-encoded content is necessary[2]

bull The objective of a scalable extension for a video coding standard is to allow the creation of a video bitstream that contains one or more sub-bitstreams that can be decoded by themselves with a complexity and reconstruction quality comparable to that achieved using single-layer coding with the same quantity of data as that in the sub-bitstream[2]

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

List of Acronymsbull AVC ndash Advanced Video Coding bull AMVP ndash Advanced motion vector predictionbull BL ndash Base Layer bull BO ndash Band Offsetbull CABAC ndash Context Adaptive Binary Arithmetic Codingbull CTB ndash Coding Tree Blockbull CTU ndash Coding Tree Unit bull CU ndash Coding Unitbull CIF ndash Common Intermediate Formatbull DASH ndash Dynamic Adaptive Streaming over HTTP bull DC ndash Direct Currentbull DCT ndash Discrete Cosine Transformbull Diff ndash Differencebull DPB ndash Decoded Picture Bufferbull DST ndash Discrete Sine Transformbull EL ndash Enhancement Layer bull ED ndash Entropy Decoderbull EO ndash Edge Offsetbull FPS ndash Frames per secondbull Filt ndash Filter bull FIR ndash Finite Impulse Response

bull GOP ndash Group of pictures bull HD ndash High Definition bull HDTV ndash High Definition Television bull HEVC ndash High Efficiency Video Coding bull HLS ndash High Level Syntax bull HTTP ndash Hyper Text Transfer Protocol bull ILR ndash Inter Layer Reference bull IEC ndash International Electro-technical Commission bull IP ndash Intra Predictionbull IQ ndash Inverse Quantizationbull IT ndash Inverse Transformbull ITU-T ndash International Telecommunication Union-Telecommunications standardization sector bull ISO ndash International Standardization Organization bull JCTVC ndash Joint Collaborative Team on Video Codingbull JPEG- Joint Picture Experts Groupbull LCU ndash Largest Coding Unit bull LM ndash Linear Mode bull LP ndash Loop Filtering bull MANE ndash Media Aware Network Elementsbull Mbps ndash Megabits per second bull MC ndash Motion compensation

bull MPD ndash Media Presentation Descriptionbull MPEG ndash Moving Picture Experts Group bull MV ndash Motion Vectorbull PB ndash Prediction Blockbull PDA ndash Personal Digital Assistant bull PSNR ndash Peak Signal to Noise Ratio bull PU ndash Prediction Unit bull QCIF ndash Quarter Common Intermediate Formatbull QP ndash Quantization Parameter bull ROI ndash Region Of Interest bull SAO ndash Sample Adaptive Offsetbull SHVC ndash Scalable High Efficiency Video Codingbull SNR ndash Signal to Noise Ratiobull SVC ndash Scalable Video Coding bull SPIE ndash Society of Photo-Optical Instrumentation Engineers bull TU ndash Transform Unit bull TB ndash Transform Block bull VCEG ndash Video Coding Experts Group bull VCL ndash Video Coding Layerbull VGA ndash Video Graphics Array bull UHD ndash Ultra High Definition bull URL ndash Uniform Resource Locatorbull 4CIF ndash 4x CIF

Overviewbull An increasing demand for video streaming to mobile devices such as

smartphones tablet computers or notebooks and their broad variety of screen sizes and computing capabilities stimulate the need for a scalable extension

bull Modern video transmission systems using the Internet and mobile networks are typically characterized by a wide range of connection qualities which are a result of the used adaptive resource sharing mechanisms In such diverse environments with varying connection qualities and different receiving devices a flexible adaptation of once-encoded content is necessary[2]

bull The objective of a scalable extension for a video coding standard is to allow the creation of a video bitstream that contains one or more sub-bitstreams that can be decoded by themselves with a complexity and reconstruction quality comparable to that achieved using single-layer coding with the same quantity of data as that in the sub-bitstream[2]

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull GOP ndash Group of pictures bull HD ndash High Definition bull HDTV ndash High Definition Television bull HEVC ndash High Efficiency Video Coding bull HLS ndash High Level Syntax bull HTTP ndash Hyper Text Transfer Protocol bull ILR ndash Inter Layer Reference bull IEC ndash International Electro-technical Commission bull IP ndash Intra Predictionbull IQ ndash Inverse Quantizationbull IT ndash Inverse Transformbull ITU-T ndash International Telecommunication Union-Telecommunications standardization sector bull ISO ndash International Standardization Organization bull JCTVC ndash Joint Collaborative Team on Video Codingbull JPEG- Joint Picture Experts Groupbull LCU ndash Largest Coding Unit bull LM ndash Linear Mode bull LP ndash Loop Filtering bull MANE ndash Media Aware Network Elementsbull Mbps ndash Megabits per second bull MC ndash Motion compensation

bull MPD ndash Media Presentation Descriptionbull MPEG ndash Moving Picture Experts Group bull MV ndash Motion Vectorbull PB ndash Prediction Blockbull PDA ndash Personal Digital Assistant bull PSNR ndash Peak Signal to Noise Ratio bull PU ndash Prediction Unit bull QCIF ndash Quarter Common Intermediate Formatbull QP ndash Quantization Parameter bull ROI ndash Region Of Interest bull SAO ndash Sample Adaptive Offsetbull SHVC ndash Scalable High Efficiency Video Codingbull SNR ndash Signal to Noise Ratiobull SVC ndash Scalable Video Coding bull SPIE ndash Society of Photo-Optical Instrumentation Engineers bull TU ndash Transform Unit bull TB ndash Transform Block bull VCEG ndash Video Coding Experts Group bull VCL ndash Video Coding Layerbull VGA ndash Video Graphics Array bull UHD ndash Ultra High Definition bull URL ndash Uniform Resource Locatorbull 4CIF ndash 4x CIF

Overviewbull An increasing demand for video streaming to mobile devices such as

smartphones tablet computers or notebooks and their broad variety of screen sizes and computing capabilities stimulate the need for a scalable extension

bull Modern video transmission systems using the Internet and mobile networks are typically characterized by a wide range of connection qualities which are a result of the used adaptive resource sharing mechanisms In such diverse environments with varying connection qualities and different receiving devices a flexible adaptation of once-encoded content is necessary[2]

bull The objective of a scalable extension for a video coding standard is to allow the creation of a video bitstream that contains one or more sub-bitstreams that can be decoded by themselves with a complexity and reconstruction quality comparable to that achieved using single-layer coding with the same quantity of data as that in the sub-bitstream[2]

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull MPD ndash Media Presentation Descriptionbull MPEG ndash Moving Picture Experts Group bull MV ndash Motion Vectorbull PB ndash Prediction Blockbull PDA ndash Personal Digital Assistant bull PSNR ndash Peak Signal to Noise Ratio bull PU ndash Prediction Unit bull QCIF ndash Quarter Common Intermediate Formatbull QP ndash Quantization Parameter bull ROI ndash Region Of Interest bull SAO ndash Sample Adaptive Offsetbull SHVC ndash Scalable High Efficiency Video Codingbull SNR ndash Signal to Noise Ratiobull SVC ndash Scalable Video Coding bull SPIE ndash Society of Photo-Optical Instrumentation Engineers bull TU ndash Transform Unit bull TB ndash Transform Block bull VCEG ndash Video Coding Experts Group bull VCL ndash Video Coding Layerbull VGA ndash Video Graphics Array bull UHD ndash Ultra High Definition bull URL ndash Uniform Resource Locatorbull 4CIF ndash 4x CIF

Overviewbull An increasing demand for video streaming to mobile devices such as

smartphones tablet computers or notebooks and their broad variety of screen sizes and computing capabilities stimulate the need for a scalable extension

bull Modern video transmission systems using the Internet and mobile networks are typically characterized by a wide range of connection qualities which are a result of the used adaptive resource sharing mechanisms In such diverse environments with varying connection qualities and different receiving devices a flexible adaptation of once-encoded content is necessary[2]

bull The objective of a scalable extension for a video coding standard is to allow the creation of a video bitstream that contains one or more sub-bitstreams that can be decoded by themselves with a complexity and reconstruction quality comparable to that achieved using single-layer coding with the same quantity of data as that in the sub-bitstream[2]

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Overviewbull An increasing demand for video streaming to mobile devices such as

smartphones tablet computers or notebooks and their broad variety of screen sizes and computing capabilities stimulate the need for a scalable extension

bull Modern video transmission systems using the Internet and mobile networks are typically characterized by a wide range of connection qualities which are a result of the used adaptive resource sharing mechanisms In such diverse environments with varying connection qualities and different receiving devices a flexible adaptation of once-encoded content is necessary[2]

bull The objective of a scalable extension for a video coding standard is to allow the creation of a video bitstream that contains one or more sub-bitstreams that can be decoded by themselves with a complexity and reconstruction quality comparable to that achieved using single-layer coding with the same quantity of data as that in the sub-bitstream[2]

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Introduction bull SHVC provides a 50 bandwidth reduction for the same video quality

when compared to the current H264AVC standard SHVC further offers a scalable format that can be readily adapted to meet network conditions or terminal capabilities Both bandwidth saving and scalability are highly desirable characteristics of adaptive video streaming applications in bandwidth-constrained wireless networks[3]

bull The scalable extension to the current H264AVC [4] video coding standard (H264SVC) [8] provided resources of readily adapting encoded video stream to meet receiving terminals resource constraints or prevailing network conditions

bull The JCT-VC is now developing the scalable extension (SHVC) [5] to HEVC in order to bring similar benefits in terms of terminal constraint and network resource matching as H264SVC does but with a significantly reduced bandwidth requirement[3]

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Types of Scalabilities

bull Temporal Spatial and SNR Scalabilitiesbull Spatial scalability and temporal scalability defines cases in which a sub-

bitstream represents the source content with a reduced picture size (or spatial resolution) and frame rate (or temporal resolution) respectively[1]

bull Quality scalability which is also referred to as signal-to-noise ratio (SNR) scalability or fidelity scalability the sub-bitstream delivers the same spatial and temporal resolution as the complete bitstream but with a lower reproduction quality and thus a lower bit rate[2]

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Block diagram of spatial scalability

Figure 1[24]

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Block Diagram of SNR Scalability

Figure 2 [24]

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull Block diagrams of spatial and SNR scalable coding are Depicted in Fig1 and 2 respectively Note that the down-sampling is a non-normative part ie not specified in the standard Normative inter-layer processing is present in spatial scalability case (up-sampling) The key idea proposed in this paper is to replace the trivial copying (dotted in Fig 1b) by de-noising inter-layer filter which improves the quality of inter-layer texture prediction so that improves the coding efficiency of the enhancement layer[24]

bull While analyzing the spectral characteristics of both down-sampling and up-sampling filters used for spatial scalability we found that removing high-frequency noise from the reference signal before prediction is effective[24]

bull Reference signal in SNR scalability case ie the reconstructed base layer picture usually contains more coding noise compared with spatial scalability case since it is coded with higher QP Therefore it is reasonable to design inter-layer filter for SNR scalability with de-noising properties [24]

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

High-Level Block Diagram of the Proposed Encoder

Figure 3 [1]

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Inter-layer Intra predictionbull A block of the enhancement layer is predicted using the reconstructed

(and up-sampled) base layer signal[2]bull Inter-layer motion prediction- The motion data of a block are completely

inferred using the (scaled) motion data of the co-located base layer blocks or the (scaled) motion data of the base layer are used as an additional predictor for coding the enhancement layer motion [2]

bull Inter-layer residual prediction- The reconstructed (and up-sampled) residual signal of the co-located base layer area is used for predicting the residual signal of an inter-picture coded block in the enhancement layer while the motion compensation is applied using enhancement layer reference pictures[2]

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Up-sampling filterbull The base-layer pixel samples needs to be up-sampled to support inter-

layer texture prediction in the spatial scalability case Presently SHVC supports spatial scalability ratios of 21 and 32

bull In order to support these two configurations of spatial scalability a set of interpolation filters were introduced in addition to the HEVC motion compensation interpolation filters [24]

bull Up-sampling filter is the key part of inter-layer texture prediction in the case of spatial scalability As shown in SHVC tool experiments inter-layer texture prediction delivers the most part of SHVC gain (~18 in terms of Luma BD-rate reduction) [22]

bull The phases in Table 1 and 2 represent theoretically accurate phase shifts used in filter coefficients design In actual implementation division free phase derivation is used [23] Filters for zero-phase shift in Tables 1 and 2 are trivial Outputs of these filters are identical to their inputs [24]

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Table 1 Luma Up-Sampling Filters [24]

Table 2 Chroma Up-Samplimg Filters [24]

Up-sampling filter

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Inter-layer texture prediction

bull H264AVC-SVC [14] presented inter-layer prediction for spatial and SNR scalabilities by using intra-BL and residual prediction under the restriction of a single-loop decoding structure[20]

bull To enable the selection of this up-sampled information for prediction in the enhancement layer the scalability extension employs a so-called ldquoreference indexrdquo approach [20] Conceptually this approach requires an enhancement layer decoder to insert the up-sampled reference layer picture into the enhancement layer RPL [18]

bull The up-sampled picture can then be signaled for reference in the same manner as usually in inter-frame prediction That is the enhancement layer bitstream signals an inter-mode CU with the reference index corresponding to the up-sampled picture inserted into the enhancement layer RPL (with a zero motion vector used for this specific reference picture) [18]

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Figure 4[31]

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Figure 5 [31]

bull Hong et al [15] proposed a scalable video coding scheme for HEVC where the residual prediction process is extended to both intra and inter prediction modes within a multi-loop decoding framework In addition to the intra-BL and residual prediction a combined prediction mode which uses the average of the EL prediction and the intra-BL prediction as the final prediction and multi- hypothesis inter prediction which produces additional predictions for EL block using BL block motion information are also presented

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Intra-BL prediction bull To utilize reconstructed base layer information two Coding Unit (CU) level

modes namely intra-BL and intra-BL skip are introduced[1] bull The first scalable coding tool in which the enhancement layer prediction

signal is formed by copying or up-sampling the reconstructed samples of the co-located area in the base layer is called Intra-BL prediction mode [2]

bull For an enhancement layer CU the prediction signal is formed by copying or for spatial scalable coding up-sampling the co-located base layer reconstructed samples Since the final reconstructed samples from the base layer are used multi-loop decoding architecture is essential [2]

bull When a CU in the EL picture is coded by using the intra-BL mode the pixels in the collocated block of the up-sampled BL are used as the prediction for the current CU [1]

bull The scalable extension of H264MPEG-4 AVC uses 4-tap FIR filters for upsampling of the luma signal [8] 8-tap filters are applied in the proposed HEVC extension For chroma bi-linear filters are used [21]

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull For supporting arbitrary resolution ratios for each enhancement layer sample position the used filter is selected based on the required phase shift [21]

bull The upsampling filters used for the IntraBL mode are designed to provide a good coding efficiency over a wide variety of base and enhancement layer signals However even within each picture video signals may show a high degree of non-stationarity[2]

bull The operation is similar to the inter-layer intra prediction in the scalable extension of H264| MPEG-4 AVC except that it is likely to use the samples of both intra and inter predicted blocks from the base layer[2]

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull Additionally quantization errors and noise may show varying characteristics in different parts of a picture Hence to adapt the upsampling filter to local signal characteristics another inter-layer intra coding mode referred to as InterBLFilt mode is introduced This mode is used in the same way as the InterBL mode

Figure 6 Intra BL mode [2]

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Intra residual predictionbull In the intra residual prediction mode the difference between the intra

prediction reference samples in the EL and collocated pixels in the up-sampled BL is generally used to produce a prediction denoted as difference prediction based on the intra prediction mode The generated difference prediction is further added to the collocated block in the up-sampled BL to form the final prediction[1]

Figure 7 Intra Residual Prediction [1]

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Weighted Intra predictionbull In this mode the (upsampled) base layer reconstructed signal constitutes

one component for prediction Another component is acquired by regular spatial intra prediction as in HEVC by using the samples from the causal neighborhood of the current enhancement layer block The base layer component is low pass filtered and the enhancement layer component is high pass filtered and the results are added to form the prediction[2]

bull The weights for the base layer signal are set such that the low frequency components are taken and the high frequency components are suppressed and the weights for the enhancement layer signal are set vice versa The weighted base and enhancement layer coefficients are added and an inverse DCT is computed to obtain the final prediction[2]

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull In our implementation both low pass and high pass filtering happen in the DCT domain as illustrated in Figure 8 First the DCTs of the base and enhancement layer prediction signals are computed and the resulting coefficients are weighted according to spatial frequencies[2]

Figure 8 Weighted intra prediction mode The (up-sampled) base layer reconstructed samples are combined with the spatially predicted enhancement

layer samples to predict an enhancement layer CU to be coded [2]

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Difference prediction modesbull The principle in difference prediction modes is to lessen the systematic

error when using the (up-sampled) base layer reconstructed signal for prediction It is accomplished by reusing the previously corrected prediction errors available to both encoder and decoder [17]

bull To this end a new signal denoted as the difference signal is derived using the difference amongst already reconstructed enhancement layer samples and (up-sampled) base layer samples [17]

bull The final prediction is made by adding a component from the (upsampled) base layer reconstructed signal and a component from the difference signal [17]This mode can be used for inter as well as intra prediction cases[2]

bull In inter difference prediction shown in Fig 9 the (upsampled) base layer reconstructed signal is added to a motion-compensated enhancement layer difference signal equivalent to a reference picture to obtain the final prediction for the current enhancement layer block[2]

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull For the enhancement layer motion compensation the same inter prediction technique as in single-layer HEVC is used but with a bilinear interpolation filter[2]

Figure 9 Inter difference prediction mode The (upsampled) base layer reconstructed signal is combined with the motion compensated difference signal from a reference picture to predict the enhancement layer CU to be

coded [2]

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Intra Predictionbull In the intra difference prediction the (up-sampled) base layer

reconstructed signal constitutes one component for the prediction The intra prediction modes that are used for spatial intra prediction of the difference signal are coded using the regular HEVC syntax [2]

Fig 10 Intra difference prediction mode The (upsampled) base layer reconstructed signal is combined with the intra predicted difference signal to

predict the enhancement layer block to be coded [2]

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Motion vector predictionbull Our scalable video extension of HEVC employs several methods to

improve the coding of enhancement layer motion information by exploiting the availability of base layer motion information[2]

bull In HEVC two modes can be used for MV coding namely ldquomergerdquo and ldquoadvanced motion vector prediction (AMVP)rdquo In the both modes some of the most probable candidates are derived based on motion data from spatially adjacent blocks and the collocated block in the temporal reference picture The ldquomergerdquo mode allows the inheritance of MVs from the neighboring blocks without coding the motion vector difference [16]

bull In HEVC TMVP is used to predict motion information for a current PU from a co-located PU in the reference picture The process is defined to require the prediction modes reference indices luma motion vectors and reference picture order counts (POCs) of the co-located PU [19]

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

bull The goal of the motion field mapping process is then to project this motion information from the reference layer to the enhancement layerrsquos resolution while also accounting for the 16times16 TMVP storage units in the reference layer[18]

bull In the offered scheme collocated base layer MVs are used in both the merge mode and the AMVP mode for enhancement layer coding The base layer MV is inserted as the first candidate in the merge candidate list and added after the temporal candidate in the AMVP candidate list The MV at the center position of the collocated block in the base layer picture is used in both merge and AVMP modes[1]

bull In HEVC the motion vectors are compressed after being coded and the compressed motion vectors are utilized in the TMVP derivation for pictures that are coded later [1]

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Inferred prediction modebull For a CU in EL coded in the inferred base layer mode its motion

information (including the inter prediction direction reference index and motion vectors) is not signaled Instead for each 4times4 block in the CU its motion information is derived from its collocated base layer block Once the motion information of a collocated base layer block is unavailable (eg the collocated base layer block is intra predicted) the 4x4 block is predicted in the same method as in the intra-BL mode[1]

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Test Sequences

No Sequence name Resolution

Type No of Frames

1 City 176144 CQIF 30

352288 CIF 30

2 Harbour 352288 CIF 30

704576 4CIF 30

Fig11 [34]

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Simulation Resultsbull For evaluating the efficiency of the proposed scalable HEVC extension we

compared the coding efficiency of the scalable approach with two layers to that of simulcast and single layer coding All layers have been coded using pictures with a GOP size of 8 pictures For both scalable coding and simulcast the same base layers are usedHere QPs of 22 27 32 and 37 for the base layer and QPs of 20 25 30 and 35 for the enhancement layer as recommended by JCT-VC [27] The simulation results for various sequences for a fixed base layer QP of 26 The scalable extension has been implemented in the HEVC reference software HM-160 which has also been used for producing the anchor bit streams

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

BD-PSNRbull Bjoslashntegaard Delta PSNR (BD-PSNR) was proposed to objectively

evaluate the coding efficiency of the video codecs [26] [28][29] BD-PSNR provides a good evaluation of the rate-distortion (R-D) performance based on the R-D curve fitting BD-PSNR is a curve fitting metric based on rate and distortion of the video sequence However this does not take the encoder complexity into account BD metrics tell more about the quality of the video sequence Ideally BD-PSNR should increase and BD-bitrate should decrease

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Fig12 BD-PSNR vs quantization parameter for City with BL-CIF and EL-CIF

Fig 13 BD-PSNR vs quantization parameter for Harbour with BL-CIF and EL-4CIF

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

BD-bitratebull BD-bitrate also determines the quality of the encoded video sequence

similar to BD-PSNR Ideally BD-bitrate should decrease for a good quality video [28][29] Figures 14 and 15 illustrate the BD-bitrate for the bitstreams of proposed algorithm compared with the bitstreams encoded using the unaltered reference software From the figures it can be seen that the BD-bitrate has decreased by 17 t0 29 which implies that the quality of the encoded bitstream using the proposed algorithm has not degraded compared to the bitstream encoded with the unaltered reference software

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Fig14 BD-bitrate vs quantization parameter for City with BL-QCIF and EL-CIF

Fig15 BD-bitrate vs quantization parameter for Harbour for BL-CIF and EL-4CIF

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Bitrate vs PSNRplots

Fig16 PSNR vs bitrate for City with BL-QCIF and EL-CIF

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Fig17 PSNR vs bitrate for Harbour with BL-CIF and EL-4CIF

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Bitstream size

Fig18 bitstream size vs quantization parameter for City with BL-QCIF and EL-CIF

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

Fig19 Encoded bitstream vs quantization parameter for Harbour with BL-CIF and EL-4CIF

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

References[1] IEEE paper by Jianle Chen Krishna Rapaka Xiang Li Vadim Seregin Liwei Guo Marta Karczewicz Geert Van der Auwera Joel Sole Xianglin Wang Chengjie Tu Ying Chen Rajan Joshi ldquo Scalable Video coding extension for HEVCrdquo Qualcomm Technology Inc Data compression conference (DCC)2013 DOC 20-22 March 2013 [2] IEEE paper by Philipp Helle Haricharan Lakshman Mischa Siekmann Jan Stegemann Tobias Hinz Heiko Schwarz Detlev Marpe and Thomas Wiegand Fraunhofer Institute for Telecommunications ndash Heinrich Hertz Institute Berlin Germany ldquoScalableVideo coding extension of HEVCrdquo Data compression conference (DCC)2013 DOC 20-22 March 2013 T Hinz et al ldquoAn HEVC Extension for Spatial and Quality Scalable Video Codingrdquo Proceedings of SPIE vol 8666 pp 866605-1 to 866605-16 Feb 2013

[3] IEEE paper ldquoScalable HEVC (SHVC)-Based Video Stream Adaptation in Wireless Networksrdquo by James Nightingale Qi Wang Christos Grecos Centre for Audio Visual Communications amp Networks (AVCN) 2013 IEEE 24th International Symposium on Personal Indoor and Mobile Radio Communications Services Applications and Business Track [4] T Weingand et al Overview of the H264AVC video coding standard IEEE Trans Circuits Syst Video Technol vol 13 no 7 pp 560-576 July 2003 [5] T Hinz et al An HEVC extension for spatial and quality scalable videocoding Proc SPIE Visual Information Processing and Communication IV Feb 2013 [6] B Oztas et al A study on the HEVC performance over lossy networks Proc 19th IEEE International Conference on Electronics Circuits and Systems (ICECS) pp785-788 Dec 2012

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

[7] J Nightingale et al HEVStream a framework for streaming andevaluation of high efficiency video coding (HEVC) content in loss-prone networks IEEE Trans Consum Electron vol58 no2 pp404-412 May 2012 [8] HSchwarz et al ldquoOverview of the scalable extension of the H264AVC standardrdquoIEEE Trans Circuits Syst Video Technology vol17 pp1103-1120Sept 2007 [9] J Nightingale et al Priority-based methods for reducing the impact of packet loss on HEVC encoded video streams Proc SPIE Real-Time Image and Video Processing 2013 Feb 2013 [10] TSchierl et al ldquoMobile Video Transmission codingrdquo IEEE Trans Circuits Syst Video Technol vol 1217 Sept 2007 [11] J Chen K Rapaka X Li V Seregin L Guo M Karczewicz G Van der Auwera J Sole X Wang C J Tu Y Chen ldquoDescription of scalable video coding technology proposal by Qualcomm (configuration 2)rdquo Joint Collaborative Team on Video Coding doc JCTVC- K0036 Shanghai China Oct 2012

[12] ISOIEC JTC1SC29WG11 and ITU-T SG 16 ldquoJoint Call for Proposals on Scalable Video Coding Extensions of High Efficiency Video Coding (HEVC)rdquo ISOIEC JTC 1SC 29WG 11 (MPEG) Doc N12957 or ITU-T SG 16 Doc VCEG-AS90 Stockholm Sweden Jul 2012

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

[13] A Segall ldquoBoG report on HEVC scalable extensionsrdquo Joint Collaborative Team on Video Coding doc JCTVC-K0354 Shanghai China Oct 2012 [14] H Schwarz D Marpe T Wiegand ldquoOverview of the Scalable Video Coding Extension of the H264AVC Standardrdquo IEEE Trans Circuits and Syst Video Technol vol 17 no 9 pp 110311130911120 2007 [15] D Hong W Jang J Boyce A Abbas ldquoScalability Support in HEVCrdquo Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISOIEC JTC1SC29WG11 JCTVC-F290 Torino Italy Jul 2011 [16] G J Sullivan J-R Ohm W-J Han T Wiegand ldquoOverview of the High Efficiency Video Coding (HEVC) Standardrdquo IEEE Trans Circuits and Syst Video Technol to be published [17] J Boyce D Hong W Jang A Abbas ldquoInformation for HEVC scalability extensionrdquo Joint Collaborative Team on Video Coding doc JCTVC-G078 Nov 2011 [18] GJ Sullivan et al ldquoStandardized extensions of High Efficiency Video Coding (HEVC)rdquo IEEE J-STSP vol 7 no 6 pp 1001 ndash 1016 Dec 2013

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

[19] J Chen V Seregin L Guo and M Karczewicz ldquoNon-TE5 on motion mapping in SHVCrdquo Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-L0336 12th Meeting Geneva CH 14ndash23 Jan 2013 [20] J Dong Y He Y He G McClellan E-S Ryu X Xiu and Y Ye ldquoDescription of scalable video coding technology proposal by InterDigitalrdquo Communications Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC-K0034 11th Meeting Shanghai CN 10ndash19 Oct 2012 [21] I Unanue et al ldquoA Tutorial on H264SVC Scalable Video Coding and its Tradeoff between Quality Coding Efficiency and Performancerdquo Recent Advances on Video Coding[online] Available httpwwwdoc88comp-516795349043html

[22] A Segall1 Chen1 Dong and E Alshina TEAl Summary Report Upsampling Filter JCTVC-LO021 Geneva Switzerland 14-23 Jan 2013

[23] J Chen BoG Report on reference layer sample location derivation in SHVC JCTVC-M0449 lncheon Korea 18-26Apr 2013

[24] Inter-Iayer Filtering for Scalable Extension of HEVC Elena Alshina Alexander Alshin Yongjin Cho and JeongHoon Park Samsung Electronics Co Ltd elena _ aalshina alexander _ balshinyongjin9cho jeonghoonsamsungcom Wei Pu Jianle Chen Xiang Li Vadim Seregin and Marta Karczewicz QualcommTechnologies Inc wpu cjianle Ixiang vseregin martakqtiqualcommcom 2013 IEEE

[25] Test sequences for scalable video coding [online] Available ftpftptntuni-hannoverdepubsvctestsequences [26] C-L Su T-M Che and C-Y Huang ldquoCluster-Based Motion Estimation Algorithm With Low Memory and Bandwidth Requirements for H264AVC Scalable Extensionrdquo IEEE Trans on CSVT vol 24 no 6 pp 1016-1024 June 2014 [27] X Li et al ldquoRate-Complexity-Distortion evaluation for hybrid video codingrdquo IEEE Trans on CSVT vol 21 pp 957 - 970 July 2011

[28] K Shah ldquoReducing the complexity of Inter-prediction mode decision for HEVCrdquo MS Thesis University of Texas at Arlington UMI Dissertation Publishing April 2014 [online] Available httpwww-eeutaeduDipCoursesEE5359KushalShah_Thesispdf

[29] SVasudevan ldquoFast intra prediction and fast residual quadtree encoding implementation in HEVCrdquo MS Thesis University of Texas at Arlington UMI Dissertation Publishing Nov 2013 [online] Available httpwww-eeutaeduDipCoursesEE5359indexhtml

[30] httpshevchhifraunhoferdetrachevc JCT-VC Document

[31] Scalable Extension Of HEVC httpwwwmpegorkrdoc2011ECA09C12ED9A8CMPEGED8FACEB9FBCECB49DED9A8CEBB08FEAB8B0EC88A0EC9B8CED81ACEC83B54-2-ED959CECA285EAB8B0-HEVC_extensionpdf

[32] (H265HEVC) Tutorial by Madhukar Budagavi mbudagavisamsungcomhttpwwwutaedufacultykrraodipCoursesEE5359budagaviiscas2014pptpdf

[33] H264 Advanced video coding httpwwwvcodexcomh264html

[34] Test sequences httpsmediaxiphorgvideoderf

[35] Test Sequences ftpftpkwbbccoukhevchm-110-anchorsbitstreams

[36] SHVC software and software manual The source code for the software and its manual is available in the following SVN repository[online] Available httpshevchhifraunhoferdesvnsvn_SHVCSoftware

[37] SHVC bitstream layer parser[online] Available httpr2d2n3potistorycom70

[38] S Riabstev ldquoDetailed overview of HEVCH265rdquo [online] Available httpsappboxcomsrxxxzr1a1lnh7709yvih

[39] KR Rao DN Kim and JJ Hwang ldquoVideo Coding Standards AVS China H264MPEG-4 Part10 HEVC VP6 DIRAC and VC-1rdquo Springer 2014

[40] Access to HM 160 Reference Software httphevchhifraunhoferde

[41] Website on PSNR httpenwikipediaorgwikiPeak_signal-to-noise_ratio