how compression worksecee.colorado.edu/~ecen5653/ecen5653/lectures/pdf/cardinal-peak-… · video...

April13,2009Slide1

HowCompressionWorks

HowdyPierceCardinalPeak,LLC

April13,2009Slide2

VideoCompression

•  Uncompressed,full‐colorvideoisBIG•  FullD1video(720x480@30fps)

720x480x24bppx29.97fps=248Mbpsofdata!

April13,2009Slide3

VideoCompression

•  Uncompressed,full‐colorvideoisBIG•  FullD1video(720x480@30fps)

720x480x24bppx29.97fps=248Mbpsofdata!

•  Subsampledcolor(4:2:0) 720x480x12bppx29.97fps=124Mbps

•  Tobeviablecommercially,weneedtogetthisintothe1.0to10.0Mbpsrange

April13,2009Slide4

2typesofcompression

•  Lossless–getbackwhatyouputin•  Lossy–getbacksomethingclosetowhatyouputin

April13,2009Slide5

HowManyBits?

•  Howmanybitsshouldittaketorepresenta32‐symbolalphabet?

SPC F Q' G R, H S‐ I T. J U; K VA L WB M XC N YD O ZE P

April13,2009Slide6

HowManyBits?

•  Weknowsomethingaboutthedistribu]on

Distribu]onbasedonWutheringHeightsbyEmilyBrönte

Sym P Sym P Sym PSPC 17.28% F 1.69% Q 0.07%' 1.08% G 1.67% R 4.66%, 1.54% H 5.09% S 4.78%‐ 0.32% I 5.64% T 6.82%. 0.82% J 0.10% U 2.34%; 0.29% K 0.62% V 0.72%A 6.09% L 3.24% W 1.66%B 1.09% M 2.08% X 0.13%C 1.90% N 5.64% Y 1.70%D 3.76% O 5.83% Z 0.03%E 10.05% P 1.26%

April13,2009Slide7

HuffmanCode

S RE

V KPW ,

I N H

Q ZJ

F GLT A O

X ‐ ;

.YD

B ‘UM C

SPC

April13,2009Slide8

HuffmanCode

S RE

V KPW ,

I N H

Q ZJ

F GLT A O

X ‐ ;

.YD

B ‘UM C

SPC

0 1

0 1 01

0

0

1

1

0

0

0

0

0

0

0 11

1

1

11

Sowewouldcodeeachsymbolasfollows:

001 E111 SPC0000 S0001 R0100000 V

…etc

April13,2009Slide9

HuffmanCode

S RE

V KPW ,

I N H

Q ZJ

F GLT A O

X ‐ ;

.YD

B ‘UM C

SPC

0 1

0 1 01

0

0

1

1

0

0

0

0

0

0

0 11

1

1

11

3bits4bits5bits6bits7bits

9bits10bits11bits

April13,2009Slide10

EntropyCoding(akaVariableLengthCoding)

•  Weendupusingfewerbitsformorecommonleners,andmorebitsforlesscommonleners

Weightedaveragenumberofbitspersymbol:4.29

Sym P bits Sym P bits Sym P bitsSPC 17.28% 3 F 1.69% 6 Q 0.07% 11' 1.08% 7 G 1.67% 6 R 4.66% 4, 1.54% 6 H 5.09% 4 S 4.78% 4‐ 0.32% 9 I 5.64% 4 T 6.82% 4. 0.82% 7 J 0.10% 10 U 2.34% 6; 0.29% 9 K 0.62% 7 V 0.72% 7A 6.09% 4 L 3.24% 5 W 1.66% 6B 1.09% 7 M 2.08% 6 X 0.13% 9C 1.90% 6 N 5.64% 4 Y 1.70% 6D 3.76% 5 O 5.83% 4 Z 0.03% 11E 10.05% 3 P 1.26% 6

April13,2009Slide11

ClaudeShannon(1916‐2001)

•  “Fatherofinforma]ontheory”

•  Laidthetheore]calgroundworkfordatacompressionandmodula]on

April13,2009Slide12

EntropyCoding:Theory

•  AccordingtoShannon’sSourceCodingTheory,theop]mallengthforacodeis

−logb(P)

•  Where

Pistheprobabilityforthesymbolbeingcoded

bisthenumberofsymbolsusedtomaketheoutputcodes(i.e.,always2forbinary)

April13,2009Slide13

EntropyCoding

•  HowclosedoesourHuffmancodecome?

Weightedaveragenumberofbitspersymbol:4.29Weightedaveragebitspersymbol,theore]callimit:4.26

Sym P bits log2(P) Sym P bits log2(P) Sym P bits log2(P)SPC 17.28% 3 2.53 F 1.69% 6 5.88 Q 0.07% 11 10.40' 1.08% 7 6.53 G 1.67% 6 5.90 R 4.66% 4 4.42, 1.54% 6 6.02 H 5.09% 4 4.30 S 4.78% 4 4.39‐ 0.32% 9 8.30 I 5.64% 4 4.15 T 6.82% 4 3.87. 0.82% 7 6.93 J 0.10% 10 10.00 U 2.34% 6 5.41; 0.29% 9 8.43 K 0.62% 7 7.34 V 0.72% 7 7.12A 6.09% 4 4.04 L 3.24% 5 4.95 W 1.66% 6 5.91B 1.09% 7 6.51 M 2.08% 6 5.59 X 0.13% 9 9.59C 1.90% 6 5.72 N 5.64% 4 4.15 Y 1.70% 6 5.88D 3.76% 5 4.73 O 5.83% 4 4.10 Z 0.03% 11 11.67E 10.05% 3 3.31 P 1.26% 6 6.31

April13,2009Slide14

EntropyCoding:Theory

•  Entropycodingislossless•  ItturnsoutthatHuffmancodesareop]mal–  IFrestrictedtocodingonesymbolata]me

•  However,itispossibletodobenerbycompressingmorethanonesymbolata]me– Arithme]ccoding

April13,2009Slide15

HowtoCreateaHuffmanCode

•  Orderthesymbolsbyprobability

•  Lowest2symbolsbecomeasub‐tree

•  Iterateun]lallsymbolsarehandled

April13,2009Slide16

HuffmanCodeExercise

•  MakeyourownHuffmancodewiththissetofsymbols:

Sym PN. 16.05%NE. 4.17%E. 32.26%SE. 8.19%S. 12.20%SW. 4.65%W. 8.03%NW. 14.45%

April13,2009Slide17

HuffmanCodeExercise

NW NE

SW NESSE W

Weightedaveragenumberofbitspersymbol:2.77Weightedaveragebitspersymbol,theore]callimit:2.71

Sym P bits log2(P)N. 16.05% 3 2.64NE. 4.17% 4 4.58E. 32.26% 2 1.63SE. 8.19% 3 3.61S. 12.20% 3 3.04SW. 4.65% 4 4.43W. 8.03% 3 3.64NW. 14.45% 3 2.79

April13,2009Slide18

VideoandImageCompression

•  Mostcommonformsofvideoandimagecompressionhavethesamecentralconcept

–  JPEG– MPEG‐1– MPEG‐2

– MPEG‐4– H.264

•  Obviously,therearedifferencestoo

April13,2009Slide19

VideoandImageCompression8x8block

April13,2009Slide20

VideoandImageCompression8x8block

101 100 94 102 97 91 88 83101 99 98 103 93 93 107 11098 97 97 97 103 101 94 10097 98 99 100 103 105 101 9699 100 104 104 100 107 109 8999 101 106 105 116 113 87 5894 69 70 66 79 70 40 2659 30 27 33 32 37 45 41

SourceMatrix

April13,2009Slide21

HowManyBits?

•  Howmanybitsshouldittaketocodethisblock?

SourceMatrix

April13,2009Slide22

3KeyStepsofImageCompression

1.  DiscreteCosineTransform2.  Quan]za]on3.  EntropyCoding

DCT Quan]za]on EntropyCoding8x8block

codeddata

April13,2009Slide23

DiscreteCosineTransform

•  Transformsonematrixintoanothermatrix

•  Reversible&lossless

•  Thetransformedmatrixhassomeinteres]ngproper]es

– Mostinforma]onrepresentedinthelow‐ordercoefficients

April13,2009Slide24

DiscreteCosineTransform

DCT

SourceMatrix

DCTCoefficients

April13,2009Slide25

DiscreteCosineTransform

DCTthe“DCcoefficient”(avgofini]almatrix)

Increasinglyhigherfrequenciesin

ver]caldomain

Increasinglyhigherfrequenciesinhorizontaldomain

DCTCoefficients

SourceMatrix

April13,2009Slide26

3KeyStepsofImageCompression

1.  DiscreteCosineTransform2.  Quan]za]on3.  EntropyCoding

DCT Quan]za]on EntropyCoding8x8block

codeddata

April13,2009Slide27

Quanfzafon

OriginalValues Quan]zedValues

•  Quan]za]onisthelossystepinimagecompression

April13,2009Slide28

Quanfzafon

•  Forimagecompression,therearetwostepstoquan]za]on:– Applyaweigh]ngmatrixtoDCTcoefficients

– ApplyauniformscalefactorMPEG‐2DefaultWeigh]ngMatrix

April13,2009Slide29

AherQuanfzing

DCTCoefficients

Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8

Quan]za]on

April13,2009Slide30

AherQuanfzing

DCTCoefficients

Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8

Quan]za]on

Thisisthedataweneedtotransmitto

thedecoder

April13,2009Slide31

ReversingtheProcess

DCTQuan]za]on

(q=8)Inverse

Quan]za]onInverseDCT

OriginalSourceMatrix RecoveredOutput(q=8)

April13,2009Slide32

HowLossyIsIt?OriginalSourceMatrix RecoveredOutput(q=8)

April13,2009Slide33

HowLossyIsIt?OriginalSourceMatrix RecoveredOutput(q=8)

ErrorMatrix

SumofSquaresoferrormatrix:4335

April13,2009Slide34

ComparingQuanfzafonFactors

Quan]zedwithMPEG‐2weigh]ngmatrixandscale=8

• Sum‐of‐squareserror:4335• Numberofnon‐zerocoefficients:11

Quan]zedwithMPEG‐2weigh]ngmatrixandscale=1

• Sum‐of‐squareserror:231• Numberofnon‐zerocoefficients:39

SameDCTCoefficients

April13,2009Slide35

3KeyStepsofImageCompression

1.  DiscreteCosineTransform2.  Quan]za]on3.  EntropyCoding

DCT Quan]za]on EntropyCoding8x8block

codeddata

April13,2009Slide36

Run‐LengthCoding

•  Applyazig‐zagscanpanern: 86,1,7,5,‐1,0,1,0,0,2,‐1,1,0,‐1,0,0,0,0,‐1,0,0,…

•  Notethattherearealotofrunsofzeros!•  Breakinto“words”ofXzeros,followedbyacoefficient

Quan]zeddatawithMPEG‐2weigh]ngmatrixandscale=8

April13,2009Slide37

Run‐LengthCoding

•  TheDCcoefficient(86)issentdirectly* =8bits

•  Remainingcoefficientsaresentas10“words”,plusaspecialend‐of‐blockword

86,1,7,5,‐1,0,1,0,0,2,‐1,1,0,‐1,0,0,0,0,‐1,0,0,…

(*)Inprac]ce,it’smuchmorecomplexthanthis,butforsakeofsimplicity…

April13,2009Slide38

MPEG‐2HuffmanTable

April13,2009Slide39

Run‐LengthCoding

•  ApplyingtheMPEG‐2Huffmancodetothissetofwords,weconsume52bits

•  So,wecodetheen]re8x8blockin60bits!

April13,2009Slide40

ContextAdapfveVLC(CAVLC)

•  OneproblemwiththeMPEG‐2approach:– Huffmantablesarefixed•  Reflectthedistribu]onofwordsinsomesmallsampleofsourcematerialwhenthestandardwaswrinen

–  Ifthisdoesn’treflectthedistribu]onofwordsinthevideobeingencoded,wegetinefficiency

•  H.264improvesonthisbyhaving5differentVLCtables– Selectedcontextuallybasedon#ofnon‐zerocoefficientsinneighboringblocks

April13,2009Slide41

ThankYou!