a cultural history of jpeg-1 and a recent...
TRANSCRIPT
January 30, 2008
A Cultural History of JPEG-1 and a Recent EnhancementDr. Joan L. Mitchell, FellowJanuary 30, 2008
2
ABSTRACT:
The "J" in JPEG (Joint Photographic Experts Group) acknowledges its two main parent organizations, ISO (International Organization for Standardization) and CCITT (International Telegraph and Telephone Consultative Committee) which is now called ITU (International Telecommunications Union). The JPEG committee was one of the first times these organizations formed a single committee for the purpose of creating a joint standard despite their different histories, cultures and procedures for settling disputes. Rather than trying to meld its parents' cultures or deciding which set of rules to use for settling conflicts, the original JPEG committee adopted a 100% consensus rule. The first part of this talk will share stories illustrating how achieving complete consensus influenced the definition of this widely-used standard.
The second part of this talk will review the technologies in JPEG-1 (DCT, ADPCM, Huffman, and arithmetic coding). Then it will explain a recent enhancement, ITU-T Rec. T.851. This additional arithmetic coder has minimum latency which makes it particularly suitable for the mobile phone industry.
3
OUTLINE:
o Standards organizations involved
o International effort
o Stories illustrating JPEG culture
o Review technologies
o Recent enhancement
4
Standards Organizations involved:
o Int’l Telegraph and Telephone Consultative Committee (CCITT) - International Telecommunications Union (ITU)
* ITU-T (Terminals) formerly CCITT* ITU-R (Radio) formerly CCIR
o Int’l Organization for Standardization (ISO)
o Int’l Electrotechnical Commission (IEC)
5
ITU-T (formerly CCITT):
o UN sponsored treaty organizationo Official voting done by nationso Issues "Recommendations"o Voluntary for nations to adopt
- Often made law of the lando If controversial, study it moreo Study Groups (SG)
- Subcommittees (SC) study questions
6
ISO
o Non-treaty agency of the United Nationso Works for good of the industryo Decentralized & self-regulatingo “Let the market decide”o Technical committees (TC)
- Study Committees (SC)* Working Groups (WG)
Experts Groups (EG)
7
JPEG (Joint Photographic Experts Group)
o CCITT SG8- Special Rapporteur Group on New Forms
of Image Communicationo ISO
- ISO/TC97/SC2/WG8- ISO/IEC JTC1/SC2/WG8- ISO/IEC JTC1/SC2/WG10- ISO/IEC JTC1/SC29/WG10- ISO/IEC JTC1/SC29/WG1 (with JBIG)
8
International Effort (11 countries):
o North America- USA, Canada
o Europe- Denmark, France, Germany, TheNetherlands, United Kingdom
o Middle East- Israel
o Far East- Japan, Republic of Korea
9
Meeting Locations (’87-’88):
o Darmstadt, Germany Mar. ’87o Copenhagen, Denmark June ’87o Washington, DC USA Oct. ’87o London, UK Dec. ’87o Copenhagen, Denmark Jan. ’88o Ottawa, Canada May ’88- Ad Hoc Rennes, France July ’88- Ad Hoc Torino, Italy Sept. ’88o London, UK Sept. ‘88
10
Meeting Locations (’89-’90):
o Livingston, NJ USA Feb. ’89o Stockholm, Sweden July ’89o Tokyo, Japan Oct. ’89o Tampa, FL USA Mar. ’90- Ad Hoc Budapest, Hungary Apr. ’90o Porto, Portugal July ’90- Ad Hoc editing Cambridge, MA Oct. ’90o Later meetings in Israel and Korea
11
JPEG Culture:
o Adopted 100% consensus ruleo How reach consensus?
- Run a contest w/ objective criteria- Create an ad hoc- Find a “third answer”- Head of Delegations (HOD) meet- National delegation caucus - Vote late in day
12
NEVER LEAVE A MEETING EARLY (if you care about the results)
o On plane discovered registration deadline
o Wrote IBM proposal as others presented
13
PUT IT IN WRITING:
o “If it ain’t in writing, it ain’t.”- Listen politely- Completely ignore
o English was official language- 4 native English speakers (after a year)
o Often technical experts not at meeting- Written information faxed home- May not understand fast American
14
“If I gore your ox today, someday you will gore my ox.”
o One proposal was best & worsto Had used test set to derive tableso Never censored/condemned/embarrassedo Also never mentioned again
15
“It’s Time for the Third Answer”
o Standards’ motto: “Equal pain”o I proposed: “Equal gain”o Two groups butting headso Their views form triangle baseo Now lift both up and find common groundo Third answer better for both
16
One day the smoke disappeared.
o Traditionally several ash trays per tableo More than 1/3 (some chain) smokedo Bill realized “Nobody is smoking.”o Since last meeting Japan had declared
all business meetings smoke free.o No ash trays on table
17
IT’S JUST A GAME! Or is it?
Our motto: Good in hardware, Good in software, And the best.We/Everyone wins
18
Have faith, we have never left unhappy.
o Almost every meeting was a cliff hanger
o Ex: 5pm Friday results appeared poor.- Objective clear winner worst quality.- 7pm threw out objective criteria
19
Blur Lines:
o Picked 3 techniques from June’87 judging
o Formed ad hocs teams to “refine” method- Multiple companies- Multiple continents- Multiple nations
20
Once you agree, unless some new information, the answer won’t change.
o Editing and/ ad hoc reaches consensus.
o Written version distributed.
o JPEG committee gives approval
o Only new information can change answer.
21
HOW CAN WE TELL THEM?
o Header more complex than algorithm
o Half already knew
o Major revision of header structure
22
Ad Hoc Group on Front End & Back End Processing:
o Agreed to one technologyo Took home tape of resultso Horrid on computer monitorso Committee assumed sour grapeso Week long visits to NY/Denmark to solveo AC prediction removes “blockiness”
23
ALWAYS THANK THE MEETING HOSTS
o Formal resolution put in minutes
o Invite meeting coordinators into room
o Read them the resolution
o Everyone claps loudly in appreciation
o Meeting adjourns
24
Recent enhancement:
Intended for mobile phone standard
Complexity Power Battery life
Compression Transmission time – Storage costs
25
TECHNOLOGY REVIEW:
Discrete Cosine Transform (DCT)
Quantization (source of most of loss)
Differential Pulse Code Modulation (DPCM)
Entropy coding– Huffman coding– Arithmetic coding
26
What is in JPEG Part 1?
Baseline sequential (lossy)– 8 bits/sample– 8x8 DCT– Huffman only – Required in all DCT-based decoders Extended sequential (lossy) option
– 8 or 12 bits/sample– 8x8 DCT– Huffman or arithmetic coding (QM-coder)
27
JPEG Part 1:
Progressive (lossy)– 8 or 12 bits/sample– 8x8 DCT– Huffman or arithmetic coding (QM-coder)– SA, SS, and/or hierarchical Lossless
– 2 to 16 bits/sample– Predictive based (DPCM) – Huffman or arithmetic coding (QM-coder)– Sequential and/or hierarchical
28
DCT Transform coding (Zig-zag order)
* DC term. Next 63 are AC terms
29
JPEG example quantization tables (dark large)
30
JPEG Progressive vs. Sequential:
Sequential: 1 pass to final qualityProgressive : Multiple passes to final quality– DC always separate pass(es)– Spectral selection (SS): band of coefficients– Successive approximation (SA)
• 1st pass missing some lsb of AC terms• 1 bit added per later pass(es)
– Can mix SS and SAFollowing images created by W.B. Pennebaker
31
DC without lsb (0.088b/p)
Entropy coder is arithmetic coding. Sequential is to nearest 8 line boundary.
32
+1-5 AC without 3 lsb (0.202 b/p)
33
+6-10 AC without 3 lsb (0.261 b/p)
34
+11-63 AC without 3 lsb (0.283 b/p)
35
+1-63 AC 3rd lsb (0.482 b/p)
36
+1-63 AC 2nd lsb (0.796 b/p)
37
DC +1-63 AC lsb (1.379 b/p)
Original = 512 x 512 bytes
38
JPEG Predictors:
Px = none (difference image)Px = RaPx = RbPx = RcPx = Ra – Rb + RcPx = Ra + (Rb – Rc)/2Px = Rb + (Ra – Rc)/2 Px = (Ra + Rb)/2
c ba x
39
Entropy Coding:
Traditionally losslessHuffman coding– Variable length codes– Integer bits– More complex modelsArithmetic coding– IBM invention– Dynamic probabilities– Better compression
40
Creation of a Huffman Code:
Given four symbols with probabilities 1/2, 1/4, 1/8, and 1/8, create a tree by aggregating the two smallest probabilities at each stage.
Then assign a "0" or "1" to each decision.
111
110
10
0
1
0
1/8
1/2
1/4
1/2
0
0
1
1
1/4
1/8
A
B
C
D
41
Variable Length Decoding:
Decoding by table lookup: bits s L(s) 000 A 1 001 A 1 010 A 1 011 A 1 100 B 2 101 B 2 110 C 3 111 D 3
start
bit?
bit?
bit?
D C
B
A
Finite-state-machine decoding:
42
Arithmetic Coding:
Binary fraction
Elias coding (student exercise)
Carry over problem
43
Binary Fraction between 0 & 1
Take number line from [0 to 1)Binary point before bit streamsDescribe an interval on number lineEnough bits to get uniquely inside
– No outside fraction has same prefix– Two extra bits guarantee inside
44
Elias Coding (aka Jerry Goertzel):
Nested sub-intervalsFinal interval size = Product of probabilities Log (Product P(i))= Sum Log P(i)Student exercise
– Draw number line from 0 to 1– Pick probabilities for symbols A, B, C– Partition line proportional to probabilities– Pick one and partition again
45
Carry over problem:
Ex: Point below at 0.0111111111.... with top of range above 0.10000000...
Resolving carry in encoder requires latency
Can resolve carry in decoder– Stuff a bit if byte 0xFF
46
Recent Enhancement:
o Optional alternative arithmetic coder
o Q15-coder
o Fee-free
o Baseline decoder not mandatory
o Converts into baseline with no loss
47
QM- coder MQ-coder
(JPEG-1 & JBIG-1)SW conventionsEnc. resolves carry– Latency potentially rest of
data stream– Byte stuffing 0xFF-> 0xFF00
Starts with 0xFFD8.jpg file extensionFixed fee
(JPEG2000 & JBIG-2)HW conventionsDec. resolves carry
Latency 1 byteBit stuffing
Fee-free.jp2 file extension
48
MQ- coder Q15- coder
(JPEG2000 & JBIG-2)HW conventionsDec. resolves carry– Latency 1 byte– Bit stuffing
Fee-free.jp2 file extension
(ITU-T Rec. T.851)SW conventionsDec. resolves carry
– Latency 1 byte– Bit stuffing
Starts with 0xFFC8 0005 ‘ac2’
Fee-free.jpe file extension (?)
49
Compression Performance:
10-20 % improvement over baseline
+/- 0.5% compared to QM-coder
Beta IJG software on ITU-T website
January 30, 2008
Thank You