image processing architecture, © 2001-2004 oleh tretiakpage 1lecture 8 ecec 453 image processing...

Image Processing Architecture, © 2001-2004 Oleh Tretiak Lecture 8

ECEC 453 Image Processing Architecture

Lecture 8, 2/13/2004

JPEG Modes of Operation, Implementation Issues, Interchange, Extensions

Oleh Tretiak

Drexel University


Review: Block Diagram — JPEG DCT

DCT Quantizer

QuantizationTables

CodingTables

Coder

Headers

Tables

Data

For color images, each component is coded separately

Encoding


Review: JPEG DCT Decoder

IDCTInverseQuantizerDecoder

Headers

Tables

Data

Quantization

TablesCodingTables

The JPEG DCT coder is (almost) symmetric

Decoding


Review: Major Components Color Data interleaving Chrominance subsampling Quantization Entropy coding

DC coefficients AC coefficients


About Color In television (and Physics) a color image specified by giving 3

intensity images Usual description are CIE (Commission Internationale de

L’Eclairage) spectral primary R, G, B components Other signals, e.g., to generate an image on a monitor, to print on

an inkjet printer, are related to RGB components by simultaneous equations

There are several standard coordinate systems. Below are the equations to transform from CIE RGB to CIE XYZ. The signal Y has special importance: it gives the intensity of the image.

X

Y

Z

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

0.490 0.310 0.200

0.177 0.813 0.011

0.000 0.010 0.990

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

R

G

B

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥


More Color Lore These equations must be applied to every pixel in the image

(many multiplications and additions!) Some commonly used color coordinates (see Appendix A)

Y, Cr, Cb — most common system for still and video image compression

Y, I, Q — used in NTSC color television systems (USA) Y, U, V — PAL color television system (Europe) L*, a*, b* — CIELAB, standard for color fax


Review: Data Interleaving with Subsampling

Example: a color image with Y (intensity), Cb, Cr, (color) components is subsampled so that one color block corresponds to four Y blocks

MCU1 = Y00 Y01 Y10 Y11 Cr00 Cb00, MCU2 = Y02 Y03 Y12 Y13 Cr01 Cb01

YCrCb01210010101012


Implications of Subsampling Original Color Image: 3 x 256 x 256 ~ 196 Kbytes Subsample chrominance: 256 x 256 + 2 x 128 x 128 ~ 98

Kbytes Luminance has 32 x 32 = 1024 DCT blocks Each chrominance component has 16 x 16 = 256 DCT blocks There are 4 luminance blocks for each pair of chrominance

block Fewer blocks —> less computing!

Luminance Chrominance


Review: Quantization Tables Quantization equation

y — DCT coefficient, q — quantization step, z — coded value All DCT blocks in a component have the same quantization

table Typically, luminance and color components have different

quantization tables qij = 1 leads to (near) lossless coding

Image quality can be varied wit an overall quality factor qij = QFxbij, bij ~ baseline quantization steps, QF ~ quality factor

QF = 1, high quality, larger QF -> lower quality, higher compression

z ij = round(yij / qij )


Quantization and Compression

Very high quality: compression = 2.33PhotoShop Image

Very low quality: compression = 115Produced by MATLABwith Quality = 0


Review: Entropy Coding DCT coefficients are pre-processed before entropy coding

DC coefficient is coded differently from other coefficients Non-zero coefficients are run-length coded

DC coefficients are preprocessed: inter-block differences Huffman coding commonly used

Different tables for DC and AC Different tables for Chrominance and Luminance

Coding tables may be included in bit stream


Coding AC Coefficients AC coefficients are coded in zig-zag order to maximize possible

runs of zeros. Code unit consist of run length

followed by coefficient size. Baseline coding of size is the

same as for DC differences (Table 2.9)

Example: run of 6 zeros, size = -18. In the table, -18is in category 5. Code is(6/5, 01101). If the Huffmancode for 6/5 is 1101, codeword = 110101101


Huffman Coding - Block Diagram


This Lecture JPEG modes of operation Implementation Issues File Formats JPEG extensions JPEG2000


Four JPEG Modes of OperationSequential

DCT(baseline)

ProgressiveDCT

Sequentiallossless

Hierarchical


Progressive Modes

Spectral Successive approximation


JPEG Hierarchical CodingDecodeDecodeDecode+224X4’X2’X1’42EncodeDecode2+-ΣΣDecode2+-Σ1X2X4X


JPEG Implementation Issues Hardware vs. Software IDCT Complexity Arithmetic Precision Required JPEG coding Tables Color Conversion and Display Considerations JPEG compressed file format


Hardware vs. Software Implementation

Example 640x480 DCT, color image -> 23 MOP (mega operations) Color conversion ~ 40% DCT ~ 45% Coding ~ 15%

Software Less than a second on microcomputers High-efficiency, scaled DCT & IDCT

Winograd & Feig method

Hardware Color conversion, subsampling, DCT are well adapted to special-purpose

computers Regular dataflow DCT (Chen, separable) better adapted to parallel

hardware


IDCT, Precision IDCT has different details, but same general characteristics as

DCT Many zero coefficients - can be used to reduce complexity

Smooth areas -> DC only Low detail areas -> 4x4 IDCT

Precision — DCT coefficients have fractional values! Keep integer only - near lossless! Dynamic range: 8x8 DCT -> x 8 expansion Example: 8 bit [0-255] input range

Subtract 128 -> range [-128, 127] After DCT -> range [-1024, 1023] worst case Rule of thumb: 12 bit accuracy for DCT constants, 16 bits for

intermediate calculations gives ‘exact’ answers


JPEG Coding and Q Tables Optimal performance is obtained with tables adapted to image Options

Baseline quantization & coding Baseline quantization, custom coding Custom quantization, baseline coding Custom quantization & coding

Big difference only for low rate (web?) images At low rate, coding has bigger effect than quantization


Color Conversion and Display Color can be represented in may ways

RGB, Y Cb Cr, YUV, CIELAB Color is usually coded at half resolution Encoding

Color convert, if necessary Downsample chrominance ~ 40% of computational load

Decoding Upsample color Convert to display coordinates

Simple algorithm structure (parallelizable)


JPEG Compressed File Format JPEG standard specifies bitstream

Compressed image = SOI , frame , EOI SOI = FFD8, EOI = FFD9

frame = [tables/misc.], frame header, { scan, [DNL segment]}* [] denotes optional component, {}* means that this can be repeated

frame header contains information about mode of operation, bits/pixel, picture dimensions, number of components, etc

scan = [tables/misc.], scan header, {ECS, [RST]}*, ECS ECS ~ entropy coded segment, RST ~ restart marker for error recovery

ECS = {MCU}* Color coordinates? Image physical (print) dimensions? Ancillary

information (comments) are not specified.


File Formats JFIF (JPEG file interchange format): De-facto JPEG file standard

Developed by Eric Hamilton at C-Cube Microsystems ftp://ftp.uu.net/graphics/jpeg/ Courtesy of UUNET Inc. jfif.pdf (our web site, notes)

SPIFF: Official JPEG file interchange format (still picture interchange file format)

Includes following compression methods None, JPEG, JBIG, Modified Huffman, MR, MMR

Backward compatible (mostly?) with JFIF ITU standard (I can’t find it on the web)

FlashPix: Potential industry standard. Hierarchical, using JPEG EXIF, DCF are standards for digitial cameras. They are based on

JPEG and lossless TIFF


How MATLAB reads JPEG * RJPGC.MEX * * This is a mex interface to the Independent Jpeg Group's (IJG) * LIBJPEG libaray. This can read RGB and grayscale JPEG images. * * * The IJG code is available at: * http://www.imagefolio.com/users/docs/pro31/imagemagick.shtml


About JFIF APP0 = FFE0 marker used to specify extensions Specify Y, Cb, Cr for components Includes ‘thumbnail’ picture for previewing

Thumbnail can be RGB, index color, or compressed Image orientations: first scan line is on top Spatial relation of components specified Pixel aspect ratio (horizontal and vertical density) are specified

Component 1 (full)Component 2 (down 2)Component 3 (down 4)


JFIF Sample File written by PhotoShop

[cbis 210] % od -c -x Mallorca | more0000000 377 330 377 340 \0 020 J F I F \0 001 002 001 \0 d ffd8 ffe0 0010 4a46 4946 0001 0201 0064

SOI, start of image

0000440 \0 \0 \0 031 h t t p : / / w w w . p 0000 0019 6874 7470 3a2f 2f77 7777 2e700000460 h o t o d i s c . c o m / \0 8 B

URL of image copyright owner?

0001200 F i l e w r i t t e n b y 4669 6c65 2077 7269 7474 656e 2062 7920

0001220 A d o b e P h o t o s h o p 250 4164 6f62 6520 5068 6f74 6f73 686f 70a8

APP0 marker Version


JPEG Extensions Part 3 of JPEG standard - ‘appendix’

Variable Quantization Selective Refinement Tiling Color Facsimile Standard

SPIFF is part of this extension


Variable Quantization JPEG does not allow changes of quantization within scan Variable quantization (depending on block content) is known to

be useful For example, graphics and image are optimized with different

coding Extension allows per-block quality factor that multiplies

quantization matrix


Selective Refinement Some parts of image can be coded in greater detail than others Three types

Hierarchical Some parts have more hierarchy levels (higher resolution)

Progressive Some parts have additional DCT coefficients

Component Some parts have fewer components. Good for coding images that are

part gray scale and part color


Tiling What can be done with huge images?

JPEG limits image to 65,535 pixels on side Solution: divide huge image into tiles

Allows random access into images Tiling options

Simple tiling: all tiles have the same size Pyramidal tiling: multiple resolutions may be included Composite tiling: tiles can be of different resolution and can

overlap


Color Facsimile Standard Much awaited extension to original fax standard Input is color image scanned at 200 dpi Specifies color transformation and coding

CIELAB color coding Chrominance components subsampled by a factor of 2 Image coded with JPEG (lossy or lossless)

Optional features allow higher spatial resolution, no color subsampling, higher precision


From University of Patras (Greece). Avaialble from www.dsp.toronto.edu/~dsp/JPEG2000/


Compression = 64

JPEG JPEG2000


Compression = 32

JPEG JPEG2000


JPEG2000 Progress

Kakadu software: http://www.kakadusoftware.com/

Apple's Quicktime v6 (for MAC only right now);Yahoo's latest Messanger [sic] application (uses JPEG2000 for

live videoMicroImages’ TNT products for Geospatial imagery (see

http://www.microimages.com for more info)

JAVA sources http://jj2000.epfl.ch (Ecole Polytechnique Federale Lausanne)


JPEG: Summary Standard came at the right time

Internet Digital publishing Digital cameras

Complexity of JPEG has been overcome by computer power 90/10 rule

90 % of applications use 10 % of JPEG

image processing architecture, © 2001-2004 oleh tretiakpage 1lecture 8 ecec 453 image processing...

Documents