Slide 1

Image Processing Point Processing Filters Dithering Image Compositing Image Compression Point Processing Filters Dithering Image Compositing Image Compression

Slide 2

Images Image stored in memory as 2D pixel array Value of each pixel controls color Depth of image is information per pixel 1 bit: black and white display 8 bit: 256 colors at any given time via colormap 16 bit: 5, 6, 5 bits (R,G,B), 2 16 = 65,536 colors 24 bit: 8, 8, 8 bits (R,G,B), 2 24 = 16,777,216 colors

Slide 3

Fewer Bits: Colormaps Colormaps typical for 8 bit framebuffer depth With screen 1024 * 768 = 786432 = 0.75 MB Each pixel value is index into colormap Colormap is array of RGB values, 8 bits each Only 2 8 = 256 at a time Poor approximation of full color i RR GGBB 0 1 2 255 RR GGBB RR GGBB 0000

Slide 4

Image Processing 2D generalization of signal processing Image as a two-dimensional signal Point processing: modify pixels independently Filtering: modify based on neighborhood Compositing: combine several images Image compression: space-efficient formats Related topics (not in this lecture or this course) Image enhancement and restoration Computer vision

Slide 5

Outline Point Processing Filters Dithering Image Compositing Image Compression

Slide 6

Point Processing Input: a[x,y], Output b[x,y] = f(a[x,y]) f transforms each pixel value separately Useful for contrast adjustment Suppose our picture is grayscale (a.k.a. monochrome). Let v denote pixel value, suppose its in the range [0,1]. f(v) = videntity; no change f(v) = 1-vnegate an image (black to white, white to black) f(v) = v p, p1 darken v f(v)

Slide 7

Point Processing f(v) = videntity; no change f(v) = 1-vnegate an image (black to white, white to black) f(v) = v p, p1 darken v f(v)

Slide 8

Gamma correction compensates for different monitors = 1.0; f(v) = v = 2.5; f(v) = v 1/2.5 = v 0.4 Monitors have a intensity to voltage response curve which is roughly a 2.5 power function Send v actually display a pixel which has intensity equal to v 2.5

Slide 9

Outline Point Processing Filters Dithering Image Compositing Image Compression

Slide 10

Signals and Filtering Audio recording is 1D signal: amplitude(t) Image is a 2D signal: color(x,y) Signals can be continuous or discrete Raster images are discrete In space: sampled in x, y In color: quantized in value Filtering: a mapping from signal to signal

Slide 11

Convolution Used for filtering, sampling and reconstruction Convolution in 1D Chalkboard

Slide 12

Convolve box and step

Slide 13

Convolution filters box tent gaussian

Slide 14

Convolution in 1D a(t) is input signal b(s) is output signal h(u) is filter Convolution in 2D Convolution filters

Slide 15

Filters with Finite Support Filter h(u,v) is 0 except in given region Represent h in form of a matrix Example: 3 x 3 blurring filter As function In matrix form

Slide 16

Blurring Filters A simple blurring effect can be achieved with a 3x3 filter centered around a pixel, More blurring is achieved with a wider n n filter: Original Image Blur 3x3 maskBlur 7x7 mask

Slide 17

Image Filtering: Blurring original, 64x64 pixels3x3 blur5x5 blur

Slide 18

Blurring Filters Average values of surrounding pixels Can be used for anti-aliasing What do we do at the edges and corners? For noise reduction, use median, not average Eliminates intensity spikes Non-linear filter

Slide 19

Example: Noise Reduction Image with noiseMedian filter (5x5)

Slide 20

Example: Noise Reduction Original image Image with noiseMedian filter (5x5)

Slide 21

Edge Filters Discover edges in image Characterized by large gradient Approximate square root Approximate partial derivatives, e.g. Filter =

Slide 22

Sobel Filter Edge detection filter, with some smoothing Approximate Sobel filter is non-linear Square and square root (more exact computation) Absolute value (faster computation)

Slide 23

Sample Filter Computation Part of Sobel filter, detects vertical edges -2 0 0 0 1 1 2 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 a b 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 h

Slide 24

Example of Edge Filter Original image Edge filter, then brightened

Slide 25

Image Filtering: Edge Detection

Slide 26

Outline Display Color Models Filters Dithering Image Compositing Image Compression

Slide 27

Dithering Compensates for lack of color resolution Eye does spatial averaging Black/white dithering to achieve gray scale Each pixel is black or white From far away, color determined by fraction of white For 3x3 block, 10 levels of gray scale

Slide 28

Dithering Dithering takes advantage of the human eye's tendency to "mix" two colors in close proximity to one another.

Slide 29

Dithering Dithering takes advantage of the human eye's tendency to "mix" two colors in close proximity to one another. original no dithering with dithering Colors = 2 24 Colors = 2 8

Slide 30

Ordered Dithering How do we select a good set of patterns? Regular patterns create some artifacts Example of good 3x3 dithering matrix

Slide 31

Floyd-Steinberg Error Diffusion Diffuse the quantization error of a pixel to its neighboring pixels Scan in raster order At each pixel, draw least error output value Add the error fractions into adjacent, unwritten pixels If a number of pixels have been rounded downwards, it becomes more likely that the next pixel is rounded upwards 7/16 3/165/161/16

Slide 32

Floyd-Steinberg Error Diffusion

Slide 33

From http://www.cs.rit.edu/~pga/pics2000/node1.htmlhttp://www.cs.rit.edu/~pga/pics2000/node1.html Enhances edges Retains high frequency Some checkerboarding

Slide 34

Color Dithering Example: 8 bit framebuffer Set color map by dividing 8 bits into 3,3,2 for RGB Blue is deemphasized because we see it less well Dither RGB separately Works well with Floyd-Steinberg Generally looks good

Slide 35

Outline Display Color Models Filters Dithering Image Compositing Image Compression

Slide 36

Image Compositing Represent an image as layers that are composited (matted) together

Slide 37

Image Compositing To support this, give image an extra alpha channel in addition to R, G, B Alpha is opacity: 0 if totally transparent, 1 if totally opaque Alpha is often stored as an 8 bit quantity; usually not displayed. Mathematically, to composite a 2 over a 1 according to matte b(x,y) = (1- (x,y))a 1 (x,y)+ (x,y)a 2 (x,y) = 0 or 1 -- a hard matte, = between 0 and 1 -- a soft matte Compositing is useful for photo retouching and special effects.

Slide 38

Special Effects: Compositing Lighting match Proper layering Contact with the real world Realism (perhaps) Applications Cel animation Blue-screen matting

Slide 39

Roger Rabbit http://members.tripod.com/~Willy_Wonka/Theatr.jpg

Slide 40

Special Effects: Green Screen Green screen Second green screen shot Compositing of everything Digital Domain (from http://www.vfxhq.com/1997/titanic-picssink.html )http://www.vfxhq.com/1997/titanic-picssink.html

Slide 41

Special Effects: Green Screen Green screen Compositing of people with ship model, sky and digital water Digital Domain (from http://www.vfxhq.com/1997/titanic-picssink.html )http://www.vfxhq.com/1997/titanic-picssink.html

Slide 42

Outline Display Color Models Filters Dithering Image Compositing Image Compression

Slide 43

Exploit redundancy Coding: some pixel values more common Interpixel: adjacent pixels often similar Psychovisual: some color differences imperceptible Distinguish lossy and lossless methods

Slide 44

Image Sizes 1024*1024 at 24 bits uses 3 MB Encyclopedia Britannica at 300 pixels/inch and 1 bit/pixes requires 25 gigabytes (25K pages) 90 minute movie at 640x480, 24 bits per pixels, 24 frames per second requires 120 gigabytes Applications: HDTV, DVD, satellite image transmission, medial image processing, fax,...

Slide 45

Exploiting Coding Redundancy Not limited to images (text, other digital info) Exploit nonuniform probabilities of symbols Entropy as measure of information content H = -S i Prob(s i ) log 2 (Prob(s i )) Low entropy non uniform probability High entropy uniform probability If source is independent random variable need H bits

Slide 46

Exploiting Coding Redundancy Idea: More frequent symbols get shorter code strings Best with high redundancy (= low entropy) Common algorithms Huffman coding LZW coding (gzip)

Slide 47

Huffman Coding Codebook is precomputed and static Use probability of each symbol to assign code Map symbol to code Store codebook and code sequence Precomputation is expensive What is symbol for image compression? lossless

Slide 48

Exploiting Interpixel Redundancy Neighboring pixels are correlated Spatial methods for low-noise image Run-length coding: Alternate values and run-length Good if horizontal neighbors are same Can be 1D or 2D (e.g. used in fax standard) WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWWWWWW WWWWWWWWWWWWWWWWBWWWWWWWWWWWWWW 12W 1B 12W 3B 24W 1B 14W Quadtrees: Recursively subdivide until cells are constant color Region encoding: Represent boundary curves of color-constant regions lossless

Slide 49

Improving Noise Tolerance Predictive coding: Predict next pixel based on prior ones Output difference to actual Transform coding Exploit frequency domain Example: discrete cosine transform (DCT) Used in JPEG lossy compression

Slide 50

Discrete Cosine Transform Used for lossy compression (as in JPEG) Subdivide image into n x n blocks (n = 8) Apply discrete cosine transform for each block Each tile is converted to frequency space

Slide 51

Discrete Cosine Transform Quantize Human eye good at seeing variations over large area Not good at seeing the exact strength of a high frequency Greatly reducing the amount of information in the high frequency components Use variable length coding (e.g. Huffman)