Image

2012, Fall

Image Processing

Quantization Uniform Quantization Halftoning & Dithering

Pixel Operation Add random noise Add luminance Add contrast Add saturation

Filtering Blur Detected edges

Warping Scale Rotate Warp

Combining Composite Morph

Ref] Thomas Funkhouser, Princeton Univ.

What is an Image?

An image is a 2D rectilinear array of samples

A pixel is a sample, not a little square

Image Resolution

Intensity resolution Each pixel has only “Depth” bits for colors/intensities

Spatial resolution Image has only “Width” x “Height” pixels

Temporal resolution Monitor refreshes images at only “Rate” Hz

Width x Height Depth Rate

NTSC 640x480 8 30

Workstation 1280x1024 24 75

Film 3000 x 2000 12 24

Laser Printer 6600x5100 1 -

Typical Resolutions

Hei

ght

WidthDepth

Source of Error

Intensity quantization Not enough intensity resolution

Spatial aliasing Not enough spatial resolution

Temporal aliasing Not enough temporal resolution

),(

22 )),(),((yx

yxPyxIEReal value Sample value

Quantization

Artifact due to limited intensity resolution Frame buffers have limited number of bits per pixel Physical devices have limited dynamic range

Uniform Quantization

)5.0),((),( yxItruncyxP

반올림 (Round)

Uniform Quantization

Image with decreasing bits per pixel

Reducing Effects of Quantization

Our eyes tend to integrate or average the fine detail into an overall intensity

Halftoning A technique used in newspaper printing The process of converting a continuous tone image into an

image that can be printed with one color ink (grayscale) or four color inks (color)

Dithering Dithering is used to create a wide variety of patterns for use

as backgrounds, fills and shadings, as well as for creating halftones, and for correcting aliasing.

Halftones are created through a process called dithering Dithering Methods( 방법론 ) , Halfton result( 결과물 )

Classical Halftoning

Use dots of varying size to represent intensities Area of dots proportional to intensity in image

Halftoning

A technique used in newspaper printing Only two intensities are possible, blob of ink and no

blob of ink

But, the size of the blob can be varied Also, the dither patterns of small dots can be used

Classical Halftoning

Original Image Halfton Image

Halfton patterns

Use cluster of pixels to represent intensity Trade spatial resolution for intensity resolution

2 by 2 pixel grid patterns

3 by 3 pixel grid patterns

mask

Halfton patterns

Dithering

Distribute errors among pixels Exploit spatial integration in our eye Display greater range of perceptible intensities

Halftoning – Moire Patterns

Moire[more-ray] patterns Repeated use of same dot pattern for particular shade results

in repeated pattern (Perceived as a moire pattern) result from overlapping repetitive elements Instead, randomize halftone pattern

Halftoning – Moire Patterns

Signal Processing (1/2)

Signal Real World : Continuous Signal Digital World: Discrete Signal

Image spatial domain

• not temporal domain

• intensity variation

over space

Signal Processing (2/2)

Sampling and reconstruction process Sampling

• The process of selecting a finite set of values from a signal

• Samples (the selected values)

Reconstruction• recreate the original continuous signal from the samples

Sampling and Reconstruction

Aliasing

In general Artifact due to under-sampling or poor reconstruction

Specifically, in graphics Spatial aliasing Temporal aliasing

Sampling below the Nyquist rate

Nyquist rate and aliasing

Nyquist rate lower bound on the sampling rate Sampling at the Nyquist rate

• sampling rate > 2* max frequency

주파수 (frequency): 초당 사이클의 횟수 (Hz)

Spatial Aliasing

Artifact due to limited spatial resolution

Spatial Aliasing

Artifact due to limited spatial resolution

Temporal Aliasing

Artifact due to limited temporal resolution Spinning Backward

• The wheel is spinning at 5 revolution per second– 3rd row appear to be spinning backward at revolution per

second

Temporal Aliasing

Artifact due to limited temporal resolution Flickering

Anti-Aliasing

Sample at higher rate Not always possible Doesn’t always solve problem

Pre-filter to form bandlimited signal Form bandlimited function(low-pass filter) Trades aliasing for blurring

Pixel Operations

Grayscale Compute luminance L L=0.30R+0.59G+0.11B (old)

0.2125R+0.7154G+0.0721B (CRT, HDTV)

Original Image Grayscale Image

Pixel Operations

Negative Image Simply inverse pixel components Ex) RGB(5, 157, 178) RGB(250, 98, 77)

Original Image Negative Image

(R, G, B) (255-R, 255-G, 255-B)

Pixel Operations

Adjusting Brightness Simply scale pixel components

• Must clamp to range (e.g., 0 to 255)

(R, G, B) (R, G , B) * scale

Pixel Operations

Adjusting Contrast Compute mean luminance L for all pixels

• Luminance = 0.30* r + 0.59*g + 0.11*b

Scale deviation from L for each pixel component• Must clamp to range (e.g., 0 to 255)

R L + (R-L)*scaleG L + (G-L)*scaleB L + (B-L)*scale

Pixel Operations

Changing Brightness

Changing Contrast

Pixel Operations

Adjusting Saturation Color Convert RGB HSV Scale from S for each pixel component Color Convert HSV RGB

• Must clamp to range (e.g., 0 to 255)

Filtering

Convolution Spatial domain

• Output pixel is weighted sum of pixels in neighborhood of input image

• Patten of weights is the “filter”

Filtering

Convolution

Filtering

Adjust Blurriness Convolve with a filter whose entries sum to one

• Each pixel becomes a weighted average of its neighbors

Filter Sum = 1

Filtering

Edge Detection Convolve with a filter that finds differences between

neighbor pixels

Filtering

Sharpening Convolve with a Sharpening filter

010

151

010

Filter

Filtering

Embossing Convert the image into grayscale Convolve with a Embossing filter Plus 0.5 for each pixel

• Must clamp to range (e.g., 0 to 255)

100

010

002

Filter

Summary

Image representation A pixel is a sample, not a little square Images have limited resolution

Halftoning and dithering Reduce visual artifacts due to quantization Distribute errors among pixels

• Exploit spatial integration in our eye

Sampling and reconstruction Reduce visual artifacts due to aliasing Filter to avoid undersampling

• Blurring is better than aliasing

Image Processing

Quantization Uniform Quantization Halftoning & Dithering

Pixel Operation Add random noise Add luminance Add contrast Add saturation

Filtering Blur Detected edges

Warping Scale Rotate Warp

Combining Composite Morph

Ref] Thomas Funkhouser, Princeton Univ.

Image Warping

Move pixels of image Mapping

• Forward

• Reverse

Resampling• Point sampling

• Triangle Filter

• Gaussian filter

Mapping

Define transformation Describe the destination (x,y) for every location (u,v) in the

source (or vice-verse, if invertible)

Source (u, v) Destination (x, y)

Example Mappings

Scale by factor x = factor * u y = factor * v

Source (u, v) Destination (x, y)

Example Mappings

Rotate by degrees x = ucosvsin y = usinvcos

Source (u, v) Destination (x, y)

Example Mappings

Shear in X by factor x = u + factor * v y = v

Shear in Y by factor x = u y = v + factor * u

Other Mappings

Any function of u and v X = fx (u, v)

Y = fy (u, v)

Image Warping Implementation I

Forward mappingfor (int u =0; u < umax; u++) {

for (int v = 0; v < vmax; v++) {

float x = fx (u, v);

float y = fy (u, v);

dst (x, y) = src(u, v) ;

}

}

Forward Mapping

Iterate over source image Many source pixels can map to same destination pixel Some destination pixels may not be covered

Image Warping Implementation II

Reverse mappingfor (int x =0; x < xmax; x++) {

for (int y = 0; y < ymax; y++) {

float u = fx-1 (x, y);

float v = fy-1 (x, y);

dst (x, y) = src(u, v);

}

}

Reverse Mapping

Iterate over destination image Must resample source May oversample, but much simpler

Resampling

Evaluate source image at arbitrary (u,v) (u, v) does not usually have integer coordinates

Resampling

Resampling Point sampling Triangle filter Gaussain filter

Point Sampling

Take value at closest pixel int iu = trunc(u + 0.5) int iv = trunc(v + 0.5) dst(x, y) = src(iu, iv)

This method is simple, but it cause aliasing

Filtering

Compute weighted sum of pixel neighborhood Weights are normalized values of kernel function Equivalent to convolution at samples

Triangle Filtering

Kernel is triangle function

Triangle Filtering (with width = 1)

Bilinearly interpolate four closest pixels a = linear interpolation of src(u1, v2) and src(u2, v2) b = linear interpolation of src(u1, v1) and src(u2, v1) dst = linear interpolation of “a” and “b”

Gaussian Filtering

Kernel is Gaussian function

Filter width =2

Filtering Methods Comparison

Trade-offs Aliasing versus blurring Computation speed

Point Bilinear Gaussian

Image Warping Implementation

Reverse mappingfor (int x =0; x < xmax; x++) {

for (int y = 0; y < ymax; y++) {

float u = fx-1 (x, y);

float v = fy-1 (x, y);

dst (x, y) = resample_src(u, v, w);

}

}

Example: Scale

scale (src, dst, sx, sy) :Float w = max (1.0/sx, 1.0/sy)

for (int x =0; x < xmax; x++) {

for (int y = 0; y < ymax; y++) {

float u = x / sx;

float v = y / sy;

dst (x, y) = resample_src(u, v, w);

}

}

Example: Rotate

Rotate (src, dst, theta) :for (int x =0; x < xmax; x++) {

for (int y = 0; y < ymax; y++) {

float u = x*cos(-theta)y*sin(-theta);

float v = x*sin(-theta)y*cos(-theta);

dst (x, y) = resample_src(u, v, w);

}

}

Example: Fun

Swirl (src, dst, theta) :for (int x =0; x < xmax; x++) {

for (int y = 0; y < ymax; y++) {

float u = rot(dist(x, xcenter)*theta);

float v = rot(dist(y, yecnter)*theta);

dst (x, y) = resample_src(u, v, w);

}

}

Combining images

Image compositing Blue-screen mattes Alpha channel

Image morphing Specifying correspondence Warping Blending

Image Compositing

Separate an image into “elements” Render independently Composite together

Application Cel animation Chroma-keying Blue-screen matting

Blue Screen Matting

Composite foreground and background images Create background image Create foreground image with blue background Insert non-blue foreground pixels into background

Problem : no partial coverage

Alpha Channel

Encodes pixel coverage informations = 0 no coverage (or transparent) = 1 full coverage (or opaque) 0 < < 1 partial coverage (or semi-transparent)

Example: = 0.3

Composite with Alpha

Composite with Alpha Control the linear interpolation of foreground and

background pixels when elements are composited

Pixels with Alpha

Alpha channel convention (r, g, b, a) represents a pixel that is a covered by

the color C = (r/a, g/a, b/a)• Color components are premultiplied by a

• Can display (r, g, b) value directly

• Closure in composition algebra

Semi-Transparent Objects

Suppose we put A over B background G

How much of B is blocked by A ? A

How much of B show through A ?A)

How much of G shows through both A and B ?A) B)

αAA+ (1-αA )αBB+ (1-αA )(1-αB )G = A over[B over G]

Opaque Objects

How do we combine 2 partially covered pixels? 3 possible colors (0, A, B) 4 regions (0, A, B, AB)

Composition Algebra

12 reasonable combinations

Image Morphing

Animate transition between two images

Cross-Dissolving

Blend images with “over” operator alpha of bottom image is 1.0 alpha of top image varies from 0.0 to 1.0

blend(i, j) = (1-t) src(i, j) + t dst(i, j) (0 <= t <= 1)

Image Morphing

Combines warping and cross-disolving

Feature Based Morphing

Beier & Neeley use pairs of lines to specify warp Given p in dst image, where is p’ in source image ?

Feature Based Morphing

Feature Matching

Image Warping Image Warping

Morphed Image

Feature Based Morphing

Morphing

Warping Warping

Black Or White-Michael Jackson

Image Processing

Quantization Uniform Quantization Halftoning & Dithering

Pixel Operation Add random noise Add luminance Add contrast Add saturation

Filtering Blur Detected edges

Warping Scale Rotate Warp

Combining Composite Morph

Ref] Thomas Funkhouser, Princeton Univ.