Image Processing Basics

What are images?

An image is a 2-d rectilinear array of pixels

Pixels as samples

A pixel is a sample of a continuous function

Images are Ubiquitous

Input Optical photoreceptors Digital camera CCD array Rays in virtual camera (why?)

Output TVs Computer monitors Printers

Properties of Images

Spatial resolution Width pixels/width cm and height pixels/ height cm

Intensity resolution Intensity bits/intensity range (per channel)

Number of channels RGB is 3 channels, grayscale is one channel

Image errors

Spatial aliasing Not enough spatial resolution

Intensity quantization Not enough intensity resolution

Two issues

Sampling and reconstruction Creating and displaying images while reducing

spatial aliasing errors

Halftoning techniques Dealing with intensity quantization

Sampling and reconstruction

Aliasing

Artifacts caused by too low sampling frequency (undersampling) or improper reconstruction

Undersampling rate determined by Nyquist limit (Shannon’s sampling theorem)

Aliasing in computer graphics

In graphics, two major types Spatial aliasing

Problems in individual images Temporal aliasing

Problems in image sequences (motion)

Spatial Aliasing

“Jaggies”

Spatial aliasing

Ref: SIGGRAPH aliasing tutorial

Spatial aliasing

Texture disintegration

Ref: SIGGRAPH aliasing tutorial

Temporal aliasing

Strobing Stagecoach wheels in movies

Flickering Monitor refresh too slow Frame update rate too slow CRTs seen on other video screens

Antialiasing

Sample at a higher rate What if the signal isn’t bandlimited? What if we can’t do this, say because the sampling

device has a fixed resolution?

Pre-filter to form bandlimited signal Low pass filter Trades aliasing for blurring

Non-uniform sampling Not always possible, done by your visual system,

suitable for ray tracing Trades aliasing for noise

Sampling Theory

Two issues What sampling rate suffices to allow a given

continuous signal to be reconstructed from a discrete sample without loss of information?

What signals can be reconstructed without loss for a given sampling rate?

Spectral Analysis

Spatial (time) domain: Frequency domain:

Any (spatial, time) domain signal (function) can be written as a sum of periodic functions (Fourier)

Fourier Transform

Fourier Transform

Fourier transform:

Inverse Fourier transform:

dxexfuF xui

2)()(

dueuFxf xui

2)()(

Sampling theorem

A signal can be reconstructed from its samples if the signal contains no frequencies above ½ the sampling frequency.

-Claude Shannon

The minimum sampling rate for a bandlimited signal is called the Nyquist rate

A signal is bandlimited if all frequencies above a given finite bound have 0 coefficients, i.e. it contains no frequencies above this bound.

Filtering and convolution

Convolution of two functions (= filtering):

Convolution theorem: Convolution in the frequency domain is the same as

multiplication in the spatial (time) domain, and Convolution in the spatial (time) domain is the same as

multiplication in the frequency domain

dxhfxhxfxg

)()()()()(

Filtering, sampling and image processing

Many image processing operations basically involve filtering and resampling. Blurring Edge detection Scaling Rotation Warping

Resampling

Consider reducing the image resolution:

Resampling

The problem is to resample the image in such a way as to produce a new image, with a lower resolution, without introducing aliasing.

Strategy- Low pass filter transformed image by

convolution to form bandlimited signal This will blur the image, but avoid aliasing

Ideal low pass filter

Frequency domain:

Spatial (time) domain:

x

xxsync

)sin(

)(

Image processing in practice

Use finite, discrete filters instead of infinite continous filters

Convolution is a summation of a finite number of terms rather than in integral over an infinite domain

A filter can now be represented as an array of discrete terms (the kernel)

n

nxhfxhxfxg

)()()()()(

Discrete Convolution

Finite low pass filters

Triangle filter

Finite low pass filters

Gaussian filter

Edge Detection

Convolve image with a filter that finds differences between neighboring pixels

111

181

111

filter

Scaling

Resample with a gaussian or triangle filter

Image processing

Some other filters

Summary

Images are discrete objects Pixels are samples Images have limited resolution

Sampling and reconstruction Reduce visual artifacts caused by aliasing Filter to avoid undersampling Blurring (and noise) are preferable to aliasing