image formation fundamentals basic concepts (continued…)

Image Formation Fundamentals

Basic Concepts (Continued…)

How are images represented in the computer?

courstey UNR computer vision course

Image digitization

• Sampling means measuring the value of an image at a finite number of points.• Quantization is the representation of the measured value at the sampled point by an

integer.

courstey UNR computer vision course

Image digitization (cont’d)

courstey UNR computer vision course

Image quantization (Example)

256 gray levels (8bits/pixel) 32 gray levels (5 bits/pixel) 16 gray levels (4 bits/pixel)

8 gray levels (3 bits/pixel) 4 gray levels (2 bits/pixel) 2 gray levels (1 bit/pixel)

courstey UNR computer vision course

Image sampling (example)

original image sampled by a factor of 2

sampled by a factor of 4 sampled by a factor of 8

courstey UNR computer vision course

Digital image

• An image is represented by a rectangular array of integers.• An integer represents the brightness or darkness of the image at that

point.• N: # of rows, M: # of columns, Q: # of gray levels

– N = , M = , Q = (q is the # of bits/pixel)– Storage requirements: NxMxQ (e.g., N=M=1024, q=8, 1MB)

(0,0)(0,1)...(0,1)(1,0)(1,1)...(1,1)............(1,0)(1,1)...(1,1)fffMfffMfNfNfNM−−−−−−

2n 2m 2q

courstey UNR computer vision course

Image formation

• There are two parts to the image formation process:– The geometry of image formation, which

determines where in the image plane the projection of a point in the scene will be located.

– The physics of light, which determines the brightness of a point in the image plane as a function of illumination and surface properties.

courstey UNR computer vision course

A Simple model of image formation

• The scene is illuminated by a single source.

• The scene reflects radiation towards the camera.

• The camera senses it via chemicals on film.

courstey UNR computer vision course

Pinhole cameras

• Abstract camera model - box with a small hole in it

• Pinhole cameras work in practice

courstey Dr. G. D. Hager

Real Pinhole Cameras

Pinhole too big - many directions are averaged, blurring the image

Pinhole too small- diffraction effects blur the image

Generally, pinhole cameras are dark, becausea very small set of raysfrom a particular pointhits the screen.

courstey Dr. G. D. Hager

The reason for lenses

Lenses gather andfocus light, allowingfor brighter images.

courstey Dr. G. D. Hager

The thin lens

1z'

−1z

=1f

Thin Lens Properties:1. A ray entering parallel to optical axis

goes through the focal point.2. A ray emerging from focal point is parallel

to optical axis3. A ray through the optical center is unaltered

courstey Dr. G. D. Hager

The thin lens

1z'

−1z

=1f

Note that, if the image plane is verysmall and/or z >> z’, then z’ is approximately equal to f

courstey Dr. G. D. Hager

Lens Realities

Real lenses have a finite depth of field, and usuallysuffer from a variety of defects

vignetting

Spherical Aberration

courstey Dr. G. D. Hager

The equation of projection

• Equating z’ and f– We have, by similar triangles,

that (x, y, z) -> (-f x/z, -f y/z, -f)– Ignore the third coordinate, and

flip the image around to get:

(x,y,z)→ ( fxz, fyz)

courstey Dr. G. D. Hager

Distant objects are smaller

courstey Dr. G. D. Hager

Parallel lines meet

common to draw film planein front of the focal point

A Good Exercise: Show this is the case!

courstey Dr. G. D. Hager

Orthographic projection

yv

xu

==

Suppose I let f go to infinity; then

courstey Dr. G. D. Hager

The model for orthographic projection

U

V

W

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟ =

1 0 0 0

0 1 0 0

0 0 0 1

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟

X

Y

Z

T

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

courstey Dr. G. D. Hager

Weak perspective

• Issue– perspective effects, but not over

the scale of individual objects– collect points into a group at

about the same depth, then divide each point by the depth of its group

– Adv: easy– Disadv: wrong

*/ Zfs

syv

sxu

===

courstey Dr. G. D. Hager

The model for weak perspective projection

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

T

Z

Y

X

fZW

V

U

/*000

0010

0001

courstey Dr. G. D. Hager

Model for perspective projection

U

V

W

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟ =

1 0 0 0

0 1 0 0

0 0 1f 0

⎛

⎝

⎜ ⎜

⎞

⎠

⎟ ⎟

X

Y

Z

T

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

courstey Dr. G. D. Hager

Intrinsic Parameters

Intrinsic Parameters describe the conversion fromunit focal length metric to pixel coordinates (and the reverse)

pK

w

y

x

os

os

w

y

x

mm

yy

xx

pix

int

100

/10

0/1

=⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛−

−=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

It is common to combine scale and focal length togetheras the are both scaling factors; note projection is unitless in this case!

courstey Dr. G. D. Hager

Image formation - Recap

Taken from MASKS (invitation to 3D vision)

world coordinate system

camera coordinate system

(R,T)

pixel coordinate system

image coordinate system

If we consider unit focal length

Scaling factor = depth of the point X

x1

Camera parameters

• Summary:– points expressed in external frame– points are converted to canonical camera coordinates– points are projected– points are converted to pixel units

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

T

Z

Y

X

W

V

U

parameters extrinsic

ngrepresenti

tionTransforma

model projection

ngrepresenti

tionTransforma

parameters intrinsic

ngrepresenti

tionTransforma

point in cam. coords.

point in metricimage coords.

point in pixelcoords.

point in world coords.courstey Dr. G. D. Hager

Camera Calibration

The problem:Compute the camera intrinsic and extrinsic

parameters using only observed camera data.

Calibration with a Rig

Use the fact that both 3-D and 2-D coordinates of feature points on a pre-fabricated object (e.g., a cube) are known.

Calibration with Multiple Plane Images

Actually used in practice these days

Calibration Continued…