ele 488 f06 ele 488 fall 2006 image processing and transmission (10-17-06) geometric transformation...
TRANSCRIPT
ELE 488 F06
ELE 488 Fall 2006Image Processing and Transmission
(10-17-06)
Geometric Transformation
translation rotation scaling Affine
Image Registration
Goodness of fit
10/17/06
ELE 488 F06
Mosaicking and Registration
Reference Image Floating Image
ELE 488 F06
Identify Features and Matchto Determine Geometric Transformation
Two cartesian coordinate systems: (x,y) and (x’,y’)
Forward: (x’,y’) = h(x,y) Reverse: (x,y) = h-1(x’,y’)
h(x,y)
h–1(x’,y’)
Image I (reference) Image II (floating)
ELE 488 F06
Four Basic Transformations
y
x
t
t
y
x
10
01
'y
'x
'y
'x=
)cos()sin(
)sin()cos(
y
x
y
x
s0
0s
'y
'x
y
x
y
x
10
1
'y
'x x
y
x
1
01
'y
'x
y
translation
rotation
scaling
shearing
(1, 1)
y
x
(1, 0)
y’
x’
(2, 1) (3, 1)
ELE 488 F06
Combining Transformations
• Successive application of rotation, translation and scaling.
w’ = M2 ( M1 w + t1) + t2 = M2 M1 w + (M2t1 + t2) = M w + t
• Involves matrix multiplications and additions• Homogeneous Coordinates
• Rotation and translation
Matrix multiplication, no addition
ELE 488 F06
Translation – Rotation – Scaling
Translation:
Rotation:
Scaling:
Note: c=0
ELE 488 F06
Combining Transformations
1) Translate2) Rotate3) Translate
1
wHHH
1
'w123 =
1
w
10
tMtM 31
1
w
dc
bAT
ELE 488 F06
• Rigid-body: Rotation and Translation– 2x2 rotation submatrix is orthonormal
• Translation/Rotation/Scaling – are determined by rotation and scaling parameters
Homogeneous Transforms
ELE 488 F06
Affine and Projective Transformations
• Affine: 6 free parameters
• Projective: 8 free parameters– More general geometric transformations between two planes– Widely used in modeling cameras
s scaling factor.
ELE 488 F06
Inverse Transform
Examples:
ELE 488 F06
Reflections
Reflect in x-axis
Reflect in y-axis
Reflect in line y=x
ELE 488 F06
Reflect about a general line y=ax+b
ELE 488 F06
Pinhole Camera Model
ELE 488 F06
Pinhole Camera Model (shifted image plane)
ELE 488 F06
Image Formation: pin hole model
P=(X,Y,Z)
Z
X
Y
p=(x,y)
f
Camera Coordinates
Image Coordinates
Projection of point (X,Y,Z) in 3D space to (x,y) in image plane
ELE 488 F06
Mosaicking and Registration
Reference Image Floating Image
Two camerasOr one camera at two positions
ELE 488 F06
Two Cameras
P=(X’,Y’,Z’)
Z
X
Y
p=(x,y)
P=(X,Y,Z)
Z’
X’
Y’
p’=(x’,y’)
Rotation + Translation + Zoom
f f’
Point in 3-D has two sets of coordinates
ELE 488 F06
Rotation of Camera About One Axis
Z
Y
X
100
0cossin
0sincos
'Z
'Y
'X
ELE 488 F06
General Rotation of Camera
rotation matrix R
Two 3-D coordinates of a point are related by a rotation matrix:
Decompose R into 3 rotations about the first camera axes:
Counter clockwise rotation about Y axis
Counter clockwise rotation about X axis
Counter clockwise rotation about Z axis
ELE 488 F06
Translation
location of 2nd camera in 1st camera coordinates
ELE 488 F06
3-D Coordinate Transformation
The two 3-D coordinates of a point are related by a 3-D rotation and a translation:
Counter clockwise rotation about Y axis
Counter clockwise rotation about X axis
Counter clockwise rotation about Z axis
)f/Z(
f
y
x
Z
1
Z/Y
Z/X
Z
Y
X
from pinhole camera
ELE 488 F06
3-D Coordinate Transformation
)/(
1
/
/
fZ
f
y
x
ZZY
ZX
Z
Y
X
)'/'(
'
'
'
'
'
'
fZ
f
y
x
Z
Y
X
Z
f
t
t
t
f
y
x
rrr
rrr
rrr
s
f
y
x
Z
Y
X
333231
232221
131211
'
'
'
Z
Y
X
t
t
t
f
Z
f
y
x
rrr
rrr
rrr
f
Z
f
y
x
333231
232221
131211
'
'
'
'
'
ELE 488 F06
3-D Coordinate Transformation
Independently scale each side of the equation
ELE 488 F06
No Translation
Homogeneous transformation
Knowing establishes the correspondence between the images.
fy
x
f
f
f
y
x
1100
0/10
00/1
1100
0/10
00/1
100
0'0
00'
1
'
'
333231
232221
131211
y
x
f
f
rrr
rrr
rrr
f
f
sy
x
ELE 488 F06
Examples
ELE 488 F06
Flat Scene
Homogeneous transformation
Knowing establishes the correspondence between the images.
See slide 9
ELE 488 F06
Planar Scene
Homogeneous transformation
Knowing establishes the correspondence between the images.
k1 X + k2 Y + k3 Z = 1 k1 x + k2 y + k3 f = f / Z
ELE 488 F06
Mosaic: Firestone Library
Photos: P. Ramadge, Mosaic: Y.P. Tan/R. Radke
The zero translation approximation was used to register these images
ELE 488 F06
Affine and Projective Transformations
• Projective Transformations are closely connected to image formation models and camera rotation
• Affine is an approximation:
ELE 488 F06
Small Rotations
Can be approximated by affine
ELE 488 F06
Three Questions of Image Registration
1. How is the spatial transformation (= warping function) applied and what family of warping functions are allowed?
2. How do we measure how well the warped floating image aligns with the reference image?
3. How to we find the best warp in the allowed class to best match the images (optimization)
ELE 488 F06
Warping Function Examples
Translation
Rotation
Scaling
Affine
Projective
ELE 488 F06
Goodness of Fit: How well does warped image align with reference
x
y
1) Use all pixel values:
2) Only use pixel values around selected “features”:
What are features?
ELE 488 F06
Features and Feature Selection
Sharp corners
vu
GOODv
u
BAD
ELE 488 F06
Registration As An Optimization Problem
Consider transformations parameterized by a vector of parameters θ
Find the transformation that minimizes the registration metric (e.g., above)
In some special cases this can be solved analytically (in closed form)
In other cases it can be approximated and solved analytically
In the most complex cases it has to be solved iteratively by a gradient-like descent method.
ELE 488 F06
Example: Translation + SSD Feature Metric
1 feature - can do this by brute force search by finding the best match
More features?
ELE 488 F06
Simplification
Find the best translation for each feature individually by a brute force search (can be refined)
This gives (called a correspondence)
Then select the translation for the entire image to minimize :
This yields
ELE 488 F06
Image Registration
Reference Image Floating Image
• 756x504 pixels16 bit color• Allowed warpings is translations• Matching metric is how well a selected subset of “features” match using SAD• best translation match. average pixels where images overlap
ELE 488 F06
Registration Using Affine Transformation
translation, rotation, scaling, shearing
ELE 488 F06
Registration Using Projective Transformation
Add: tilting