close range photogrammetry
TRANSCRIPT
Close Range Photogrammetry
Saju John Mathew EE 5358
Monday, 24th March 2008
University of Texas at Arlington
Monday, March 24, 2008 EE 5358 Computer Vision 1
Overview
• Definitions• Equipment• Mathematical Explanations• Working• Applications
Monday, March 24, 2008 EE 5358 Computer Vision 2
Close Range Photogrammetry(CRP)
Photogrammetry is a measurement technique where the coordinates of the points in 3D of an object are calculated by the measurements made in two photographic images(or more) taken starting from different positions.
CRP is generally used in conjunction with object to camera distances of not more than 300 meters (984 feet).
Monday, March 24, 2008 EE 5358 Computer Vision 3
Vertical Aerial Photographs
Monday, March 24, 2008 EE 5358 Computer Vision 4
University of Texas at Arlington at approx. 30 meters
University of Texas at Arlington at approx. 200 meters
CRP
Monday, March 24, 2008 EE 5358 Computer Vision 5
Acquisition of Data: Camera
Cameras can be broadly classified into two:
• Metric• Single Cameras• Stereometric Cameras
• Non-metric
Monday, March 24, 2008 EE 5358 Computer Vision 6
Photogrammetric Camera that enables geometrically accurate reconstruction of the optical model
of the object scene from its stereo photographs
Single Cameras • Total depth of field • Photographic material• Nominal focal length• Format of photographic material• Tilt range of camera axis and number of intermediate
stops
Monday, March 24, 2008 EE 5358 Computer Vision 7
Metric Cameras
Metric Cameras (contd.)
Stereometric Cameras• Base Length• Nominal Focal Length• Operational Range• Photographic Material• Format of photographic material• Tilt range of optical axes and
number of intermediate tilt stops
Monday, March 24, 2008 EE 5358 Computer Vision 8
Non-metric Cameras
Cameras that have not been designed especially for photogrammetric purposes:
• A camera whose interior orientation
is completely or partially unknown
and frequently unstable.
Monday, March 24, 2008 EE 5358 Computer Vision 9
Non-metric Cameras
Advantages• General availability• Flexibility in focusing range• Price is considerably less than for metric cameras• Can be hand-held and thereby oriented in any direction
Disadvantages• Lenses are designed for high resolution at the expense of high
distortion• Instability of interior orientation (changes after every exposure)• Lack of fiducial marks• Absence of level bubbles and orientation provisions precludes
the determination of exterior orientation before exposure
Monday, March 24, 2008 EE 5358 Computer Vision 10
Data Reduction
• Analog 1900 to 1960
• Analytical 1960 onwards
• Semi-analytical
• Digital 1980 onwards
Monday, March 24, 2008 EE 5358 Computer Vision 11
Analytical Photogrammetry
Based on camera parameters, measured photo coordinates and ground control
Accounts for any tilts that exist in photos
Solves complex systems of redundant equations by implementing least squares method
Monday, March 24, 2008 EE 5358 Computer Vision 12
ReviewCollinearity Condition
The exposure station
of a photograph, an object
point and its photo image
all lie along a straight
line.
Monday, March 24, 2008 EE 5358 Computer Vision 13
AZa
Xa
YaXL
YL
ZL
X
Z
Y
Ox
yL
aTilted photo
plane
f xa
ya
Image Coordinate System
• Ground Coordinate System - X, Y, Z
wrt Ground Coordinate System
• Exposure Station Coordinates – XL, YL, ZL
• Object Point (A) Coordinates – Xa, Ya, Za
• Rotated coordinate system parallel to ground
coordinate system (XYZ) – x’, y’, z’
wrt Rotated Coordinate System Rotated image coordinates – xa’, ya’, za’
xa’ , ya’ and za’ are related to the measured
photo coordinates xa, ya, focal length (f) and the
three rotation angles omega, phi and kappa.
Monday, March 24, 2008 EE 5358 Computer Vision 14
Z
Y
X
A
Za
Ya
Xa
YL
XL
ZL
x’
z’y’
L
za’
ya’
Xa’
Rotation Formulas
Developed in a sequence of three independent two-dimensional rotations.• ω rotation about x’ axis
x1 = x’
y1 = y’Cosω + z’Sinω
z1 = -y’Sinω + z’Cosω
• φ rotation about y’ axis
x2 = -z1Sinφ + x1Cosφ
y2 = y1
z2 = z1Cosφ + x1Sinφ
• κ rotation about z’ axis
x = x2Cosκ + y2Sin κ
y = -x2Sinκ + y2Cosκ
z = z2
Monday, March 24, 2008 EE 5358 Computer Vision 15
Rotation Matrix
X = MX’
Rotation Matrix
The sum of the squares of the three “direction cosines” in any row or in any
column is unity. M -1 = MT
X’ = MTX
Monday, March 24, 2008 EE 5358 Computer Vision 16
x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’ z = m31x’ + m32y’ + m33z’
x = x’(CosφCosκ) + y’(SinωSinφCosκ + CosωSinκ) + z’(-CosωSinφCosκ + SinωSinκ) y = x’(-CosφSinκ) + y’(-SinωSinφSinκ + CosωCosκ) + z’(CosωSinφSinκ + SinωCosκ ) z = x’(Sinφ) + y’(-SinωCosφ) + z’(CosωCosφ)
'
'
'
333231
232221
131211
z
y
x
mmm
mmm
mmm
z
y
x
Collinearity Condition Equations
Collinearity condition equations developed from similar triangles
* Dividing xa and ya by za
* Substitute –f for za
* Correcting the offset of Principal
point (xo, yo)
Monday, March 24, 2008 EE 5358 Computer Vision 17
AL
a
LA
a
LA
a
ZZ
z
YY
y
XX
x
''' x = m11x’ + m12y’ + m13z’ y = m21x’ + m22y’ + m23z’ z = m31x’ + m32y’ + m33z’
aLA
LAa
LA
LAa
LA
LAa
aLA
LAa
LA
LAa
LA
LAa
aLA
LAa
LA
LAa
LA
LAa
zZZ
ZZmz
ZZ
YYmz
ZZ
XXmz
zZZ
ZZmz
ZZ
YYmz
ZZ
XXmy
zZZ
ZZmz
ZZ
YYmz
ZZ
XXmx
'''
'''
'''
333231
232221
131211
)()()(
)()()(
)()()(
)()()(
333231
232221
333231
131211
LALALA
LALALAoa
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXmfyy
ZZmYYmXXm
ZZmYYmXXmfxx
Collinearity Equations
• Nonlinear• Nine unknowns
• ω, φ, κ
• XA, YA and ZA
• XL, YL and ZL
Taylor’s Theorem is used to linearize the nonlinear equations
substituting
Monday, March 24, 2008 EE 5358 Computer Vision 18
)()()(
)()()(
)()()(
)()()(
333231
232221
333231
131211
LALALA
LALALAoa
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXmfyy
ZZmYYmXXm
ZZmYYmXXmfxx
0
)(
)(!
)(
n
nn
axn
af
)()()(
)()()(
)()()(
232221
131211
333231
LALALA
LALALA
LALALA
ZZmYYmXXms
ZZmYYmXXmr
ZZmYYmXXmq
Linearizing Collinearity Equations
Rewriting the Collinearity Equations
• F0 and G0 are functions F and G evaluated at the initial approximations for the nine unknowns
• dω, dφ, dκ are the unknown corrections to be applied to the initial approximations
• The rest of the terms are the partial derivatives of F and G wrt to their respective unknowns at the initial approximations
Monday, March 24, 2008 EE 5358 Computer Vision 19
ao
ao
yq
sfyG
xq
rfxF
aAA
AA
AA
LL
LL
LL
o
aAA
AA
AA
LL
LL
LL
o
ydZZ
GdY
Y
GdX
X
G
dZZ
GdY
Y
GdX
X
Gd
Gd
Gd
GG
xdZZ
FdY
Y
FdX
X
F
dZZ
FdY
Y
FdX
X
Fd
Fd
Fd
FF
000
000000
000
000000
Taylor’s Theorem
Applying LLSM to Collinearity Equations
Monday, March 24, 2008 EE 5358 Computer Vision 20
• Residual terms must be included in order to make the equations consistent
J = xa – Fo ; K = ya – Go
b terms are coefficients equal to the partial derivatives
Numerical values for these coefficient terms are obtained by using initial approximations for the unknowns.
The terms must be solved iteratively (computed corrections are added to the initial approximations to obtain revised approximations) until the magnitudes of corrections to initial approximations become negligible.
yaAAALLL
xaAAALLL
VKdZbdYbdXbdZbdYbdXbdbdbdb
VJdZbdYbdXbdZbdYbdXbdbdbdb
262524262524232221
161514161514131211
Analytical Stereomodel
Mathematical calculation of three-dimensional ground coordinates of points in the stereomodel by analytical photogrammetric techniques
Three steps involved in forming an Analytical Stereomodel:
• Interior Orientation• Relative Orientation• Absolute Orientation
Monday, March 24, 2008 EE 5358 Computer Vision 21
Analytical Interior Orientation
Requires camera calibration information and quantification of the effects of atmospheric refraction.
2D coordinate transformation is used to relate the comparator coordinates to the fiducial coordinate system to correct film distortion.
Lens distortion and principal-point information from camera calibration are used to refine the coordinates so that they are correctly related to the principal point and free from lens distortion.
Atmospheric refraction corrections are applied.
Monday, March 24, 2008 EE 5358 Computer Vision 22
Analytical Relative Orientation
Process of determining the elements of exterior orientation
Fix the exterior orientation elements of the left photo of the stereopair to zero values
Common method in use to find these elements is through Space Resection by Collinearity(see slide below)
Each object point in the stereomodel contributes 4 equations
5 unknown orientation elements + 3 unknowns(X, Y & Z)
Monday, March 24, 2008 EE 5358 Computer Vision 23
Space Resection by Collinearity
Formulate the collinearity equations for a number of control points whose X, Y and Z ground coordinates are known and whose images appear in the tilted photo. The equations are then solved for the six unknown elements of exterior orientation which appear in them.
Space Resection collinearity equations for a point A
A two dimensional conformal coordinate transformation is used
X = ax’ – by’ + Tx X, Y – ground control coordinates for the point
Y = ay’ + bx’ + Ty x’, y’ – ground coordinates from a vertical photograph
a, b, Tx, Ty – transformation parameters
Monday, March 24, 2008 EE 5358 Computer Vision 24
yaLLL
xaLLL
VKdZbdYbdXbdbdbdb
VJdZbdYbdXbdbdbdb
262524232221
161514131211
Analytical Absolute Orientation
Utilizes a 3D conformal coordinate transformationRequires a min. of 2 horizontal and 3 vertical control
pointsStereomodel coordinates of control points are related to
their 3D coordinates in a cartesian coordinate systemCoordinates of all stereomodel points in the ground
system can be computed by applying the transformation parameters
Monday, March 24, 2008 EE 5358 Computer Vision 25
Bundle Adjustment
Adjust all photogrammetric measurements to ground
control values in a single solution
Unknown quantities • X, Y and Z object space coordinates of all object points• Exterior orientation parameters of all photographs
Measurements • x and y photo coordinates of images of object points• X, Y and/or Z coordinates of ground control points• Direct observations of the exterior orientation parameters of the photographs
Monday, March 24, 2008 EE 5358 Computer Vision 26
Bundle Adjustment-Observations
Photo Coordinates - Fundamental Photogrammetric Measurements made with a comparator or analytical plotter. According to accuracy and precision the coordinates are weighed
Control Points – determined through field survey Exterior Orientation Parameters – especially helpful in understanding the angular
attitude of a photograph Regardless of whether exterior orientation parameters were observed, a least squares
solution is possible since the number of observations is always greater than the number of unknowns.
xij, yij – measured photo coordinates of the image
of point j on photo i related to fiducial axis system
xo, yo – coordinates of principal points in fiducial axis
system
f - focal length/principal distance X j, Yj, Zj – coordinates of point j in object space
m11i, m12i, …….,m33i – rotation parameters for photo i XLi, YLi, ZLi – coordinates of incident nodal point
of camera lens in object space
Monday, March 24, 2008 EE 5358 Computer Vision 27
)()()(
)()()(
)()()(
)()()(
333231
232221
333231
131211
LijiLijiLiji
LijiLijiLijioij
LijiLijiLiji
LijiLijiLijioij
ZZmYYmXXm
ZZmYYmXXmfyy
ZZmYYmXXm
ZZmYYmXXmfxx
Bundle Adjustment - Weights
Photo coordinates σ0
2 – reference variance
σxij2 , σyij
2 – variances in xij and yij resp.
σxijyij = σyijxij – covariance of xij and yij
• Ground Control coordinates• σXj
2, σYj2, σZj
2 – variances in Xj00, Yj
00, Zj00 resp.
• σXjYj = σYjXj – covariance of Xj00 with Yj
00
• σXjZj = σZjXj – covariance of Xj00 with Zj
00
• σYjZj = σZjYj – covariance of Yj00 with Zj
00
• Exterior Orientation Parameters
Monday, March 24, 2008 EE 5358 Computer Vision 28
1
2
2
2
ijijij
ijijij
yxy
yxx
oijW
2
2
10
01
ij
ij
y
x
ijW
1
2
2
2
20
..
jjjjj
jjjjj
jjjjj
ZYZXZ
ZYYXY
ZXYXX
W
1
2
2
2
2
2
2
.
iLiLiLiLiLiiLiiLiiL
iLiLiLiLiLiiLiiLiiL
iLiLiLiLiLiiLiiLiiL
iLiiLiiLiiiiii
iLiiLiiLiiiiii
iLiiLiiLiiiiii
ZYZXZZZZ
ZYYXYYYY
ZXYXXXXX
ZYX
ZYX
ZYX
iW
Direct Linear Transformation (DLT)
This method does not require fiducial marks and can be solved without supplying initial approximations for the parameters
Collinearity equations along with the correction for lens distortion
δx, δy – lens distortion
fx – pd in the x direction
fy – pd in the y direction
Rearranging the above two equations
Monday, March 24, 2008 EE 5358 Computer Vision 29
)()()(
)()()(
)()()(
)()()(
333231
2322210
333231
1312110
LALALA
LALALAyya
LALALA
LALALAxxa
ZZmYYmXXm
ZZmYYmXXmfxy
ZZmYYmXXm
ZZmYYmXXmfxx
1
1
11109
8765
11109
4321
ZLYLXL
LZLYLXLy
ZLYLXL
LZLYLXLx
ya
xa
LmfmyL
LmfmyL
LmfmyL
LZmYmXmfxL
LmfmxL
LmfmxL
LmfmxL
y
y
y
LLLx
x
x
x
/)(
/)(
/)(
/)(
/)(
/)(
/)(
233307
223206
213105
13121104
133303
123202
113101
DLT(contd.)
The resulting equations are solved iteratively using LSM
Advantages - No initial approximations are required for the unknowns. Limitations - Requirement of atleast six 3D object space control points - Lower accuracy of the solution as compared with a rigorous bundle adjustment
Monday, March 24, 2008 EE 5358 Computer Vision 30
)(
/
/
/
/)(
333231
3311
3210
319
23222108
LLL
LLLy
ZmYmXmL
LmL
LmL
LmL
LZmYmXmfyL
18
4
1
11109
765
321
L
L
LLL
LLL
LLL
Z
Y
X
L
L
L
Analytical Self Calibration
The equations take into account adjustment of the calibrated focal length, principal-point offsets and symmetric radial and decentering lens distortion.
xa, ya – measured photo coordinates related to fiducials
xo, yo – coordinates of the principal point
= xa – xo where
= ya - yo
Monday, March 24, 2008 EE 5358 Computer Vision 31
q
sfyxpyxprprkrkrkyyy
q
rfyxpyxprprkrkrkxxx
aaaaaaaaaa
aaaaaaaaaa
]2)3()[1()(
]2)3()[1()(
122
222
36
34
22
10
222
122
36
34
22
10
axay
)()()(
)()()(
)()()(
232221
131211
333231
LALALA
LALALA
LALALA
ZZmYYmXXms
ZZmYYmXXmr
ZZmYYmXXmq
Analytical Self Calibration(contd.)
k1, k2, k3 = symmetric radial lens distortion coefficients
p1, p2, p3 = decentering distortion coefficients
f = calibrated focal length
r, s, q = collinearity equation terms
Provides a calibration of the camera under original conditions which existed when the photographs were taken.
Geometric Requirements
- Numerous redundant photographs from multiple locations are required, with sufficient roll diversity
- Many well-distributed image points be measured over the entire format to determine lens distortion parameters
The numerical stability of analytical self calibration is of serious concern.
Monday, March 24, 2008 EE 5358 Computer Vision 32
222aaa yxr
Applications
Automobile ConstructionMachine Construction, Metalworking, Quality ControlMining EngineeringObjects in MotionShipbuildingStructures and BuildingsTraffic EngineeringBiostereometrics
Monday, March 24, 2008 EE 5358 Computer Vision 33
Biomedical Applications
Linear tape and caliper measurements of inherently irregular three-dimensional biological structures are inadequate for many purposes.
Subtle movements produced by breathing, pulsation of blood, and reflex correction for control of postural stability.
Short patient involvement times, avoids contact with the patient and thereby avoiding risk of deforming the area of interest and spreading infection.
All medical photogrammetric measurements require further interpretation and analysis to allow meaningful information to be given to the end-user.
Monday, March 24, 2008 EE 5358 Computer Vision 34
Bibliography
• Kamara, H.M. (1979). Handbook of Non-Topographic Photogrammetry, American Society of Photogrammetry.
• Wolf , Paul R., Dewitt, Bon A. (2000). Elements of Photogrammetry, McGraw Hill.
• Devarajan, Venkat and Chauhan, Kriti (Spring 2008). Lecture Notes: Mathematical Foundation of Photogrammetry, EE 5358 University of Texas at Arlington.
• Karara, H.M. (1989). Non-Topographic Photogrammetry, American Society for Photogrammetry and Remote Sensing.
• Mitchell, H.L. and Newton, I. (2002). Medical photogrammetric measurement: overview and prospects. ISPRS Journal of Photogrammetry & Remote Sensing, 56, 286-294.
Monday, March 24, 2008 EE 5358 Computer Vision 35
Acknowledgments
Dr. Venkat Devarajan
Kriti Chauhan
Monday, March 24, 2008 EE 5358 Computer Vision 36