Photogrammetric Computer Vision

Wolfgang FörstnerBernhard P. Wrobel

Statistics, Geometry, Orientation and Reconstruction

Geometry and Computing 11

Geometry and Computing

Volume 11

Series editors

Herbert Edelsbrunner, Department Computer Science, Durham, NC, USALeif Kobbelt, RWTH Aachen University, Aachen, GermanyKonrad Polthier, AG Mathematical Geometry Processing, Freie Universität Berlin,Berlin, Germany

Geometric shapes belong to our every-day life, and modeling and optimization of such formsdetermine biological and industrial success. Similar to the digital revolution in imageprocessing, which turned digital cameras and online video downloads into consumer products,nowadays we encounter a strong industrial need and scientific research on geometryprocessing technologies for 3D shapes.Several disciplines are involved, many with their origins in mathematics, revived withcomputational emphasis within computer science, and motivated by applications in thesciences and engineering. Just to mention one example, the renewed interest in discretedifferential geometry is motivated by the need for a theoretical foundation for geometryprocessing algorithms, which cannot be found in classical differential geometry.

Scope: This book series is devoted to new developments in geometry and computation andits applications. It provides a scientific resource library for education, research, and industry.The series constitutes a platform for publication of the latest research in mathematics andcomputer science on topics in this field.

• Discrete geometry• Computational geometry• Differential geometry• Discrete differential geometry• Computer graphics• Geometry processing• CAD/CAM• Computer-aided geometric design• Geometric topology• Computational topology• Statistical shape analysis• Structural molecular biology• Shape optimization• Geometric data structures• Geometric probability• Geometric constraint solving• Algebraic geometry• Graph theory• Physics-based modeling• Kinematics• Symbolic computation• Approximation theory• Scientific computing• Computer vision

More information about this series at http://www.springer.com/series/7580

Wolfgang Förstner • Bernhard P. Wrobel

PhotogrammetricComputer VisionStatistics, Geometry, Orientationand Reconstruction

123

Wolfgang FörstnerInstitut für Geodäsie und GeoinformationRheinische Friedrich-Wilhelms-UniversitätBonn

BonnGermany

Bernhard P. WrobelInstitut für GeodäsieTechnische Universität DarmstadtDarmstadtGermany

ISSN 1866-6795 ISSN 1866-6809 (electronic)Geometry and ComputingISBN 978-3-319-11549-8 ISBN 978-3-319-11550-4 (eBook)DOI 10.1007/978-3-319-11550-4

Library of Congress Control Number: 2016954546

© Springer International Publishing Switzerland 2016This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material isconcerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproductionon microfilms or in any other physical way, and transmission or information storage and retrieval, electronicadaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does notimply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws andregulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in this book are believedto be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty,express or implied, with respect to the material contained herein or for any errors or omissions that may have beenmade.

Printed on acid-free paper

This Springer imprint is published by Springer NatureThe registered company is Springer International Publishing AGThe registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This textbook on Photogrammetric Computer Vision – Statistics, Geometry, Orientationand Reconstruction provides a statistical treatment of the geometry of multiple view anal-ysis useful for camera calibration, orientation, and geometric scene reconstruction.

The book is the first to offer a joint view of photogrammetry and computer vision, twofields that have converged in recent decades. It is motivated by the need for a conceptuallyconsistent theory aiming at generic solutions for orientation and reconstruction problems.

Large parts of the book result from teaching bachelor’s and master’s courses for stu-dents of geodesy within their education in photogrammetry. Most of these courses weresimultaneously offered as subjects in the computer science faculty.

The book provides algorithms for various problems in geometric computation and invision metrology, together with mathematical justification and statistical analysis allowingthorough evaluation.

The book aims at enabling researchers, software developers, and practitioners in thephotogrammetric and GIS industry to design, write, and test their own algorithms andapplication software using statistically founded concepts to obtain optimal solutions andto realize self-diagnostics within algorithms. This is essential when applying vision tech-niques in practice. The material of the book can serve as a source for different levels ofundergraduate and graduate courses in photogrammetry, computer vision, and computergraphics, and for research and development in statistically based geometric computer vi-sion methods.

The sixteen chapters of the book are self-contained, are illustrated with numerous fig-ures, have exercises, and are supported by an appendix and an index. Many of the examplesand exercises can be verified or solved using the Matlab routines available on the homepage of the book, which also contains solutions to some of the exercises.

Acknowledgements: The book gained a lot through the significant support of numer-ous colleagues. We thank Camillo Ressl and Jochen Meidow for their careful reading ofthe manuscript and Carl Gerstenecker and Boris Kargoll for their critical review of PartI on statistics. The language proofreading by Silja Weber, Indiana University, is highlyappreciated. Thanks for fruitful comments, discussions and support of the accompanyingMatlab Software to Martin Drauschke, Susanne Wenzel, Falko Schindler, Thomas Läbe,Richard Steffen, Johannes Schneider, and Lutz Plümer. We thank the American Societyfor Photogrammetry and Remote Sensing for granting us permission to use material ofthe sixth edition of the ‘Manual of Photogrammetry’.

Wolfgang FörstnerBernhard P. Wrobel Bonn, 2016

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Tasks for Photogrammetric Computer Vision . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Modelling in Photogrammetric Computer Vision . . . . . . . . . . . . . . . . . . . . . . 61.3 The Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.4 On Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Part I Statistics and Estimation

2 Probability Theory and Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.1 Notions of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.2 Axiomatic Definition of Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.3 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.4 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.6 Quantiles of a Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.7 Functions of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.8 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482.9 Generating Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.1 Principles of Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613.2 Testability of an Alternative Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.3 Common Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.1 Estimation Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754.2 The Linear Gauss–Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 814.3 Gauss–Markov Model with Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.4 The Nonlinear Gauss–Markov Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1024.5 Datum or Gauge Definitions and Transformations . . . . . . . . . . . . . . . . . . . . . 1084.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1154.7 Robust Estimation and Outlier Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1414.8 Estimation with Implicit Functional Models . . . . . . . . . . . . . . . . . . . . . . . . . . 1604.9 Methods for Closed Form Estimations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1764.10 Estimation in Autoregressive Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1834.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

vii

viii Contents

Part II Geometry

5 Homogeneous Representations of Points, Lines and Planes . . . . . . . . . . . 1955.1 Homogeneous Vectors and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1955.2 Homogeneous Representations of Points and Lines in 2D . . . . . . . . . . . . . . . 2055.3 Homogeneous Representations in IPn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2095.4 Homogeneous Representations of 3D Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . 2165.5 On Plücker Coordinates for Points, Lines and Planes . . . . . . . . . . . . . . . . . . 2215.6 The Principle of Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2295.7 Conics and Quadrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2365.8 Normalizations of Homogeneous Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2415.9 Canonical Elements of Coordinate Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 2425.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

6 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2476.1 Structure of Projective Collineations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2486.2 Basic Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2506.3 Concatenation and Inversion of Transformations . . . . . . . . . . . . . . . . . . . . . . 2616.4 Invariants of Projective Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2666.5 Perspective Collineations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2776.6 Projective Correlations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2826.7 Hierarchy of Projective Transformations and Their Characteristics . . . . . . 2846.8 Normalizations of Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2856.9 Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2866.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

7 Geometric Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2917.1 Geometric Operations in 2D Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2927.2 Geometric Operations in 3D Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2997.3 Vector and Matrix Representations for Geometric Entities . . . . . . . . . . . . . . 3117.4 Minimal Solutions for Conics and Transformations . . . . . . . . . . . . . . . . . . . . 3167.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

8 Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3258.1 Rotations in 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3258.2 Concatenation of Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3378.3 Relations Between the Representations for Rotations . . . . . . . . . . . . . . . . . . 3388.4 Rotations from Corresponding Vector Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . 3398.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

9 Oriented Projective Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3439.1 Oriented Entities and Constructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3449.2 Transformation of Oriented Entities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3559.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

10 Reasoning with Uncertain Geometric Entities . . . . . . . . . . . . . . . . . . . . . . . 35910.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36010.2 Representing Uncertain Geometric Elements . . . . . . . . . . . . . . . . . . . . . . . . . . 36410.3 Propagation of the Uncertainty of Homogeneous Entities . . . . . . . . . . . . . . . 38610.4 Evaluating Statistically Uncertain Relations . . . . . . . . . . . . . . . . . . . . . . . . . . 39310.5 Closed Form Solutions for Estimating Geometric Entities . . . . . . . . . . . . . . 39510.6 Iterative Solutions for Maximum Likelihood Estimation . . . . . . . . . . . . . . . . 41410.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432

Contents ix

Part III Orientation and Reconstruction

11 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44111.1 Scene, Camera, and Image Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44111.2 The Setup of Orientation, Calibration, and Reconstruction . . . . . . . . . . . . . 44911.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

12 Geometry and Orientation of the Single Image . . . . . . . . . . . . . . . . . . . . . . 45512.1 Geometry of the Single Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45612.2 Orientation of the Single Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48912.3 Inverse Perspective and 3D Information from a Single Image . . . . . . . . . . . 52312.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537

13 Geometry and Orientation of the Image Pair . . . . . . . . . . . . . . . . . . . . . . . . 54713.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54713.2 The Geometry of the Image Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54913.3 Relative Orientation of the Image Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56813.4 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59613.5 Absolute Orientation and Spatial Similarity Transformation . . . . . . . . . . . . 60713.6 Orientation of the Image Pair and Its Quality . . . . . . . . . . . . . . . . . . . . . . . . 60813.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615

14 Geometry and Orientation of the Image Triplet . . . . . . . . . . . . . . . . . . . . . . 62114.1 Geometry of the Image Triplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62214.2 Relative Orientation of the Image Triplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63214.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641

15 Bundle Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64315.1 Motivation for Bundle Adjustment and Its Tasks . . . . . . . . . . . . . . . . . . . . . . 64415.2 Block Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64515.3 Sparsity of Matrices, Free Adjustment and Theoretical Precision . . . . . . . 65115.4 Self-calibrating Bundle Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67415.5 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69615.6 Outlier Detection and Approximate Values . . . . . . . . . . . . . . . . . . . . . . . . . . . 70715.7 View Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71515.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722

16 Surface Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72716.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72716.2 Parametric 21/2D Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73316.3 Models for Reconstructing One-Dimensional Surface Profiles . . . . . . . . . . . . 74216.4 Reconstruction of 21/2D Surfaces from 3D Point Clouds . . . . . . . . . . . . . . . 75716.5 Examples for Surface Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76316.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765

Appendix: Basics and Useful Relations from Linear Algebra . . . . . . . . . . . . . 767A.1 Inner Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767A.2 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767A.3 Inverse, Adjugate, and Cofactor Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769A.4 Skew Symmetric Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 770A.5 Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772A.6 Idempotent Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774A.7 Kronecker Product, vec(·) Operator, vech(·) Operator . . . . . . . . . . . . . . . . . 775

x Contents

A.8 Hadamard Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776A.9 Cholesky and QR Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776A.10 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777A.11 The Null Space and the Column Space of a Matrix . . . . . . . . . . . . . . . . . . . . 777A.12 The Pseudo-inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779A.13 Matrix Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 781A.14 Tensor Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 782A.15 Variance Propagation of Spectrally Normalized Matrix . . . . . . . . . . . . . . . . . 783

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 799

List of Algorithms

1 Estimation in the linear Gauss–Markov model . . . . . . . . . . . . . . . . . . . . . . . . . . 912 Estimation in the Gauss–Markov model with constraints . . . . . . . . . . . . . . . . . 1083 Random sample consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1564 Robust estimation in the Gauss–Helmert model with constraints . . . . . . . . . . 1685 Reweighting constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1696 Estimation in the model with constraints between the observations only . . . . 171

7 Algebraic solution for estimating 2D homography from point pairs . . . . . . . . . 3898 Direct LS estimation of 2D line from points with isotropic accuracy . . . . . . . . 4019 Direct LS estimation of a 2D point from lines with positional uncertainty . . . 40310 Direct LS estimation of the mean of directions with isotropic uncertainty. . . . 40411 Direct LS estimation of the mean of axes with isotropic uncertainty . . . . . . . . 40512 Direct LS estimation of a rotation from direction pairs . . . . . . . . . . . . . . . . . . . 40813 Direct LS estimation of similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41114 Direct LS estimation of 3D line from points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41415 Estimation in the Gauss–Helmert model with reduced coordinates . . . . . . . . . 416

16 Algebraic estimation of uncertain projection from six or more points . . . . . . . 49617 Optimal estimation of a projection matrix from observed image points . . . . . 49918 Decomposition of uncertain projection matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 50019 3D circle with given radius determined from its image . . . . . . . . . . . . . . . . . . . . 536

20 Base direction and rotation from essential matrix . . . . . . . . . . . . . . . . . . . . . . . . 58321 Optimal triangulation from two images and spherical camera model . . . . . . . . 600

22 Sequential spatial resections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70923 Sequential similarity transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710

xi

List of Symbols

Table 0.1 List of symbols: A – Msymbol meaningA , B , C names of planes, setsA, B, C homogeneous vectors of planesA0, Ah Euclidean, homogeneous part of the homogeneous coordinate vector A of plane AAX ,AY ,AZ homogeneous vectors of coordinate planes, perpendicular to the axes X, Y , and ZBd d-dimensional unit ball in IRd

Cov(., .) covariance operatorCR(., ., ., .) cross ratioD 6× 6 matrix dualizing a lineδ(x) Dirac’s delta functionDiag(.) diagonal matrix of vector or list of matricesdiag(.) vector of diagonal elements of a matrixdet(.) = |.| determinante[d]i ith basic unit vector in d-space, e.g., e[3]2 = [0, 1, 0]T

D(.) dispersion operatorE(.) expectation operatorI (L) Plücker matrix of a 3D lineI (L) dual Plücker matrix of a 3D lineI (s)(L) 2× 4 matrix of selected independent rowsIn n× n unit matrixJ = {1, ..., j, ..., J} set of indicesJxy , Jx,y Jacobian ∂x/∂yJr Jacobian ∂x/∂xr, with reduced vector xr of xJs Jacobian ∂xs/∂x of spherical normalizationHf or H(f) Hessian matrix [∂2f(x)/(∂xi∂xj)] of function f(x)H name of homographyH general homography, 2× 2, 3× 3, or 4× 4 matrixl vector of observations in an estimation procedurel , m , n names of 2D linesl, m, n homogeneous vectors of 2D linesL , M , N names of 3D linesL, M, N homogeneous vectors of 3D linesl0, lh Euclidean, homogeneous part of homogeneous coordinate vector l of 2D line lL0, Lh Euclidean, homogeneous part of homogeneous coordinate vector L of 3D line Llx, ly ,LX ,LY ,LZ line parameters of coordinate axesL coordinates of 3D line L dual to 3D line LM motion, special homography in 2D or 3DIN set of natural numbersM (µ,Σ) distribution characterized only by mean µ and covariance matrix Σ

xiii

Table 0.2 List of symbols: N – Zsymbol meaningN (µ,Σ) normal distribution with mean µ and covariance matrix ΣN normal equation matrixN(.) operator for achieving Frobenius norm 1, for vectors: spherical normalizationNe(.) operator for Euclidean normalization of homogeneous vectorsNσ(.) operator for spectral normalization of matricesnull(.), nullT(.) orthonormal matrix: basis vectors of null space as columns, transposeo origin of coordinate systemO(Z), o(z) coordinates of the centre of perspectivityIPn n-dimensional projective spaceIP∗n dual n-dimensional projective spaceI I (X), I I (A) Pi-matrix of a 3D point or a planeI I (X), I I (A) dual Pi-matrix of a 3D point or a planeI I (s)(X), I I (s)(A) 3× 4 matrix of selected independent rowsr(x|a, b) rectangle function in the range [a, b]rxy correlation coefficient of x and yR rotation matrix, correlation matrixIRn n-dimensional Euclidean space over IRIRn \ 0 n-dimensional Euclidean space without origins(x) step functionSL(n) special group of linear transformations with determinant 1SO(n) special group of orthogonal transformations (rotations)so(n) Lie group of skew matricesSa, S(a), Sa, S(a) inhomogeneous, homogeneous skew symmetric matrix depending on a 3-vectorSi 3× 3 skew symmetric matrix of 3× 1 vector e

[3]i

S(s)(x) 2× 3 matrix with two selected independent rowsSd unit sphere of dimension d in IRd+1, set of points x ∈ IRd+1 with |x| = 1σx standard deviation of xσxy covariance of x and yΣxy covariance matrix of x and yTn oriented projective spaceT∗n dual oriented projective spaceW xx weight matrix of parameters xx unknown parameters in an estimation procedurex , y , z names of 2D pointsx, y, z homogeneous vectors of points in 2DX , Y , Z names of 3D pointsX, Y, Z homogeneous vectors of points in 3Dx0, xh Euclidean, homogeneous part of the homogeneous coordinate vector x of point xX0, Xh Euclidean, homogeneous part of the homogeneous coordinate vector X of point X

Table 0.3 List of symbols: fonts, operatorssymbol meaning%" permillex

n×1, µ inhomogeneous vectors, with indicated size

x, µ homogeneous vectorsA

m×n, R inhomogeneous matrices, with indicated size, generally n ≤ m

K, P homogeneous matricesλmax(.) largest eigenvalue(.)∞ entity at infinity, transformation referring to entities at infinityi ∈ I = {1, ..., I} index and index set(.)T transpose(.)−T transpose of inverse matrix(.)+ pseudo-inverse matrix(.)a approximated vector or matrix within iterative estimation procedure(.)∗ adjugate matrix(.)O cofactor matrix(.)r reduced, minimal vector(.)(s) reduced matrix with selected independent rows|.| absolute value of scalar, Euclidean norm of a vector, determinant of matrix||.|| Frobenius norm⟨., .⟩A inner product, e.g., ⟨x,y⟩A = xTAy⟨., ., .⟩ triple product of three 3-vectors,

identical to the determinant of their 3× 3 matrix [., ., .]⟨., ., ., .⟩ cross ratio of four numbers◦ operation, defined locally(.) dualizing or Hodge operatorx⊥ vector perpendicular to x∇x(p) nabla operator, gradient, Jacobian ∂x/∂p(.) stochastic variablevecA vec operatorx ∼ H (q) stochastic variable x follows distribution H (q)!(.) estimated value"(.) true value∃ there existsA⊙ B Hadamard productA⊗ B Kronecker product∩ intersection operator (‘cap’)∧ join operator (‘wedge’)∼= proportional to (vectors, matrices)∝ proportional to (functions)¬ not, antipode of an entity having negative homogeneous coordinates⇔ if and only if.= defining equation:= assignmenta

!= b constraint: a should be equal to b, or E(a) = b

a += b, a +

= b two elements are equivalent in oriented projective geometry[., .] closed interval(., .] semi-open interval⌊x⌋ floor function, largest integer smaller than x⌈x⌉ ceiling function, smallest integer larger than x

Table 0.4 List of Symbols in Part III (1)abbreviation meaningα parallactic angle between two raysA infinite homography, mapping from plane at infinity to image plane, also called H∞A = [C ,D] design matrix, Jacobian w.r.t. parameters,

partitioned for scene coordinates and orientation parameters(A,B,C) principal planes of camera coordinate system, rows of projection matrix PAl′ (Al′ ) projection plane to image line l ′B Jacobian of constraints w.r.t. observationsb, B base vectorc principal distancec(.) coordinate in camera coordinate systemc(x) function to derive inhomogeneous from homogeneous coordinatesDE number of parameters of observed image featureC 3× 3 matrix for conicsDT number of parameters for transformation or projectionDI number of parameters for scene feature(it) ∈ E index set E ⊂ I × T for observed image features fite′(e′), e′′(e′′) epipoles of image pairE epipolar planeE, Ett′ essential matrix, of images t and t′

E it matrix for selecting scene points observed in imagesF, Ftt′ fundamental matrix, of images t and t′

Fi(ki) scene feature Fi with coordinates ki, indices i ∈ IFi0(ki0) control scene feature Fi0 with coordinates ki0, indices i ∈ I0fit(lit) image feature fit with observed coordinates lit, indices (it) ∈ Ef it projection function for scene feature i and image tgit projection relation for scene feature i and image tG3,G4 d× d selection matrix Diag([1T

d , 0])G6 6× 6 selection matrix Diag({I3, 03×3})H (H) homography, perspective mappingH∞ infinite homography, mapping from plane at infinity to image plane, also called AHA homography, mapping plane A in object space to image planeH (xH) principal pointH matrix of constraints for fixing gaugeHg flight height over groundi(.) coordinate in image coordinate system{1, ..., i, ...I} = I index set for scene features(i, j) discrete image coordinates, unit pixelsl ′(X ), l ′(x ′′) epipolar lines of image pair, depending on scene or on image pointκ rotation angle around Z-axis of camera system, gear angleκ1, κ2 principal curvatures of surfacek vector of unknown coordinatesK1,K2 principal points of opticsK calibration matrixℓ(l ′′, l ′′′) projection operator to obtain line l ′Lx′ (Lx′ ) projection ray to image point x ′m(.) coordinate in model coordinate system of two or more imagesm scale difference of x′- and y′-image coordinatesM (M) motion or similarity

Table 0.5 List of Symbols in Part III (2)abbreviation meaningn(.) coordinate in normal camera coordinate system (parallel to scene coordinate system)N normal equation matrixNpp, Nkk normal equation matrices reduced to orientation parameters and coordinatesω rotation angle around X-axis of camera, roll angleO(Z) coordinates of projection centre℘2(x ′, l ′′), ℘3(x ′, l ′′) prediction of point from point and line in two other imagesφ rotation around Y -axis of camera, tilt angleP projection with projection matrix for pointsPt(pt) tth image with parameters of projectionP0t(p0t) image with observed parameters p0t of projection P0t, indices t ∈ T0p vector of unknown orientation parametersP projection matrix for pointsPd (d− 1)× d unit projection matrix [Id−1|0]q vector of parameters modelling non-linear image distortionsQ 3× 6-projection matrix for lines, 4× 4 matrix for quadricsQ6 3× 6 unit projection matrix [I3|03×3]R (R) rotation matrixs, S image scale s and image scale number 1/Ss shear of image coordinate systems vector of additional parameters for modelling systematic errors{1, ..., t, ...T} = T index set for images (time)T = [[Ti,jk]] trifocal tensorv ′(v′) vanishing pointx ′, x ′ observable image point, ideal image point (without distortion)

Z(a) 2× 2 matrix operator Z : a2×1

→#a1 −a2a2 a1

$

Table 0.6 Abbreviationsabbreviation meaningAO absolute orientationAR autoregressiveBLUE best linear unbiased estimatorDLT direct linear transformationEO exterior orientationGIS geoinformation system(s), geoinformation scienceGHM Gauss–Helmert modelGPS global positioning systemGSD ground sampling distanceIMU inertial measuring unitIO interior orientationLS least squaresMAD median absolute differenceMAP maximum a posterioriML maximum likelihoodMSE mean square errorPCA principal component analysisRANSAC random sample consensusRMSE root mean square errorSLERP spherical linear interpolationSVD singular value decomposition