structure-from-motion algorithm · 2011-10-13 · © 2006 noah snavely reproduced with permission...

© 2006 Noah Snavely

Structure-from-Motion Algorithm

Bundle Adjustment

© 2006 Noah Snavely

15,464

76,389

37,383

© 2006 Noah Snavely

Reproduced with permission of Yahoo! Inc. © 2005 by Yahoo! Inc.

YAHOO! and the YAHOO! logo are trademarks of Yahoo! Inc.

© 2006 Noah Snavely

Structure from motion

© 2006 Noah Snavely

Photo Tourism overview

Scene

reconstruction

Photo

Explorer Input photographs Relative camera

positions and orientations

Point cloud

Sparse correspondence

© 2006 Noah Snavely

Scene reconstruction

• Automatically estimate

– position, orientation, and focal length of cameras

– 3D positions of feature points

Feature detection

Pairwise

feature matching

Incremental

structure

from motion

Correspondence

estimation

© 2006 Noah Snavely

Feature detection

Detect features using SIFT [Lowe, IJCV 2004]

© 2006 Noah Snavely

Feature detection

• Detect features using SIFT [Lowe, IJCV 2004]

© 2006 Noah Snavely

Pairwise feature matching

• Match features between each pair of images

© 2006 Noah Snavely

Feature matching

Refine matching using RANSAC [Fischler & Bolles 1987]

to estimate fundamental matrices between pairs

© 2006 Noah Snavely

Structure from motion

Camera 1

Camera 2

Camera 3

R1,t1

R2,t2

R3,t3

p1

p4

p3

p2

p5

p6

p7

minimize

f (R, T, P)

© 2006 Noah Snavely

Minimize the re-projection error

Estimate:-M = Projection matrix, X = 3D points

© 2006 Noah Snavely

Levenberg–Marquardt algorithm

Minimize

Where, , J = Jacobian matrix

At its minimum, the sum of squares, S(β), the gradient of S with respect to δ will be zero.

The above first-order approximation of gives

In the vector notation,

Taking the derivative with respect to δ and setting the result to zero gives,

© 2006 Noah Snavely

Incremental structure from motion

© 2006 Noah Snavely

Incremental structure from motion

© 2006 Noah Snavely

3D reconstruction

© 2006 Noah Snavely

Demo

© 2006 Noah Snavely

• Advantages

Handle large number of views

Handle missing data

• Limitations

Large minimization problem (parameters grow

with number of views) – Takes lot of time

requires good initial condition

© 2006 Noah Snavely

REFERENCES:

Noah Snavely, Steven M. Seitz, Richard Szeliski. Photo Tourism: Exploring image

collections in 3D. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006),

2006.

Noah Snavely, Steven M. Seitz, Richard Szeliski. Modeling the World from Internet

Photo Collections. International Journal of Computer Vision, 2007.

Source Code: http://phototour.cs.washington.edu/bundler/#S4

Youtube Link: http://www.youtube.com/watch?v=9M4KWgRGNa0

Slides Courtesy by :- Noah Snavely, Assistant Professor, Cornell University.