ee795: computer vision and intelligent systems - unlvb1morris/ecg795/docs/slides11.pdf · ee795:...
TRANSCRIPT
![Page 1: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/1.jpg)
http://www.ee.unlv.edu/~b1morris/ecg795/
EE795: Computer Vision and
Intelligent Systems
Spring 2012
TTh 17:30-18:45 FDH 204
Lecture 11
130226
![Page 2: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/2.jpg)
Outline
• Review
▫ Feature-Based Alignment
▫ Image Warping
▫ 2D Alignment Using Least Squares
• Mosaics
• Panoramas
2
![Page 3: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/3.jpg)
Feature-Based Alignment • After detecting and matching features, may want to verify if the
matches are geometrically consistent ▫ Can feature displacements be described by 2D and 3D geometric
transformations
• Provides • Geometric registration
▫ 2D/3D mapping between images • Pose estimation
▫ Camera position with respect to a known 3D scene/object • Intrinsic camera calibration
▫ Find internal parameters of cameras (e.g. focal length, radial distortion)
3
![Page 4: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/4.jpg)
Richard Szeliski Image Stitching
4
Image Warping
• image filtering: change range of image
• g(x) = h(f(x))
• image warping: change domain of image • g(x) = f(h(x))
f
x
h
f
x
f
x
h
f
x
![Page 5: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/5.jpg)
Richard Szeliski Image Stitching
5
Image Warping
• image filtering: change range of image • g(x) = h(f(x))
• image warping: change domain of image
• g(x) = f(h(x))
h
h
f
f g
g
![Page 6: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/6.jpg)
Richard Szeliski Image Stitching
6
Parametric (global) warping
• Examples of parametric warps:
translation rotation aspect
affine perspective
cylindrical
![Page 7: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/7.jpg)
Richard Szeliski Image Stitching
7
2D coordinate transformations
• translation: x’ = x + t x = (x,y)
• rotation: x’ = R x + t
• similarity: x’ = s R x + t
• affine: x’ = A x + t
• perspective: x’ H x x = (x,y,1) (x is a homogeneous coordinate)
• These all form a nested group (closed w/ inv.)
![Page 8: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/8.jpg)
Richard Szeliski Image Stitching
8
Image Warping
• Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’) = f(h(x))?
f(x) g(x’) x x’
h(x)
![Page 9: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/9.jpg)
Richard Szeliski Image Stitching
9
Forward Warping
• Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’)
f(x) g(x’) x x’
h(x)
• What if pixel lands “between” two pixels?
![Page 10: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/10.jpg)
Richard Szeliski Image Stitching
10
Forward Warping
• Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’)
f(x) g(x’) x x’
h(x)
• What if pixel lands “between” two pixels?
• Answer: add “contribution” to several pixels,
normalize later (splatting)
• See griddata.m
![Page 11: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/11.jpg)
Richard Szeliski Image Stitching
11
Inverse Warping
• Get each pixel g(x’) from its corresponding location x = h-1(x’) in f(x)
f(x) g(x’) x x’
h(x)
• What if pixel comes from “between” two pixels?
![Page 12: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/12.jpg)
Richard Szeliski Image Stitching
12
Inverse Warping
• Get each pixel g(x’) from its corresponding location x = h-1(x’) in f(x)
• What if pixel comes from “between” two pixels?
• Answer: resample color value from
interpolated (prefiltered) source image
• See interp2.m
f(x) g(x’) x x’
![Page 13: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/13.jpg)
Forward vs. Inverse Warping
• Which type of warping is better?
• Usually inverse warping is preferred
▫ It eliminates holes
▫ However, it requires an invertible warp function
Not always possible
13
Alyosha Efros
![Page 14: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/14.jpg)
Least Squares Alignment
• Given a set of matched features 𝑥𝑖 , 𝑥𝑖′ ,
minimize sum of squared residual error
▫ 𝐸𝐿𝑆 = 𝑟𝑖2 =𝑖 𝑓 𝑥𝑖; 𝑝 − 𝑥𝑖
′ 2𝑖
𝑓 𝑥𝑖; 𝑝 - is the predicted location based on the transformation 𝑝
• The unknowns are the parameters 𝑝
▫ Need to have a model for transformation
▫ Estimate the parameters based on matched features
14
![Page 15: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/15.jpg)
Linear Least Squares Alignment • Many useful motion models have a linear
relationship between motion and parameters 𝑝
▫ Δ𝑥 = 𝑥′ − 𝑥 = 𝐽 𝑥 𝑝
𝐽 =𝜕𝑓
𝜕𝑝 - the Jacobian of the transform 𝑓 with respect to
the motion parameters 𝑝
• Linear least squares
▫ 𝐸𝐿𝐿𝑆 = 𝐽 𝑥𝑖 𝑝 − Δ𝑥𝑖2 =𝑖 𝑝𝑇𝐴𝑝 − 2𝑝𝑇𝑏 + 𝑐
Quadratic form
• The minimum is found by solving the normal equations
▫ 𝐴𝑝 = 𝑏 𝐴 = 𝐽𝑇(𝑖 𝑥𝑖)𝐽(𝑥𝑖) – Hessian matrix
𝑏 = 𝐽𝑇(𝑖 𝑥𝑖)Δ𝑥𝑖 ▫ Gives the LLS estimate for the motion parameters
15
![Page 16: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/16.jpg)
Jacobians of 2D Transformations
16
![Page 17: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/17.jpg)
Improving Motion Estimates • A number of techniques can improve upon linear least
squares • Uncertainty weighting
▫ Weight the matches based certainty of the match – texture in the match region
• Non-linear least squares ▫ Iterative algorithm to guess parameters and iteratively
improve guess
• Robust least squares ▫ Explicitly handle outliers (bad matches) – don’t use L2
norm
• RANSAC ▫ Randomly select subset of corresponding points, compute
initial estimate of 𝑝, count the inliers from all the other correspondences, good match has many inliers
17
![Page 18: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/18.jpg)
Richard Szeliski Image Stitching
18
Image Mosaics
• + + … + =
Goal: Stitch together several images into a
seamless composite
![Page 19: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/19.jpg)
Richard Szeliski Image Stitching
19
Motion models
Translation
2 unknowns
Affine
6 unknowns
Perspective
8 unknowns
3D rotation
3 unknowns
![Page 20: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/20.jpg)
Richard Szeliski Image Stitching
20
Plane perspective mosaics
▫ 8-parameter generalization of affine motion
works for pure rotation or planar surfaces
▫ Limitations:
local minima
slow convergence
difficult to control interactively
![Page 21: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/21.jpg)
Richard Szeliski Image Stitching
21
Image warping with homographies
image plane in front image plane below
![Page 22: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/22.jpg)
Richard Szeliski Image Stitching
22
Rotational mosaics
▫ Directly optimize rotation and focal length
▫ Advantages:
ability to build full-view panoramas
easier to control interactively
more stable and accurate estimates
![Page 23: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/23.jpg)
Richard Szeliski Image Stitching
23
3D → 2D Perspective Projection
u
(Xc,Yc,Zc)
uc f
![Page 24: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/24.jpg)
Richard Szeliski Image Stitching
24
Rotational mosaic
• Projection equations
1. Project from image to 3D ray
• (x0,y0,z0) = (u0-uc,v0-vc,f)
2. Rotate the ray by camera motion
• (x1,y1,z1) = R01 (x0,y0,z0)
3. Project back into new (source) image
• (u1,v1) = (fx1/z1+uc,fy1/z1+vc)
![Page 25: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/25.jpg)
Richard Szeliski Image Stitching
25
mosaic PP
Image reprojection
• The mosaic has a natural interpretation in 3D
▫ The images are reprojected onto a common plane
▫ The mosaic is formed on this plane
![Page 26: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/26.jpg)
Richard Szeliski Image Stitching
26
Image Mosaics (stitching)
• Blend together several overlapping images into one seamless mosaic (composite)
▫ [Szeliski & Shum, SIGGRAPH’97]
▫ [Szeliski, FnT CVCG, 2006]
• + + … + =
![Page 27: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/27.jpg)
Richard Szeliski Image Stitching
27
Mosaics for Video Coding
• Convert masked images into a background sprite for content-based coding
• + + +
• =
![Page 28: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/28.jpg)
Richard Szeliski Image Stitching
28
Establishing correspondences
1. Direct method:
▫ Use generalization of affine motion model [Szeliski & Shum ’97]
2. Feature-based method
▫ Extract features, match, find consisten inliers [Lowe ICCV’99; Schmid ICCV’98, Brown&Lowe ICCV’2003]
▫ Compute R from correspondences (absolute orientation)
![Page 29: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/29.jpg)
Richard Szeliski Image Stitching
29
Stitching demo
![Page 30: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/30.jpg)
Richard Szeliski Image Stitching
30
Panoramas
• What if you want a 360 field of view?
mosaic Projection Cylinder
![Page 31: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/31.jpg)
Richard Szeliski Image Stitching
31
Cylindrical panoramas
• Steps ▫ Reproject each image onto a cylinder ▫ Blend ▫ Output the resulting mosaic
![Page 32: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/32.jpg)
Cylindrical Panoramas
• Map image to cylindrical or spherical coordinates
▫ need known focal length
Richard Szeliski Image Stitching
32
f = 180 (pixels) Image 384x300 f = 380 f = 280
![Page 33: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/33.jpg)
Richard Szeliski Image Stitching
33
▫ Map 3D point (X,Y,Z) onto cylinder
Cylindrical projection
X
Y
Z
unit cylinder
unwrapped cylinder
• Convert to cylindrical coordinates
cylindrical image
• Convert to cylindrical image coordinates
– s defines size of the final image
![Page 34: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/34.jpg)
Richard Szeliski Image Stitching
34
Cylindrical warping
•Given focal length f and image center (xc,yc)
X
Y
Z
(X,Y,Z)
(sinq,h,cosq)
![Page 35: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/35.jpg)
Richard Szeliski Image Stitching
35
Spherical warping
•Given focal length f and image center (xc,yc)
X
Y
Z
(x,y,z)
(sinθcosφ,cosθcosφ,sinφ)
cos φ
φ
cos θ cos φ
sin φ
![Page 36: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/36.jpg)
Richard Szeliski Image Stitching
36
3D rotation
•Rotate image before placing on unrolled sphere
(x,y,z)
(sinθcosφ,cosθcosφ,sinφ)
cos φ
φ
cos θ cos φ
sin φ
_ _
_ _
p = R p
![Page 37: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/37.jpg)
Distortion Correction • Radial distortion
▫ Correct for “bending” in wide field of view lenses
• Fisheye lens
▫ Extreme “bending” in ultra-wide fields of view
37
Adapted from R. Szeliski
![Page 38: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/38.jpg)
Richard Szeliski Image Stitching
38
Image Stitching
1. Align the images over each other
▫ camera pan ↔ translation on cylinder
2. Blend the images together
![Page 39: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/39.jpg)
Richard Szeliski Image Stitching
39
Image stitching steps
1. Take pictures on a tripod (or handheld)
2. Warp images to spherical coordinates
3. Extract features
4. Align neighboring pairs using RANSAC
5. Write out list of neighboring translations
6. Correct for drift
7. Read in warped images and blend them
8. Crop the result and import into a viewer
![Page 40: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/40.jpg)
Richard Szeliski Image Stitching
40
Matching features
What do we do about the “bad” matches?
![Page 41: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/41.jpg)
Richard Szeliski Image Stitching
41
RAndom SAmple Consensus
Select one match, count inliers
![Page 42: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/42.jpg)
Richard Szeliski Image Stitching
42
RAndom SAmple Consensus
Select one match, count inliers
![Page 43: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/43.jpg)
Richard Szeliski Image Stitching
43
Least squares fit
Find “average” translation vector
![Page 44: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/44.jpg)
Richard Szeliski Image Stitching
44
Assembling the panorama
• Stitch pairs together, blend, then crop
![Page 45: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/45.jpg)
Richard Szeliski Image Stitching
45
Problem: Drift
• Error accumulation
▫ small (vertical) errors accumulate over time
▫ apply correction so that sum = 0 (for 360° pan.)
![Page 46: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/46.jpg)
Richard Szeliski Image Stitching
46
Problem: Drift
• Solution ▫ add another copy of first image at the end ▫ this gives a constraint: yn = y1 ▫ there are a bunch of ways to solve this problem
add displacement of (y1 – yn)/(n -1) to each image after the first
compute a global warp: y’ = y + ax run a big optimization problem, incorporating this
constraint best solution, but more complicated known as “bundle adjustment”
(x1,y1)
copy of first
image
(xn,yn)
![Page 47: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/47.jpg)
Richard Szeliski Image Stitching
47
Full-view Panorama
+
+
+
+
![Page 48: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/48.jpg)
Richard Szeliski Image Stitching
48
Texture Mapped Model
![Page 49: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/49.jpg)
Richard Szeliski Image Stitching
49
Global alignment
• Register all pairwise overlapping images
• Use a 3D rotation model (one R per image)
• Use direct alignment (patch centers) or feature based
• Infer overlaps based on previous matches (incremental)
• Optionally discover which images overlap other images using feature selection (RANSAC)
![Page 50: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/50.jpg)
Richard Szeliski Image Stitching
50
Recognizing Panoramas
[Brown & Lowe, ICCV’03]
![Page 51: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/51.jpg)
Richard Szeliski Image Stitching
51
Finding the panoramas
![Page 52: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/52.jpg)
Richard Szeliski Image Stitching
52
Finding the panoramas
![Page 53: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/53.jpg)
Richard Szeliski Image Stitching
53
Finding the panoramas
![Page 54: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/54.jpg)
Richard Szeliski Image Stitching
54
Finding the panoramas
![Page 55: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/55.jpg)
Fully automated 2D stitching
• Free copy from Microsoft Essentials
▫ http://windows.microsoft.com/en-us/windows-live/photo-gallery-get-started
Richard Szeliski Image Stitching
55
![Page 56: EE795: Computer Vision and Intelligent Systems - UNLVb1morris/ecg795/docs/slides11.pdf · EE795: Computer Vision and Intelligent Systems ... Image Stitching Richard Szeliski 25 mosaic](https://reader035.vdocuments.mx/reader035/viewer/2022062413/5aa851747f8b9a86188b67c9/html5/thumbnails/56.jpg)
Richard Szeliski Image Stitching
56
Final thought:
What is a “panorama”?
• Tracking a subject ▫ Panorama
• Repeated (best) shots ▫ Photo Fuse
• Multiple exposures ▫ Photo Fuse
• “Infer” what photographer wants?