computational photography: image registration jinxiang chai

Click here to load reader

Post on 22-Dec-2015

215 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

  • Slide 1
  • Computational Photography: Image Registration Jinxiang Chai
  • Slide 2
  • Review Image warping Image morphing
  • Slide 3
  • Image Warping Warping function - similarity, affine, projective etc
  • Slide 4
  • Image Warping Warping function - similarity, affine, projective etc Image warping - forward warping and two-pass 1D warping - backward warping x y u v S(x,y) T(u,v) forward Inverse
  • Slide 5
  • Image Warping Warping function - similarity, affine, projective etc Image warping - forward warping and two-pass 1D warping - backward warping Resampling filter - point sampling - bilinear filter x y u v S(x,y) T(u,v) forward Inverse
  • Slide 6
  • Image Morphing Point based image morphing Vector based image morphing
  • Slide 7
  • Image Registration Image warping: given h and f, compute g g(x) = f(h(x)) h f g? Image registration: given f and g, compute h h? f g
  • Slide 8
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization)
  • Slide 9
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization) Align images (mosaics)
  • Slide 10
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization) Align images (mosaics) View morphing
  • Slide 11
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization) Align images (mosaics) View morphing Special effects
  • Slide 12
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization) Align images (mosaics) View morphing Special effects Medical image registration
  • Slide 13
  • Why Image Registration? Lots of uses Correct for camera jitter (stabilization) Align images (mosaics) View morphing Special effects Medical image registration Image based modeling/rendering Etc.
  • Slide 14
  • Image Registration How do we align two images automatically? Two broad approaches: Feature-based alignment Find a few matching features in both images compute alignment Direct (pixel-based) alignment Search for alignment where most pixels agree
  • Slide 15
  • Outline Image registration - feature-based approach - pixel-based approach
  • Slide 16
  • Required Readings Section 6.1 (Szeliski book) Section 8.1.3 (Szeliski book) Section 8.2 (Szeliski book) Section 8.4 (Szeliski book)
  • Slide 17
  • Outline Image registration - feature-based approach - pixel-based approach
  • Slide 18
  • Feature-based Alignment 1.Find a few important features (aka Interest Points) 2.Match them across two images 3.Compute image transformation function h
  • Slide 19
  • Feature-based Alignment 1.Find a few important features (aka Interest Points) 2.Match them across two images 3.Compute image transformation function h How to choose features Choose only the points (features) that are salient, i.e. likely to be there in the other image How to find these features?
  • Slide 20
  • Feature-based Alignment 1.Find a few important features (aka Interest Points) 2.Match them across two images 3.Compute image transformation function h How to choose features Choose only the points (features) that are salient, i.e. likely to be there in the other image How to find these features? windows where has two large eigenvalues Harris Corner detector
  • Slide 21
  • Feature Detection -Two images taken at the same place with different angles - Projective transformation H 3X3
  • Slide 22
  • Feature Matching ? -Two images taken at the same place with different angles - Projective transformation H 3X3
  • Slide 23
  • Feature Matching ? -Two images taken at the same place with different angles - Projective transformation H 3X3 How do we match features across images? Any criterion?
  • Slide 24
  • Feature Matching ? -Two images taken at the same place with different angles - Projective transformation H 3X3 How do we match features across images? Any criterion?
  • Slide 25
  • Feature Matching Intensity/Color similarity The intensity of pixels around the corresponding features should have similar intensity
  • Slide 26
  • Feature Matching Intensity/Color similarity The intensity of pixels around the corresponding features should have similar intensity Cross-correlation, SSDCross-correlationSSD
  • Slide 27
  • Feature Matching Intensity/Color similarity The intensity of pixels around the corresponding features should have similar intensity Cross-correlation, SSDCross-correlationSSD Distance constraint The displacement of features should be smaller than a given threshold
  • Slide 28
  • Feature-space Outlier Rejection bad Good
  • Slide 29
  • Feature-space Outlier Rejection Can we now compute H 3X3 from the blue points?
  • Slide 30
  • Feature-space Outlier Rejection Can we now compute H 3X3 from the blue points? No! Still too many outliers
  • Slide 31
  • Feature-space Outlier Rejection Can we now compute H 3X3 from the blue points? No! Still too many outliers What can we do?
  • Slide 32
  • Feature-space Outlier Rejection Can we now compute H 3X3 from the blue points? No! Still too many outliers What can we do? Robust estimation!
  • Slide 33
  • Robust EstimationRobust Estimation: A Toy Example How to fit a line based on a set of 2D points?
  • Slide 34
  • Robust EstimationRobust Estimation: A Toy Example How to fit a line based on a set of 2D points? RANSACRANSAC: an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliersoutliers
  • Slide 35
  • RANSACRANSAC: RANdom SAmple Consensus Objective Robust fit of model to data set S which contains outliers Algorithm (i)Randomly select a sample of s data points from S and instantiate the model from this subset. (ii)Determine the set of data points S i which are within a distance threshold t of the model. The set S i is the consensus set of samples and defines the inliers of S. (iii)If the subset of S i is greater than some threshold T, re- estimate the model using all the points in S i and terminate (iv)If the size of S i is less than T, select a new subset and repeat the above. (v)After N trials the largest consensus set S i is selected, and the model is re-estimated using all the points in the subset S i
  • Slide 36
  • RANSAC Repeat M times: Sample minimal number of matches to estimate two view relation (affine, perspective, etc). Calculate number of inliers or posterior likelihood for relation. Choose relation to maximize number of inliers.
  • Slide 37
  • RANSAC Line Fitting Example Task: Estimate best line
  • Slide 38
  • RANSAC Line Fitting Example Sample two points
  • Slide 39
  • RANSAC Line Fitting Example Fit Line
  • Slide 40
  • RANSAC Line Fitting Example Total number of points within a threshold of line.
  • Slide 41
  • RANSAC Line Fitting Example Repeat, until get a good result
  • Slide 42
  • RANSAC Line Fitting Example Repeat, until get a good result
  • Slide 43
  • RANSAC Line Fitting Example Repeat, until get a good result
  • Slide 44
  • How Many Samples? Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99 proportion of outliers e s5%10%20%25%30%40%50% 2235671117 33479111935 435913173472 54612172657146 64716243797293 748203354163588 8592644782721177
  • Slide 45
  • How Many Samples? Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99 proportion of outliers e s5%10%20%25%30%40%50% 2235671117 33479111935 435913173472 54612172657146 64716243797293 748203354163588 8592644782721177 Affine transform
  • Slide 46
  • How Many Samples? Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99 proportion of outliers e s5%10%20%25%30%40%50% 2235671117 33479111935 435913173472 54612172657146 64716243797293 748203354163588 8592644782721177 Projective transform
  • Slide 47
  • RANSAC for Estimating Projective Transformation RANSAC loop: 1.Select four feature pairs (at random) 2.Compute the transformation matrix H (exact) 3.Compute inliers where SSD(p i , H p i ) < 4.Keep largest set of inliers 5.Re-compute least-squares H estimate on all of the inliers
  • Slide 48
  • RANSAC
  • Slide 49
  • Feature-based Registration Works for small or large motion Model the motion within a patch or whole image using a parametric transformation model
  • Slide 50
  • Feature-based Registration Works for small or large motion Model the motion within a patch or whole image using a parametric transformation model How to deal with motions that cannot be described by a small number of parameters?
  • Slide 51
  • Outline Image registration - feature-based approach - pixel-based approach