prof. feng liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/lecture8.pdfpanorama building: history 4...
TRANSCRIPT
![Page 1: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/1.jpg)
Prof. Feng Liu
Spring 2020
http://www.cs.pdx.edu/~fliu/courses/cs510/
04/23/2020
![Page 2: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/2.jpg)
Last Time
Re-lighting
◼ HDR
2
![Page 3: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/3.jpg)
Today
Panorama
◼ Overview
◼ Feature detection
3With slides by Prof. C. Dyer and K. Grauman
![Page 4: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/4.jpg)
Panorama Building: History
4
Along the River During Ching Ming Festival
by Z.D Zhang (1085-1145 )
San Francisco from Rincon Hill, 1851,
by Martin Behrmanx
![Page 5: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/5.jpg)
Panorama Building: A Concise History
The state of the art and practice is good at assembling
images into panoramas
◼ Mid 90s –Commercial Players (e.g. QuicktimeVR)
◼ Late 90s –Robust stitchers (in research)
◼ Early 00s –Consumer stitching common
◼ Mid 00s –Automation
5
![Page 6: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/6.jpg)
Stitching Recipe
Align pairs of images
Align all to a common frame
Adjust (Global) & Blend
6
![Page 7: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/7.jpg)
Stitching Images Together
7
![Page 8: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/8.jpg)
When do two images “stitch”?
8
![Page 9: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/9.jpg)
Images can be transformed to match
9
![Page 10: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/10.jpg)
Images are related by Homographies
10
![Page 11: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/11.jpg)
Compute Homographies
11
![Page 12: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/12.jpg)
Automatic Feature Points Matching
Match local neighborhoods around points
Use descriptors to efficiently compare SIFT
◼ [Lowe 04] most common choice
12
![Page 13: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/13.jpg)
Stitching Recipe
Align pairs of images
Align all to a common frame
Adjust (Global) & Blend
13
![Page 14: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/14.jpg)
Wide Baseline Matching
• Images taken by cameras that are far apart make the
correspondence problem very difficult
• Feature-based approach: Detect and match feature
points in pairs of images
Credit: C. Dyer
![Page 15: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/15.jpg)
• Detect feature points
• Find corresponding pairs
Matching with Features
Credit: C. Dyer
![Page 16: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/16.jpg)
Matching with Features
Problem 1:
◼ Detect the same point
independently in both images
no chance to match!
We need a repeatable detector
Credit: C. Dyer
![Page 17: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/17.jpg)
Matching with Features
Problem 2:
◼ For each point correctly
recognize the corresponding point
?
We need a reliable and distinctive descriptor
Credit: C. Dyer
![Page 18: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/18.jpg)
Local: features are local, so robust to occlusion and clutter (no prior segmentation)
Invariant (or covariant) to many kinds of geometric and photometric transformations
Robust: noise, blur, discretization, compression, etc. do not have a big impact on the feature
Distinctive: individual features can be matched to a large database of objects
Quantity: many features can be generated for even small objects
Accurate: precise localization
Efficient: close to real-time performance
Properties of an Ideal Feature
Credit: C. Dyer
![Page 19: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/19.jpg)
Problem 1: Detecting Good Feature Points
Credit: C. Dyer
[Image from T. Tuytelaars ECCV 2006 tutorial]
![Page 20: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/20.jpg)
Hessian
Harris
Lowe: SIFT (DoG)
Mikolajczyk & Schmid:Hessian/Harris-Laplacian/Affine
Tuytelaars & Van Gool: EBR and IBR
Matas: MSER
Kadir & Brady: Salient Regions
Others
Feature Detectors
Credit: C. Dyer
![Page 21: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/21.jpg)
C. Harris, M. Stephens, “A Combined Corner and Edge Detector,” 1988
Harris Corner Point Detector
Credit: C. Dyer
![Page 22: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/22.jpg)
We should recognize the point by looking through a small window
Shifting a window in any direction should give a large change in response
Harris Detector: Basic Idea
Credit: C. Dyer
![Page 23: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/23.jpg)
“flat” region:
no change in
all directions
“edge”:
no change along
the edge direction
“corner”:
significant change
in all directions
Harris Detector: Basic Idea
Credit: C. Dyer
![Page 24: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/24.jpg)
2
,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y= + + −
Change of intensity for a (small) shift by [u,v] in image I:
IntensityShifted intensity
Weighting function
orWeighting function w(x,y) =
Gaussian1 in window, 0 outside
Harris Detector: Derivation
Credit: R. Szeliski
![Page 25: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/25.jpg)
Calculus: Taylor Series Expansion
25
𝑓 𝑥 + 𝑢 = 𝑓 𝑥 + 𝑢𝑓′ 𝑥 + 𝑂(𝑢2)
A real function f (x+u) can be approximated as the 2nd order of its Taylor series expansion at a point x.
![Page 26: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/26.jpg)
Apply 2nd order Taylor series expansion:
=
=
=
++=
yx
yx
yx
y
yx
x
yxIyxIyxwC
yxIyxwB
yxIyxwA
BvCuvAuvuE
,
,
2
,
2
22
),(),(),(
),(),(
),(),(
2),(
( , )A C u
E u v u vC B v
=
xyxII x = /),(
yyxII y = /),(
Harris Detector
Credit: R. Szeliski
![Page 27: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/27.jpg)
( , ) ,u
E u v u v Mv
Expanding E(u,v) in a 2nd order Taylor series, we have, for small
shifts, [u,v], a bilinear approximation:
2
2,
( , )x x y
x y x y y
I I IM w x y
I I I
=
where M is a 2 2 matrix computed from image derivatives:
Note: Sum computed over small neighborhood around given pixel
xyxII x = /),(
yyxII y = /),(
Harris Corner Detector
Credit: R. Szeliski
![Page 28: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/28.jpg)
( , ) ,u
E u v u v Mv
Intensity change in shifting window: eigenvalue analysis
1, 2 – eigenvalues of M
direction of the
slowest change
direction of the
fastest change
(max)-1/2
(min)-1/2
Ellipse E(u,v) = const
Harris Corner Detector
Credit: R. Szeliski
![Page 29: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/29.jpg)
1 and 2 both large
Image patch
SSD surface
Selecting Good Features
Credit: C. Dyer
![Page 30: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/30.jpg)
large 1, small 2
SSD surface
Selecting Good Features
Credit: C. Dyer
![Page 31: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/31.jpg)
small 1, small 2
SSD surface
Selecting Good Features
Credit: C. Dyer
![Page 32: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/32.jpg)
1
2
“Corner”
1 and 2 both large,
1 ~ 2;
E increases in all
directions
1 and 2 are small;
E is almost constant
in all directions
“Edge”
1 >> 2
“Edge”
2 >> 1
“Flat”
region
Classification of
image points using
eigenvalues of M:
Harris Corner Detector
Credit: C. Dyer
![Page 33: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/33.jpg)
Harris Corner Detector
Measure of corner response:
( )2
det traceR M k M= −
1 2
1 2
det
trace
M
M
=
= +
k is an empirically-determined constant; e.g., k = 0.05
Credit: C. Dyer
![Page 34: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/34.jpg)
Harris Corner Detector
1
2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on
eigenvalues of M
• R is large for a corner
• R is negative with large
magnitude for an edge
• |R| is small for a flat
region
R > 0
R < 0
R < 0|R| small
Credit: C. Dyer
![Page 35: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/35.jpg)
Harris Corner Detector: Algorithm
Algorithm:
1. Find points with large corner
response function R
(i.e., R > threshold)
2. Take the points of local maxima
of R (for localization) by non-
maximum suppression
Credit: C. Dyer
![Page 36: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/36.jpg)
Harris Detector: Example
Credit: C. Dyer
![Page 37: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/37.jpg)
Compute corner response R = 12 – k(1 + 2)2
Harris Detector: Example
Credit: C. Dyer
![Page 38: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/38.jpg)
Harris Detector: Example
Find points with large corner response: R > thresholdCredit: C. Dyer
![Page 39: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/39.jpg)
Take only the points of local maxima of R
Harris Detector: Example
Credit: C. Dyer
![Page 40: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/40.jpg)
Harris Detector: Example
Credit: C. Dyer
![Page 41: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/41.jpg)
Interest points extracted with Harris (~ 500 points)
Harris Detector: Example
Credit: C. Dyer
![Page 42: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/42.jpg)
Harris Detector: Example
Credit: C. Dyer
![Page 43: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/43.jpg)
Harris Detector: Summary
Average intensity change in direction [u,v] can be
expressed in bilinear form:
Describe a point in terms of eigenvalues of M:
measure of corner response:
A good (corner) point should have a large intensity change in all directions, i.e., R should be a large
positive value
( , ) ,u
E u v u v Mv
( )2
1 2 1 2R k = − +
Credit: C. Dyer
![Page 44: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/44.jpg)
Harris Detector Properties
Rotation invariance
Ellipse rotates but its shape (i.e., eigenvalues)
remains the same
Corner response R is invariant to image rotation
Credit: C. Dyer
![Page 45: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/45.jpg)
But not invariant to image scale
Fine scale: All points will
be classified as edges
Coarse scale: Corner
Harris Detector Properties
Credit: C. Dyer
![Page 46: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/46.jpg)
Harris Detector Properties
Quality of Harris detector for different scale
changes
Repeatability rate:
# correct correspondences
# possible correspondences
C. Schmid et al., “Evaluation of Interest Point Detectors,” IJCV 2000
Credit: C. Dyer
![Page 47: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/47.jpg)
47
Invariant Local Features
Goal: Detect the same interest points
regardless of image changes due to
translation, rotation, scale, viewpoint
![Page 48: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/48.jpg)
48
Geometry◼ Rotation
◼ Similarity (rotation + uniform scale)
◼ Affine (scale dependent on direction)
valid for: orthographic camera, locally planar
object
Photometry◼ Affine intensity change (I → a I + b)
Models of Image Change
Credit: C. Dyer
![Page 49: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/49.jpg)
SIFT Detector [Lowe ’04]
Difference-of-Gaussian (DoG)
is an approximation of the
Laplacian-of-Gaussian (LoG)
− =
Lowe, D. G., “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, 60, 2, pp. 91-110, 2004Credit: C. Dyer
![Page 50: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/50.jpg)
SIFT Detector
Credit: C. Dyer
![Page 51: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/51.jpg)
SIFT Detector
Credit: C. Dyer
![Page 52: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/52.jpg)
SIFT Detector Algorithm Summary
Detect local maxima in
position and scale of
squared values of
difference-of-Gaussian
Fit a quadratic to
surrounding values for
sub-pixel and sub-
scale interpolation
Output = list of (x, y, )
points
Blur
Resample
Subtract
Credit: C. Dyer
![Page 53: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/53.jpg)
References on Feature Descriptors
A performance evaluation of local descriptors, K.
Mikolajczyk and C. Schmid, IEEE Trans. PAMI 27(10),
2005
Evaluation of features detectors and descriptors
based on 3D objects, P. Moreels and P. Perona, Int. J. Computer Vision 73(3), 2007
Credit: C. Dyer
![Page 54: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/54.jpg)
Student paper presentation
54
Color Harmonization
D. Cohen-Or, O. Sorkine, R. Gal, T. Leyv, and, Y. Xu.
ACM SIGGRAPH 2006
Presenter: Chen, Aihua
![Page 55: Prof. Feng Liuweb.cecs.pdx.edu/~fliu/courses/cs510/lectures/Lecture8.pdfPanorama Building: History 4 Along the River During Ching Ming Festival by Z.D Zhang (1085-1145 ) San Francisco](https://reader030.vdocuments.mx/reader030/viewer/2022041105/5f067c5c7e708231d4183ab6/html5/thumbnails/55.jpg)
Next Time
Panorama
◼ Feature matching
Student paper presentations◼ 04/28: Crocker, Orion
Color Conceptualization. X. Hou and L. Zhang.
ACM Multimedia 2007
◼ 04/28: Dimaggio, Antonio
Colorization Using Optimization. A. Levin, D. Lischinski,
Y. Weiss. ACM SIGGRAPH 2004
◼ 04/30: Everett, Riley
Photographic tone reproduction for digital images.
E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, SIGGRAPH 2012
◼ 04/30: Fan, Yubin
Burst photography for high dynamic range and low-light imaging on mobile
cameras. Samuel W. Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams,
Jonathan T. Barron, Florian Kainz, Jiawen Chen, and Marc Levoy. SIGGRAPH
Asia 2016 55