local features: detection and description tuesday october 8 devi parikh virginia tech slide credit:...
TRANSCRIPT
![Page 1: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/1.jpg)
1
Local features:detection and description
Tuesday October 8Devi Parikh
Virginia Tech
Slide credit: Kristen Grauman
Disclaimer: Most slides have been borrowed from Kristen Grauman, who may have borrowed some of them from others. Any time a slide did not already have a credit on it, I have credited it to Kristen. So there is a chance some of these credits are inaccurate.
![Page 2: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/2.jpg)
2
Announcements
• Project proposal grades out
• PS3 out today
Slide credit: Devi Parikh
![Page 3: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/3.jpg)
3
Last time
• Image warping based on homography• Detecting corner-like points in an image
Slide credit: Kristen Grauman
![Page 4: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/4.jpg)
Today
• Local invariant features– Detection of interest points
• (Harris corner detection)• Scale invariant blob detection: LoG
– Description of local patches• SIFT : Histograms of oriented gradients
Slide credit: Kristen Grauman4
![Page 5: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/5.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description: Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
],,[ )1()1(11 dxx x
],,[ )2()2(12 dxx x
Kristen Grauman 5
![Page 6: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/6.jpg)
Goal: interest operator repeatability
• We want to detect (at least some of) the same points in both images.
• Yet we have to be able to run the detection procedure independently per image.
No chance to find true matches!
Slide credit: Kristen Grauman6
![Page 7: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/7.jpg)
Goal: descriptor distinctiveness
• We want to be able to reliably determine which point goes with which.
• Must provide some invariance to geometric and photometric differences between the two views.
?
Slide credit: Kristen Grauman7
![Page 8: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/8.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description:Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
Slide credit: Kristen Grauman8
![Page 9: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/9.jpg)
yyyx
yxxx
IIII
IIIIyxwM ),(
x
II x
y
II y
y
I
x
III yx
Recall: Corners as distinctive interest points
2 x 2 matrix of image derivatives (averaged in neighborhood of a point).
Notation:
Slide credit: Kristen Grauman 9
![Page 10: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/10.jpg)
Since M is symmetric, we have TXXM
2
1
0
0
iii xMx
The eigenvalues of M reveal the amount of intensity change in the two principal orthogonal gradient directions in the window.
Recall: Corners as distinctive interest points
Slide credit: Kristen Grauman10
![Page 11: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/11.jpg)
“flat” region1 and 2 are small;
“edge”:1 >> 2
2 >> 1
“corner”:1 and 2 are large, 1 ~ 2;
One way to score the cornerness:
Recall: Corners as distinctive interest points
Slide credit: Kristen Grauman11
![Page 12: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/12.jpg)
Harris corner detector
1) Compute M matrix for image window surrounding each pixel to get its cornerness score.
2) Find points with large corner response (f > threshold)
3) Take the points of local maxima, i.e., perform non-maximum suppression
Slide credit: Kristen Grauman12
![Page 13: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/13.jpg)
Harris Detector: Steps
Slide credit: Kristen Grauman13
![Page 14: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/14.jpg)
Harris Detector: StepsCompute corner response f
Slide credit: Kristen Grauman14
![Page 15: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/15.jpg)
Harris Detector: StepsFind points with large corner response: f > threshold
Slide credit: Kristen Grauman 15
![Page 16: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/16.jpg)
Harris Detector: StepsTake only the points of local maxima of f
Slide credit: Kristen Grauman 16
![Page 17: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/17.jpg)
Harris Detector: Steps
Slide credit: Kristen Grauman17
![Page 18: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/18.jpg)
Properties of the Harris corner detector
Rotation invariant?
Scale invariant?
TXXM
2
1
0
0
Yes
Slide credit: Kristen Grauman18
![Page 19: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/19.jpg)
Properties of the Harris corner detector
Rotation invariant?
Scale invariant?
All points will be classified as edges
Corner !
Yes
No
Slide credit: Kristen Grauman19
![Page 20: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/20.jpg)
Scale invariant interest points
How can we independently select interest points in each image, such that the detections are repeatable across different scales?
Slide credit: Kristen Grauman20
![Page 21: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/21.jpg)
Automatic scale selection
Intuition: • Find scale that gives local maxima of some function f
in both position and scale.
f
region size
Image 1f
region size
Image 2
s1 s2Slide credit: Kristen Grauman 21
![Page 22: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/22.jpg)
What can be the “signature” function?
Slide credit: Kristen Grauman22
![Page 23: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/23.jpg)
Recall: Edge detection
gdx
df
f
gdx
d
Source: S. Seitz
Edge
Derivativeof Gaussian
Edge = maximumof derivative
23
![Page 24: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/24.jpg)
gdx
df
2
2
f
gdx
d2
2
Edge
Second derivativeof Gaussian (Laplacian)
Edge = zero crossingof second derivative
Source: S. Seitz
Recall: Edge detection
24
![Page 25: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/25.jpg)
From edges to blobs• Edge = ripple• Blob = superposition of two ripples
Spatial selection: the magnitude of the Laplacianresponse will achieve a maximum at the center ofthe blob, provided the scale of the Laplacian is“matched” to the scale of the blob
maximum
Slide credit: Lana Lazebnik
25
![Page 26: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/26.jpg)
Blob detection in 2D
Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D
2
2
2
22
y
g
x
gg
Slide credit: Kristen Grauman26
![Page 27: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/27.jpg)
Blob detection in 2D: scale selection
Laplacian-of-Gaussian = “blob” detector2
2
2
22
y
g
x
gg
fil
ter
scal
es
img1 img2 img3Bastian Leibe
27
![Page 28: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/28.jpg)
Blob detection in 2D
We define the characteristic scale as the scale that produces peak of Laplacian response
characteristic scale
Slide credit: Lana Lazebnik
28
![Page 29: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/29.jpg)
Example
Original image at ¾ the size
Kristen Grauman 29
![Page 30: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/30.jpg)
Original image at ¾ the size
Kristen Grauman
30
![Page 31: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/31.jpg)
Kristen Grauman
31
![Page 32: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/32.jpg)
Kristen Grauman
32
![Page 33: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/33.jpg)
Kristen Grauman
33
![Page 34: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/34.jpg)
Kristen Grauman
34
![Page 35: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/35.jpg)
Kristen Grauman
35
![Page 36: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/36.jpg)
)()( yyxx LL
s1
s2
s3
s4
s5
List of (x, y, σ)
scale
Scale invariant interest points
Interest points are local maxima in both position and scale.
Squared filter response maps
Slide credit: Kristen Grauman
36
![Page 37: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/37.jpg)
Scale-space blob detector: Example
Image credit: Lana Lazebnik
37
![Page 38: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/38.jpg)
We can approximate the Laplacian with a difference of Gaussians; more efficient to implement.
2 ( , , ) ( , , )xx yyL G x y G x y
( , , ) ( , , )DoG G x y k G x y
(Laplacian)
(Difference of Gaussians)
Technical detail
Slide credit: Kristen Grauman
38
![Page 39: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/39.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description:Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
],,[ )1()1(11 dxx x
],,[ )2()2(12 dxx x
Slide credit: Kristen Grauman
39
![Page 40: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/40.jpg)
Geometric transformations
e.g. scale, translation, rotation
Slide credit: Kristen Grauman40
![Page 41: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/41.jpg)
Photometric transformations
Figure from T. Tuytelaars ECCV 2006 tutorial
Slide credit: Kristen Grauman
41
![Page 42: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/42.jpg)
Raw patches as local descriptors
The simplest way to describe the neighborhood around an interest point is to write down the list of intensities to form a feature vector.
But this is very sensitive to even small shifts, rotations.
Slide credit: Kristen Grauman42
![Page 43: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/43.jpg)
SIFT descriptor [Lowe 2004]
• Use histograms to bin pixels within sub-patches according to their orientation.
0 2p
Why subpatches?
Why does SIFT have some illumination invariance?
Slide credit: Kristen Grauman43
![Page 44: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/44.jpg)
CSE 576: Computer Vision
Making descriptor rotation invariant
Image from Matthew Brown
• Rotate patch according to its dominant gradient orientation
• This puts the patches into a canonical orientation.Slide credit: Kristen Grauman
44
![Page 45: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/45.jpg)
• Extraordinarily robust matching technique• Can handle changes in viewpoint
• Up to about 60 degree out of plane rotation• Can handle significant changes in illumination
• Sometimes even day vs. night (below)• Fast and efficient—can run in real time• Lots of code available
• http://people.csail.mit.edu/albert/ladypack/wiki/index.php/Known_implementations_of_SIFT
Steve Seitz
SIFT descriptor [Lowe 2004]
45
![Page 46: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/46.jpg)
Example
NASA Mars Rover images
Slide credit: Kristen Grauman
46
![Page 47: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/47.jpg)
NASA Mars Rover imageswith SIFT feature matchesFigure by Noah Snavely
Example
Slide credit: Kristen Grauman
47
![Page 48: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/48.jpg)
SIFT properties
• Invariant to– Scale – Rotation
• Partially invariant to– Illumination changes– Camera viewpoint– Occlusion, clutter
Slide credit: Kristen Grauman48
![Page 49: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/49.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description:Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
Slide credit: Kristen Grauman
49
![Page 50: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/50.jpg)
Matching local features
Kristen Grauman50
![Page 51: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/51.jpg)
Matching local features
?
To generate candidate matches, find patches that have the most similar appearance (e.g., lowest SSD)
Simplest approach: compare them all, take the closest (or closest k, or within a thresholded distance)
Image 1 Image 2
Kristen Grauman51
![Page 52: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/52.jpg)
Ambiguous matches
At what SSD value do we have a good match?
To add robustness to matching, can consider ratio : distance to best match / distance to second best match
If low, first match looks good.
If high, could be ambiguous match.
Image 1 Image 2
? ? ? ?
Kristen Grauman52
![Page 53: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/53.jpg)
Matching SIFT Descriptors• Nearest neighbor (Euclidean distance)• Threshold ratio of nearest to 2nd nearest descriptor
Lowe IJCV 2004Slide credit: Kristen Grauman
53
![Page 54: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/54.jpg)
Recap: robust feature-based alignment
Source: L. Lazebnik
54
![Page 55: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/55.jpg)
Recap: robust feature-based alignment
• Extract features
Source: L. Lazebnik
55
![Page 56: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/56.jpg)
Recap: robust feature-based alignment
• Extract features• Compute putative matches
Source: L. Lazebnik
56
![Page 57: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/57.jpg)
Recap: robust feature-based alignment
• Extract features• Compute putative matches• Loop:
• Hypothesize transformation T (small group of putative matches that are related by T)
Source: L. Lazebnik
57
![Page 58: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/58.jpg)
Recap: robust feature-based alignment
• Extract features• Compute putative matches• Loop:
• Hypothesize transformation T (small group of putative matches that are related by T)
• Verify transformation (search for other matches consistent with T)
Source: L. Lazebnik
58
![Page 59: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/59.jpg)
Recap: robust feature-based alignment
• Extract features• Compute putative matches• Loop:
• Hypothesize transformation T (small group of putative matches that are related by T)
• Verify transformation (search for other matches consistent with T)
Source: L. Lazebnik
59
![Page 60: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/60.jpg)
Applications of local invariant features
• Wide baseline stereo• Motion tracking• Panoramas• Mobile robot navigation• 3D reconstruction• Recognition• …
Slide credit: Kristen Grauman
60
![Page 61: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/61.jpg)
Automatic mosaicing
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
Slide credit: Kristen Grauman
61
![Page 62: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/62.jpg)
Wide baseline stereo
[Image from T. Tuytelaars ECCV 2006 tutorial]
Slide credit: Kristen Grauman 62
![Page 63: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/63.jpg)
Recognition of specific objects, scenes
Rothganger et al. 2003 Lowe 2002
Schmid and Mohr 1997 Sivic and Zisserman, 2003
Kristen Grauman
63
![Page 64: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/64.jpg)
Summary
• Interest point detection– Harris corner detector– Laplacian of Gaussian, automatic scale selection
• Invariant descriptors– Rotation according to dominant gradient direction– Histograms for robustness to small shifts and
translations (SIFT descriptor)
64
![Page 65: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/65.jpg)
Numerical Issues
• When computing H– Say true match is [50 100 1] [50 100]– [50.5 100 1]
• [50.5 100]
– [50 100 1.5]• [33 67]
– Scale co-ordinates to lie between 0 and 2
65
![Page 66: Local features: detection and description Tuesday October 8 Devi Parikh Virginia Tech Slide credit: Kristen Grauman 1 Disclaimer: Most slides have been](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5517f2df550346a2228b4696/html5/thumbnails/66.jpg)
Questions?
See you Thursday!
66
Slide credit: Devi Parikh