Outline SIFT Speeded Up Robust Feature -SURF Conclusion
SIFT: Scale Invariant Feature Transform
Ahmed Othman
Systems Design DepartmentUniversity of Waterloo, Canada
October, 23, 2012
1 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
1 SIFTIntroductionScale-space extrema detectionKeypoint localizationOrientation assignmentKeypoint descriptor
2 Speeded Up Robust Feature -SURF
3 Conclusion
2 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Introduction
Objectives
Extract different features from object image.These features are:
Invariant to image scale and rotation.Used to recognize this object in different images.
3 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Introduction
SIFT Output
Testing Data
4 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Introduction
SIFT Output
Matching Process.
5 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Introduction
SIFT Output
Matching Process.6 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Introduction
SIFT Steps
SIFT is consist of four stages.1 Scale-space extrema detection:
Search over all scales of the image.Detect candidate key points.
2 Keypoint localization.Measure the stability of the candidate point.
3 Orientation assignment.Image gradient directions are used to assign orientations tothe keypoint locations
4 Keypoint descriptor.Each point is represented with a descriptor vector.
7 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Scale-space extrema detection
Scale-space extrema detection
Detecting key points that are invariant to different scales ofthe image.Use Gaussian filter to detect the candidate points.Define the scale space of an image L(x , y ,�) as:
L(x , y ,�) = G(x , y ,�) ⇤ I(x , y), (1)
G(x , y ,�) =1
2⇡�2 exp�(x2+y2)/2�2(2)
Difference of Gaussian function D(x , y ,�) are used todetect stable points by convolving to the image.
D(x , y ,�) = (G(x , y , k�)� G(x , y ,�)) ⇤ I(x , y) (3)
D(x , y ,�) = L(x , y , k�)� L(x , y ,�). (4)
k is a constant multiplicative factor.8 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Scale-space extrema detection
Difference of Gaussians.
9 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Scale-space extrema detection
Scale-space extrema detection
Every point is compared to:1 Its eight neighbors in 3 ⇥ 3 map.2 Its nine neighbors in the scale above and below.
The point is selected if :1 Larger than the other 26 points (local maxima).2 Less than the other 26 points (local minima).
Key point detection.
10 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization
The candidate points are revisited and outliers arediscarded.Two types of points are Discarded:
1 Points with low contrast.2 points that poorly localized along an edge.
Use the Taylor expansion of the scale space D(x , y ,�).The sample point is in the origin.
11 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization - Low contrast points removal
The Taylor series of the scale space is defined as:
D(x) = D +@DT
@xx +
12
xT @2D@x2 x (5)
The location of the extremum x̂ is determined as:
x̂ = �@2D�1@x2
@D@x
(6)
The Taylor value at the extremum is:
D(x̂) = D +12@DT@xx̂ (7)
Points that its |D(x)| < 0.03 are discarded.
12 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization - Poor edge point removal
DOG have strong response along the edge.The location along the edge may be poorly determined.The poor point in the DOG have:
1 A large principal curvature across the edge.2 A small principal curvature in the perpendicular direction.
A thresholded principle curvature (PC) is used to removepoor edge points.
13 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization -Poor edge point removal
PC computed from 2 ⇥ 2 Hessian matrix:
H =
Dxx DxyDxy Dyy
�(8)
The trace Tr(H) and the determinant Det(H)are computed.
Tr(H) = Dxx + Dyy = ↵+ �, (9)
Det(H) = DxxDyy � (Dxy )2 = ↵�. (10)
↵ the largest magnitude eignvalue � is the lowest one.
14 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization -Poor edge point removal
let r be:r =
↵
�. (11)
The ratio of the principle curvature is checked by checking :
Tr(H)2
Det(H)2 <(r + 1)2
r. (12)
Points with r greater than 10 are discarded.
15 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint localization
Keypoint localization
Keypoint localization
16 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Orientation assignment
Orientation assignment
Based on image properties, an orientation is assigned toeach keypoint.This makes the descriptor invariant to image rotation.The gradient magnitude m(x , y) and the orientation ✓(x , y)of every smoothed image L(x , y) are used.
m(x, y) =p
(L(x + 1, y) � L(x � 1, y))2 + (L(x, y + 1) � L(x, y � 1))2)) (13)
✓(x, y) = tan� 1(((L(x, y + 1) � L(x, y � 1)))/(L(x + 1, y) � L(x � 1, y))) (14)
17 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Orientation assignment
Orientation assignment
The process is as follow:1 A 36 (for 360 degree) bins histogram is built from the
orientation of the points within a neighbiours of the keypoints.
2 Sample added to the histogram weighted by:Gradient magnitude,A Gaussian filter with � equal 1.5 times that of the scale ofthe keypoint.
3 The highest peak and the 80% of the highest peak aredetected.
Orientation Assignement.
18 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Orientation assignment
Orientation assignment
Orientation Assignement.
19 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
Keypoint descriptor
Keypoint descriptor
Use the orientation histogram of the previous step.Look at 16 ⇥ 16 around the key point.Divide 16 ⇥ 16 region into four 4 ⇥ 4 blocks 8 bins each.For each block compute a histogram of gradient magnitudeand orientation.The descriptor vector of size 128 is formed for every point.
SIFT Descriptors.20 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
SURF- Speeded Up Robust Feature
Key points are detected using Hessian matrix.A descriptor vector is built describing the neighbourhood ofevery keypointMatching process is performed based on the descriptorvector.
21 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
SURF- Descriptor vector
Orientation assignment is done using Haar waveletresponse in x and y directions.A 20s rectangle is generated around each keypoint.The rectangle is splitted into 4 ⇥ 4 subregions.
22 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
SURF- Descriptor vector
For each subregion:the Haar wavelet response in horizontal and verticaldirection dx , and dy .dx , and dy are summed over each subregion.The sum of the absolute value of the responses |dx | and|dy | are calculated.
A four dimension vector [dxdy |dx ||dy |] is extracted.
Each sub region have 4 vectors come up to 64.Matching process is performed based on the descriptorvector.
23 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
SIFT - SURF
Keypoints are detected.Different orientations are assigned around the keypoints.A set of discriptors are assigned to the key points.These descriptors are used for object recognition.
24 / 25
Outline SIFT Speeded Up Robust Feature -SURF Conclusion
References
Distinctive image features from scale-invariant keypoints.by David Lowe,International journal of computer visionvolume 60, pages (91–110), 2004.Vedio lecture by Dr. Mubarak Shah, Professor of ComputerScience at university of Central Florida, USA.Google images for SIFT.SURF: Speeded Up Robust Features.Herbert Bay, Tinne Tuytelaars, and Luc Van GoolComputer Vision 2006.
25 / 25