cs 556: computer vision lecture 3 - college of...
TRANSCRIPT
CS 556: Computer Vision
Lecture 3
Prof. Sinisa Todorovic
1
Interest Points
• Harris corners
• Hessian points
• SIFT Difference-of-Gaussians
• SURF
2
Properties of Interest Points
• Locality -- robust to occlusion, noise
• Saliency -- rich visual cue
• Stable under affine transforms
• Distinctiveness -- differ across distinct objects
• Efficiency -- easy to compute3
Example of Detecting Harris Corners
4
Harris Corner Detector
5
homogeneous region⇓
no change in all directions
edge⇓
no change along the edge
corner⇓
change in all directions
Source: Frolova, Simakov, Weizmann Institute
scanning window
Harris Corner Detector
6
2D convolutionSource: Frolova, Simakov, Weizmann Institute
E(x, y) = w(x, y) ⇤ [I(x + u, y + v)� I(x, y)]2
Harris Corner Detector
7
change weighting window shifted image original image
Source: Frolova, Simakov, Weizmann Institute
E(x, y) =X
m,n
w(m, n)[I(x + u + m, y + v + n)� I(x + m, y + n)]2
Harris Detector Example
8
input image
E(x, y)I(x, y)
2D map of changes
Harris Corner Detector
9
image derivatives along x and y axes
Taylor series expansion
For small shiftsu � 0v � 0
I(x + u, y + v) ⇡ I(x, y) +@I
@x
u +@I
@y
v
I(x + u, y + v) ⇡ I(x, y) + [Ix
I
y
]
u
v
�
Harris Corner Detector
10
E(x, y) = w(x, y) ⇤✓
I(x, y) + [Ix
I
y
]
u
v
�� I(x, y)
◆2
= w(x, y) ⇤✓
[Ix
I
y
]
u
v
�◆2
= w(x, y) ⇤✓
[Ix
I
y
]
u
v
�◆T ✓[I
x
I
y
]
u
v
�◆
Harris Corner Detector
11
E(x, y) = w(x, y) ⇤ [u v]
I
x
I
y
�[I
x
I
y
]
u
v
�
= [u v]✓
w(x, y) ⇤
I
2x
(x, y) I
x
(x, y)Iy
(x, y)I
x
(x, y)Iy
(x, y) I
2y
(x, y)
�◆
| {z }M(x,y)
u
v
�
Weighted Image Gradient
12
Image is discrete ⇒ Gradient is approximate
We always find the gradient of the kernel !
w(x, y;�) ⇤ I
x
(x, y) = w
x
(x, y;�) ⇤ I(x, y)
w(x, y;�) ⇤ Iy(x, y) = wy(x, y;�) ⇤ I(x, y)
Harris Corner Detector
13
E(x, y) = [u v]M(x, y)
u
v
�
M(x, y) =
(wx
⇤ I)2 (wx
⇤ I)(wy
⇤ I)(w
x
⇤ I)(wy
⇤ I) (wy
⇤ I)2
�
(x,y)
Eigenvalues and Eigenvectors of M
• Eigenvectors of M -- directions of the largest change of E(x,y)
• Eigenvalues of M -- the amount of change along eigenvectors14
��1
��2
E(x, y) = [u v]M(x, y)
u
v
�
Detection of Harris Corners using the Eigenvalues of M
15
�2
�1
Harris Detector
16
f(�) =�1(�)�2(�)
�1(�) + �2(�)=
det(M(�))trace(M(�))
objective function
Harris Detector Example
17
input image
Harris Detector Example
18
f values color codedred = high valuesblue = low values
Harris Detector Example
19
f values thresholded
Harris Detector Example
20
Harris features = Spatial maxima of f values
Example of Detecting Harris Corners
21
22
Hessian determinantHessian determinant
!"
#$%
&=
yyxy
xyxx
IIII
IHessian )(
2))(det( xyyyxx IIIIHessian −=
Ixx
Ixy
Hessian detector (Beaudet, 1978)Hessian detector (Beaudet, 1978)
Iyy
Source: Tuytelaars
Hessian Detector
23
E(x, y) = [u v]H(x, y)
uv
�
H(x, y;�) = w(x, y;�) ⇤
Ixx
(x, y) Ixy
(x, y)Ixy
(x, y) Iyy
(x, y)
�
H(x, y;�) =
wxx
(x, y;�) ⇤ I(x, y) wxy
(x, y;�) ⇤ I(x, y)w
xy
(x, y;�) ⇤ I(x, y) wyy
(x, y;�) ⇤ I(x, y)
�
Hessian Detector
24
objective function
f(�) =�1(�)�2(�)
�1(�) + �2(�)=
det(H(�))trace(H(�))
Properties of Harris Corners
• Invariance to variations of imaging parameters:
• Illumination?
• Camera distance, i.e., scale ?
• Camera viewpoint, i.e., affine transformation?
25
Harris/Hessian Detector is NOT Scale Invariant
26 Source: L. Fei-Fei
“edge”
“edge”“edge”
“corner”
27
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
28
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
29
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
30
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
31
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
32
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
Source: Tuytelaars
33
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
34
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
35
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
36
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
37
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
38
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σxIfmii "
�
Source: Tuytelaars
39
Automatic scale selectionAutomatic scale selection
)),((1
σxIfmii�
Function responses for increasing scale Scale trace (signature)
)),((1
σ ""xIfmii� Source: Tuytelaars
Popular Kernels: Difference of Gaussians
40
DoG(�) = G(x, y; k�)�G(x, y;�)
G(x, y;�) =
1p2⇡�
exp
✓�x
2+ y
2
2�
2
◆
Difference ofGaussians
Gaussian
w(�) ⇤ I
Selected Scales = Extrema of DoG/Laplacian
• Convolve the image with Gaussians whose sigma increases
• Then, subsample, and repeat the convolutions
• Finally, find extrema in the 3D DoG or Laplacian space41 Source: D. Lowe
SIFT Detector
• Find a local maximum of
• simultaneously in:
• 2D space of the image
• Scale
42
scale
x
y
← DoG →
← D
oG →
E(x, y;�) = DOG(x, y;�) ⇤ I(x, y)
SURF Detector
43
E(x, y) = [u v]H(x, y)
uv
�
Hessian matrix
H(x, y;�) =
wxx
(x, y;�) ⇤ I(x, y) wxy
(x, y;�) ⇤ I(x, y)w
xy
(x, y;�) ⇤ I(x, y) wyy
(x, y;�) ⇤ I(x, y)
�
implemented using Haar wavelets