Interest Point Detection

Lecture-4

Local features: main components1) Detection: Identify the

interest points

2) Description :Extract feature vector descriptor surrounding each interest point.

3) Matching: Determine correspondence between descriptors in two views

],,[ )1()1(11 dxx x

],,[ )2()2(12 dxx x

Kristen Grauman

Where can we use it? Automate object tracking Point matching for computing disparity Stereo calibration

Estimation of fundamental matrix Motion based segmentation Recognition 3D object reconstruction Robot navigation Image retrieval and indexing

Goal: interest operator repeatability

• We want to detect (at least some of) the same points in both images.

• Yet we have to be able to run the detection procedure independently per image.

No chance to find true matches!

Kristen Grauman

Goal: descriptor distinctiveness

• We want to be able to reliably determine which point goes with which.

• Must provide some invariance to geometricand photometric differences between the two views.

?

Kristen Grauman

Some patches can be localizedor matched with higher accuracy than

others.

Local features: main components1) Detection: Identify the

interest points

2) Description:Extract vector feature descriptor surrounding each interest point.

3) Matching: Determine correspondence between descriptors in two views

Kristen Grauman

What is an interest point Expressive texture

The point at which the direction of the boundary of object changes abruptly

Intersection point between two or more edge segments

Synthetic & Real Interest Points

Corners are indicated in red

Properties of Interest Point Detectors Detect all (or most) true interest points No false interest points Well localized. Robust with respect to noise. Efficient detection

Possible Approaches to Corner Detection Based on brightness of images

Usually image derivatives Based on boundary extraction

First step edge detection Curvature analysis of edges

Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give

a large change in intensity

C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988

Basic Idea

“flat” region:no change in all directions

“edge”:no change along the edge direction

“corner”:significant change in all directions

Aperture Problem

Correlation

k lljkihlkfhf ,,

KernelImage

hf

h1 h2 h3

h4 h5 h6

h7 h8 h9

hf1 f2 f3

f4 f5 f6

f7 f8 f9

998877

665544

332211

*

hfhfhfhfhfhf

hfhfhfhf

f

Correlation

k lljkihlkfhf ,, Cross correlation

k l

ljkiflkfff ,, Auto correlation

Correlation Vs SSD 2,,

k l

ljkihlkfSSD Sum of Squares Difference

k l

ljkiflkfff ,,

k l

ljkihlkfljkihlkfSSD 22 ,,,2,

minimize

minimize

maximize

k l

lkfljkihSSD ,,2

k l

lkfljkihnCorrelatio ,,maximize

k l

lkfljkihSSD ,,2

Mathematics of Harris Detector Change of intensity for the shift (u,v)

yx

yxIvyuxIyxwvuE,

2

intensityintensity shiftedfunctionwindow

]),(),([),(),(

Window functions

Auto-correlation

UNIFORM GAUSSIAN

Auto-Correlation

Taylor Series

ƒ(x) Can be represented at point a in terms of its derivatives

Mathematics of Harris Detector

yx

yx vIuIyxwvuE,

2])[,(),(

yx y

x

II

vuyxwvuE,

2

),(),(

yxyx

y

x

vu

IIII

vuyxwvuE,

),(),(

v

uII

II

yxwvuvuEyx

yxy

x

,),(),(

yx

yxIvyuxIyxwvuE,

2

intensityintensity shiftedfunctionwindow

]),(),([),(),(

yx

yx yxIvIuIyxIyxwvuE,

2

intensityintensity shiftedfunction window

]),(),([),(),(

Taylor Series

vu

MvuvuE ),(

Mathematics of Harris Detector

E(u,v) is an equation of an ellipse, where M is the covariance

Let 1 and 2 be eigenvalues of M

vu

MvuvuE ),(

yx yyyx

yxxx

IIIIIIII

yxwM,

),(

direction of the slowest change

direction of the fastest change

(max)-1/2

(min)-1/2

Eigen Vectors and Eigen ValuesThe eigen vector, x, of a matrix A is a special vector, with the following property

Where is called eigen value

To find eigen values of a matrix A first find the roots of:

Then solve the following linear system for each eigen value to find corresponding eigen vector

Example

Eigen Values

Eigen Vectors

Eigen Values

Eigen Vectors

Mathematics of Harris Detector

1

2

“Corner”1 and 2 are large,1 ~ 2;E increases in all directions

1 and 2 are small;E is almost constant in all directions

“Edge” 1 >> 2

“Edge” 2 >> 1

“Flat” region

Classification of image points using eigenvalues of M:

Mathematics of Harris Detector Measure of cornerness in terms of 1, 2

2det traceMkMR 22121 kR

2

1

00

M

Mathematics of Harris Detector

1

2 “Corner”

“Edge”

“Edge”

“Flat”

• R depends only on eigenvalues of M

• R is large for a corner

• R is negative with large magnitude for an edge

• |R| is small for a flatregion

R > 0

R < 0

R < 0|R| small

Compute corner response

Find points with large corner response: R> thresholdFind points with large corner response: R> threshold

Take only the points of local maxima of R

If pixel value is greater than its neighbors then it is a local maxima.

Other Version of Harris Detectors

21 R

21

21

1)()det(

Mtrace

MR

Triggs

Szeliski (Harmonic mean)

1R Shi-Tomasi

1

2

Algorithm

Reading Material Section 4.1.1 Feature Detectors

Richard Szeliski, "Computer Vision: Algorithms and Application“, Springer.