surf: speeded up robust features

1 aiRobots Lab., EE Dept., NCKUaiRobots Lab., EE Dept., NCKU

SURF: Speeded Up Robust Features

授課教授 : 連震杰教授

Group number: 20Advisor: Tzuu-Hseng S. Li

Group members: E24956552 何雅芳E24951099 蕭信揚N26984224 李佳樺

aiRobots Laboratory, Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, R.O.C.


Outline

• Introduction• Related Work • Fast-Hessian Detector• SURF Descriptor

– Orientation Assignment– Descriptor Components

• Matching• Experiments• Conclusion


Outline





Introduction

• The task of finding point correspondences between two images of the same scene or object is part of many computer vision applications.

• This article presents a novel scale- and rotation-invariant detector and descriptor, coined SURF (Speeded-Up Robust Features).

• SURF approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.


Introduction (cont’d)

• The search for discrete image point correspondences can be divided into three main steps.

Step1. DetectorInterest points are selected

Step2. DescriptorExtract the vector for matching

Step3. MatchOften based on a distance

between the vector

Most valuable property:Repeatability

(whether it reliably finds the same interest points under different viewing condition.)

Focus on scale and

image rotation invariant.


Outline





Related Work

• Interest Point DetectionHarris corner detector ‧most widely used

‧based on the eigenvalues

‧not scale-invariant

Automatic scale selection detector

‧experimented both the determinant of the

Hessian matrix as well as Laplacian.

Scale-invariant feature detectors

(Mikolajczyk , Schmid)

‧Harris-Laplace and Hessian-Laplace

‧The location is selected by the determinant

of Hessian matrix.

‧The scale is selected by the Laplacian.

SIFT ‧Approximated the LoG by a DoG filter.

=> (1) Using the determinant of the Hessian matrix rather than its trace (the Laplacian) seems advantageous, (2) approximations like the DoG can bring speed at a low cost in terms of lost accuracy.


Related Work (cont’d)

• Interest Point DescriptionSIFT ‧computes a histogram of local oriented gradients around

the interest point and stores the bins in a 128-dimensional

vector.

PCA-SIFT ‧Yields a 36-dimensional descriptor (=>Fast)

‧To be less distinctive than SIFT

GLOH ‧More distinctive with the same number of dimensions.

‧Computationally more expensive.

=> The SIFT descriptor still seems to be the most appealing descriptor for practical uses, and hence also the most widely used nowadays.


Related Work (cont’d)

• Our approach

Step1. Fast-Hessian detectorBased on the Hessian matrix but

use a very basic approximation – DoG

Step2. SURF DescriptorDescribes a distribution of Haar-wavelet

Responses within the interest point neighborhood

Step3. MatchPresent a new indexing step based on

the sign of the Laplacian(Speed up & increase the robustness)

+Integral image

(reduce the computation time)

+Integral image

(reduce the computation time)

Integral image:),(x, ),()x(

0 0yxjiII

xi

i

yj

j

★Question:Why can this

methodreduce the

computationtime?

DBCA

(1)Fast implementation of box type convolution filters(2)Independent of its size

A

B

C

DB-D

C-D Σ

Property…

(x,y)

)x(I

Answer…


Outline





Fast-Hessian Detector

• Hessian matrix H( x , σ) in x at scale σ is defined as

• Approximation LoG with box filters => DoG

),x( ),x(

),x( ),x(),x(

yyxy

xyxx

LL

LLH

)x()(),x(,2

2

Igx

Lxx

Gaussian second order derivative

x-dir y-dir xy-dir

Gaussian) ofdeviation dradscale(Stan:

),(x

yx

),x(),x(

)x())()((),x(2

2

2

2

xxxx

xx

LkL

Igx

kgx

D

Box filters (instead of Gaussian)

x-dir y-dir xy-dir

9x9 box filter with σ=1.22

approx )9.0()det( xyyyxx DDDH


• The scale space is analysed by up-scaling the filter size rather than iteratively reducing the image size.

• The scale space is divided into octaves. An octave represents a series of increasing filter response maps.

Fast-Hessian Detector (cont’d)sc

ale

9 x 9 (σ=1.2)

15 x 15 (σ=2.0)

21 x 21 (σ=2.8)

27 x 27 (σ=3.6)

Octave1 (increase:6)

For each new octave, the filter size increase is doubled.(going from 6 to 12 to 24…)

It is selected as the interest point only if it is larger thanall of these neighbors.

15 x 15 (σ=2.0)

27 x 27 (σ=2.8)

39 x 39 (σ=5.2)

51 x 51 (σ=6.8)

Octave2 (increase:12)


Q & A (Fast-Hessian Detector)

• Question1. 以放大 filter 的 size 代替將圖片縮小，有什麼好處 ?

• Answer1. 因為 integral image 的使用，使得計算量不會隨 filter 的 size 增加，且沒有將圖片縮小，圖片就不會失真。


Outline





SURF Descriptor

Orientation Assignment

Fixing a reproducible orientation based on information from acircular region around the interest point.

Descriptor Components

Construct a square region alignedto the selected orientation, andextract the SURF descriptor from it.

Interestpoint Features


Outline





Orientation Assignment

• Haar Wavelet

• Orientation

dx

dy

4s

a-b

a

b

6s

Image

Interest point

dx

dy

π/3(dx1,dy1)

(dx2,dy2)Orientation

ABC

DEF

=A-B-D+E

=B-C-E+F

Σ=-A+2B-C+D-2E+F


Outline





Descriptor Components

• Constructing a square region centered around the interest point, and oriented along the orientation.

• The region is split up regularly into smaller 4 × 4 square sub-regions.

• (4x4)x4=> a 64 dimensional vector

20s

Horizontal direction

Vertical direction

),,,(v yxyx dddd

Haar wavelet(filter size 2s)


• Question1. Why to use Σ|dx| and Σ|dy| ?• Answer1.

• Question2. Why to use Haar wavelet response?• Answer2.

Q&A(SURF Descriptor)


Outline





Matching

• For fast indexing during the matching stage, the sign of the Laplacian (i.e. the trace of the Hessian matrix) for the underlying interest point is included.

• In the matching stage, we only compare features if they have the same type of contrast.

yyxx DDH )(Tr


Outline





Experiments

• Scale variant + rotation


Experiments (cont’d)

• Rotation



• Blurred



• Photometric deformations


Outline





Conclusion

• SURF outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.

• Future work will aim at optimizing the code for additional speed up.


★ Thanks for your attention!!

surf: speeded up robust features

Documents