Beyond Bags of Features: Spatial Pyramid Matching for Recognizing

Natural Scene Categories

Svetlana Lazebnik, Cordelia Schmid, Jean Ponce

Presented by: Lubomir Bourdev

Many of the slides by: Svetlana Lazebnik

Key Idea

• Pyramid Match Kernel (Grauman & Darrell)Pyramid in feature space, ignore location

• Spatial Pyramid (this work)Pyramid in image space, quantize features

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

Weak (edge orientations) Strong (SIFT)

OR

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

- Vector quantization

- Usually K-means clustering

- Vocabulary size (16 to 400)

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

Algorithm1. Extract interest point descriptors (dense scan)

2. Construct visual word dictionary

3. Build spatial histograms

4. Create intersection kernels

5. Train an SVM

My experiment: Butterfly Classification

Peacock Zebra

Butterflies

• Dataset from Lazebnik / Schmid / Ponce

70 train / 64 test

50 train / 41 test

• Images centered on the butterfly• Significant background clutter• Large pose/viewpoint variations • Scale variations: up to x4

Butterfly Results

Linear Intersection

Weak (16) 82.6% 82.6%

Strong (200) 81.9% 89.5%

Dims

16

200

Linear Intersection

Weak (16) 88.6% 86.7%

Strong (200) 84.8% 89.5%

Dims

1360

17000

Spatial pyramid levels: 1 (No pyramid)

Spatial pyramid levels: 4