fast intersection kernel svms for realtime object detection joint work with: alex berg (columbia...
TRANSCRIPT
![Page 1: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/1.jpg)
Fast intersection kernel SVMs for Realtime Object Detection
Joint work with: Alex Berg (Columbia University & UC Berkeley)
and Jitendra Malik (UC Berkeley)
Subhransu Maji UC Berkeley
![Page 2: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/2.jpg)
Fast intersection kernel SVMs for Realtime Object Detection
- IKSVM is a (simple) generalization of a linear SVM- Can be evaluated very efficiently (sublinear in #SV)- Other kernels (including ) have a similar form - Methods applicable to current most successful object recognition/detection strategies.
Maji, Berg & Malik, CVPR 2008
![Page 3: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/3.jpg)
Detection: Is this an X?
Ask this question over and over again,varying position, scale, category, pose…Speedups: hierarchical, early reject, feature sharing, cueingbut same underlying question!
![Page 4: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/4.jpg)
Detection: Is this an X?
Ask this question over and over again,varying position, scale, multiple categories…Speedups: hierarchical, early reject, feature sharing,but same underlying question!
![Page 5: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/5.jpg)
Detection: Is this an X?
Ask this question over and over again,varying position, scale, multiple categories…Speedups: hierarchical, early reject, feature sharing,but same underlying question!
Boosted dec. trees, cascades + Very fast evaluation - Slow training (esp. multi-class)Linear SVM + Fast evaluation + Fast training - Need to find good featuresNon-linear kernelized SVM + Better class. acc. than linear - Medium training - Slow evaluation
![Page 6: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/6.jpg)
Detection: Is this an X?
Ask this question over and over again,varying position, scale, multiple categories…Speedups: hierarchical, early reject, feature sharing,but same underlying question!
Boosted dec. trees, cascades + Very fast evaluation - Slow training (esp. multi-class)Linear SVM + Fast evaluation + Fast training - Need to find good featuresNon-linear kernelized SVM + Better class. acc. than linear - Medium training - Slow evaluation
This work
![Page 7: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/7.jpg)
Outline
What is Intersection Kernel SVM? Brief Overview of Support Vector Machines Multi-scale features based on Oriented Energy
Algorithms Algorithm to make classification fast (exact) Algorithm to make classification very fast (approximate)
Experimental Results Summary of where this matters
![Page 8: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/8.jpg)
Outline
What is Intersection Kernel SVM? Brief Overview of Support Vector Machines Multi-scale features based on Oriented Energy
Algorithms Algorithm to make classification fast (exact) Algorithm to make classification very fast (approximate)
Experimental Results Summary of where this matters
![Page 9: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/9.jpg)
B1
b11
b12
0 bxw
1 bxw 1 bxw
1bxw if1
1bxw if1)(
xf 2||||
2 Margin
w
Examples are;
(x1,..,xn,y) with
y{-1.1}
Support Vector Machines
![Page 10: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/10.jpg)
Kernel Support Vector Machines
Kernel Function• Inner Product in Hilbert Space• Learn Non Linear Boundaries
Gaussian Kernel
Classification Function
![Page 11: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/11.jpg)
Feature Representation
Discriminative Classifier
(+ examples) (- examples)
Training Stage
![Page 12: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/12.jpg)
Multiscale Oriented Energy feature
Concatenate orientation histograms for each orange region.Differences from HOG: -- Hierarchy of regions -- Only performing L1 normalization once (at 16x16)
![Page 13: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/13.jpg)
What is the Intersection Kernel?
Histogram Intersection kernel between histograms a, b
![Page 14: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/14.jpg)
What is the Intersection Kernel?
Histogram Intersection kernel between histograms a, b
K small -> a, b are differentK large -> a, b are similar
Intro. by Swain and Ballard 1991 to compare color histograms.Odone et al 2005 proved positive definiteness.Can be used directly as a kernel for an SVM.Compare toGeneralizations: Pyramid Match Kernel (Grauman et. al.), Spatial Pyramid Match Kernel (Lazebnik et.al)
![Page 15: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/15.jpg)
Linear SVM, Kernelized SVM, IKSVM
Decision function is where:
Linear:
Non-linearUsingKernel
HistogramIntersectionKernel
![Page 16: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/16.jpg)
Kernelized SVMs slow to evaluate
Arbitrary Kernel
HistogramIntersectionKernel
Feature corresponding to a support vector l
Feature vector to evaluate
Kernel EvaluationSum over all support vectors
SVM with Kernel Cost: # Support Vectors x Cost of kernel comp.IKSVM Cost: # Support Vectors x # feature dimensions
Decision function is where:
![Page 17: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/17.jpg)
Algorithm 1
Decision function is where:
Just sort the support vectorvalues in each coordinate, andpre-compute
To evaluate, find position ofin the sorted support vectorvalues (cost: log #sv)look up values, multiply & add
![Page 18: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/18.jpg)
Algorithm 1
Decision function is where:
Just sort the support vectorvalues in each coordinate, andpre-compute
To evaluate, find position ofin the sorted support vectorvalues (cost: log #sv)look up values, multiply & add
#support vectors x #dimensions
log( #support vectors ) x #dimensions
![Page 19: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/19.jpg)
Algorithm 2
Decision function is where:
#support vectors x #dimensionslog( #support vectors ) x #dimensions
For IK hi is piecewise linear, and quite smooth, blue plot. We can approximate with fewer uniformly spaced segments, red
plot. Saves time & space!
![Page 20: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/20.jpg)
Algorithm 2
Decision function is where:
#support vectors x #dimensionslog( #support vectors ) x #dimensions
constant x #dimensions
For IK hi is piecewise linear, and quite smooth, blue plot. We can approximate with fewer uniformly spaced segments, red
plot. Saves time & space!
![Page 21: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/21.jpg)
Toy Example : accuracy/runtime vs. #bins
•Runtime independent of #bins (on left)•Accuracy improves with #bins (on right)
![Page 22: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/22.jpg)
•Runtime independent of #sup vec! (for approximate)•2-3 orders of magnitude faster than LibSVM.•Runtime memory requirement independent of #sup vec!
Toy Example : accuracy/runtime vs. #sup vec
![Page 23: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/23.jpg)
Results - INRIA Pedestrian Dataset
•Outperforms linear significantly using pHOG features.•About 3-4x slower than linear SVM. Most time spent on computing features anyway.•IKSVM on HOG beats linear on HOG (not shown in the table)
![Page 24: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/24.jpg)
Errors
![Page 25: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/25.jpg)
Results - DC Pedestrians/Caltech-101
![Page 26: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/26.jpg)
Results - Single Scale UIUC Cars
![Page 27: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/27.jpg)
Results – ETHZ DatasetDataset: Ferrari et al., ECCV 2006 255 images, over 5 classes training = half of positive images for a class + same number from the other classes (1/4 from each) testing = all other images large scale changes; extensive clutter
![Page 28: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/28.jpg)
Method Applelogo Bottle Giraffe Mug Swan Avg
PAS* 65.0 89.3 72.3 80.6 64.7 76.7
Our 86.1 81.0 62.1 78.0 100 81.4
Beats many current techniques without any changes to our features/classification framework.
Shape is an important cue (use Pb instead of OE) Recall at 0.3 False Positive per Image (shown
below)
Results – ETHZ Dataset
*Ferarri et.al, IEEE PAMI - 08
![Page 29: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/29.jpg)
Other kernels allow similar trick
Decision function is where:
IKSVM SVM
hi not piece-wise linear,but we can still use anapproximation for fastevaluation.
hi are piece-wise linear,uniformly spacedpiece-wise linear approx.is fast.
![Page 30: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/30.jpg)
Results outside computer vision
Accuracy of IK vs Linear on Text classification
Error rate of directly + iksvm (blue) + best kernel (green) + linear (red) on SVM benchmark datasets
![Page 31: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/31.jpg)
Conclusions Exact evaluation in O(log #SV), approx in O(1) (same as linear!) Runtime for approximate is O(1) (same as linear!) Significantly outperforms linear on variety of vision/non vision
datasets Technique applies to any additive kernel (e.g. pyramid match kernel,
spatial pyramid match kernel, –chi^2, etc) Represents some of the best Caltech 256, Pascal VOC 2007
methods. Training time is much worse compared to linear (Dual coordinate
descent, PEGASOS) Inside news! Train Additive Kernel SVMs quickly using online
stochastic gradient descent. Trains IKSVM based INRIA pedestrian detector ~50K feats of 4K dim
in 100s. (compared to 3-4hours using LibSVM).
![Page 32: Fast intersection kernel SVMs for Realtime Object Detection Joint work with: Alex Berg (Columbia University & UC Berkeley) and Jitendra Malik (UC Berkeley)](https://reader036.vdocuments.mx/reader036/viewer/2022062407/56649d1e5503460f949f208d/html5/thumbnails/32.jpg)
Thank You!