![Page 1: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/1.jpg)
Jifeng Dai
2011/09/27
![Page 2: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/2.jpg)
Introduction
Structural SVM
Kernel Design
Segmentation and parameter learning
Object Feature Descriptors
Experimental results
Conclusions and Future Work
![Page 3: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/3.jpg)
CVPR 2011 Oral
![Page 4: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/4.jpg)
Things to do:
![Page 5: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/5.jpg)
Contributions:
1) Propose a kernelized structural support vector machine approach to learn discriminatively the mapping from image to a segmentation mask.
2) Combine high level object similarity information with multiple low level segmentation cues into a novel kernel.
3) Traditional segmentation regularizations are preserved in the framework and explicitly enforced during the learning process. This way smoothness of the solution does not need to be “re-learned” from training examples.
![Page 6: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/6.jpg)
Complex output
The dog chased the catxS VPNP
Det NV
NP
Det N
y2
S VPVP
Det NV
NP
V N
y1
S
NPVP
Det NV
NP
Det N
yk
…
![Page 7: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/7.jpg)
Training Examples:
Hypothesis Space:
The dog chased the catx
S VPNP
Det NV
NP
Det N
y1
S VPVP
Det NV
NP
V N
y2
S
NPVP
Det NV
NP
Det N
y58
S VPNP
Det NV
NP
Det N
y12
S VPNP
Det NV
NP
Det N
y34
S VPNP
Det NV
NP
Det N
y4
Training: Find that solve
Problems• How to predict efficiently?• How to learn efficiently?• Manageable number of parameters?
![Page 8: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/8.jpg)
The idea behind Structured SVM is to discriminatively learn a scoring function over input/output pairs (i.e. over image/mask pairs).
![Page 9: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/9.jpg)
Loss function:
Two important choices:1) Restrict the search to Ys, subset of Y
composed by smooth segmentation masks.
![Page 10: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/10.jpg)
Two important choices:1) Restrict the search to Ys, subset of Y
composed by smooth segmentation masks.
2) using kernel functions so that we could work in the dual formulation.
![Page 11: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/11.jpg)
HOG…
Object Similarity KernelMask Similarity Kernel
![Page 12: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/12.jpg)
Mask Similarity Kernel
1) Shape Kernel
2) Local Color Model Kernel
3) Global Color Model Kernel
![Page 13: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/13.jpg)
Graph cuts
Mask smooth term
In which
So (6) and (7) take the form:
Graph cuts!!!
![Page 14: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/14.jpg)
Parameters are optimized on a validation set
![Page 15: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/15.jpg)
HOG grid or detector response feature
![Page 16: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/16.jpg)
Datasets:
1)the Dresses dataset (600 images)2)the Weizmann horses dataset (328 images)3)the Oxford 17 category flower dataset (849
images)
![Page 17: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/17.jpg)
How to measure performance?
![Page 18: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/18.jpg)
Comparison with previous works:
![Page 19: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/19.jpg)
Comparison with previous works:
![Page 20: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/20.jpg)
Comparison with previous works: Oxford Flower Dataset
Previous work:
![Page 21: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/21.jpg)
Examples:
![Page 22: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/22.jpg)
Examples:
![Page 23: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/23.jpg)
Examples:
![Page 24: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/24.jpg)
Contributions:
1) Propose a kernelized structural support vector machine approach to learn discriminatively the mapping from image to a segmentation mask.2) Combine high level object similarity information with multiple low level segmentation cues into a novel kernel.3) Traditional segmentation regularizations are preserved in the framework and explicitly enforced during the learning process. This way smoothness of the solution does not need to be “re-learned” from training examples.
![Page 25: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/25.jpg)
Future Work:
1)Model the boundary curves (driven by low-level cues).
2) Instead of relying on a single global object similarity kernel, dividing the kernel into a parts-based representation.
3) Establish a theoretical connection between the complexity of the top-down models the algorithm can learn and the number of segmentations needed in the training set.
![Page 26: Jifeng Dai 2011/09/27. Introduction Structural SVM Kernel Design Segmentation and parameter learning Object Feature Descriptors Experimental](https://reader034.vdocuments.mx/reader034/viewer/2022051315/56649e935503460f94b98aea/html5/thumbnails/26.jpg)