Object Segmentation Based on Multiple Features Fusion and Conditional Random Field
CASIA_IGIT
National Laboratory of Pattern Recognition(NLPR)
Institute of Automation, Chinese Academy of Sciences(CASIA)
Reporter: Kun Ding(丁昆)
2013.10.17
Outline• System Overview
• System Characteristics
•Results and Conclusions
Outline• System Overview
• System Characteristics
•Results and Conclusions
System Overview
•Object Segmentation Pipeline
FeatureEngineering
Superpixel Segmentation
Superpixels Features Probabilistic Output Final ResultsInput Image
Feature Extraction SVM Classification GrabCut
Stage 1 : Superpixel Classification
Stage 2 : Pixel-based CRF Smoothing
System Overview
• Superpixel Classification• Superpixel Segmentation • Graph-based image segmentation
• Feature Extraction: • To be detailed in next section
• SVM Classification[1]• RBF kernel with Probabilistic Output
System Overview
•Pixel-Based CRF Smoothing• Fusing several kinds of information as data term• Solving with GrabCut with only a few iterations
SVM Probabilistic Output CRF Smoothing Output
BinarizeFirst Iteration
SecondIteration
Outline
• System Overview
• System Characteristics
•Results and Conclusions
System Characteristics • Superpixel Segmentation -- Efficient Graph-Based Image Segmentation[2]• Fast, property of edge-preserving• Speeding up the whole procedure• Improving the separability between foreground and
background
Superpixels and their edge-preserving property
System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion
Gradient
Texture
Color and skin
Geometrical
Saliency
Results of Object Detection
PCA
Dense SIFT[3][4] dictionary with Bag-of-Words description
Multi-scale LBP histogram
RGB histogram and HS histogram with skin detection
Position, direction and roundness
Color spatial distribution, multi-scale local and global contrast
Probability derived from AdaBoost, with manifold ranking[6] refinement
System Characteristics • Feature Engineering – Superpixel-Based Multiple Features Fusion• Illustration of object detection
Object Detection result Rectangle Density as Probability Refined with Manifold Ranking
System Characteristics •Pixel-Based CRF Smoothing – GrabCut[7]•Modified data term • Solving by maxflow iteratively
SVM Result Object Detection Result
GMM Result for Foreground and Background
CRF Smoothing Output
Outline• System Overview
• System Characteristics
•Results and Conclusions
Conclusion and Results Exhibition•Results Exhibition
Conclusion and Results Exhibition•Conclusion• Superpixel classification• Feature fusion works• CRF smoothing improves the results of SVM
• Object parts sometimes lost• Context information is inadequate
[1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm.
[2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2): 167-181.
[3] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.
[4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of computer vision algorithms[C]//Proceedings of the international conference on Multimedia. ACM, 2010: 1469-1472.
Selected References
Selected References[5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011, 33(2): 353-367.
[6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173
[7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts[C]//ACM Transactions on Graphics (TOG). ACM, 2004, 23(3): 309-314.
Thank you very much!Any questions?
CASIA_IGIT
Leader: Ying Wang (王颖)Members: Kun Ding (丁昆)
Huxiang Gu (谷鹄翔)Yongchao Gong (宫永超)
E-mails: {ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn