![Page 1: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/1.jpg)
C t t i h d Cl ifiContent-enriched Classifier for Web Video Classificationo eb deo C ass cat o
ByBin Cui & Ce Zhang
Dept of CS Peking UniversityDept. of CS Peking University
Gao CongSchool of CE, Nanyang Technological
Presented by
University, Singapore
SIGIR 2010Presented by
Ahmed Ibrahim
![Page 2: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/2.jpg)
OutlineOutline• Introduction• Current Approaches• Current Approaches• Proposed Approach
– Content -enriched Classifier– Content-enriched Similarity – CSE Classifier Algorithm
• Experimental ResultsExperimental Results• Conclusions & Critique• Proposed approach extension
![Page 3: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/3.jpg)
IntroductionIntroduction
• In video sharing services the userIn video sharing services, the user browses the web by categories.
• Real time categorization plays a key roll for organizing, browsing, and retrieving online video.
3CS848 Winter 2011
![Page 4: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/4.jpg)
Web Video ProcessingWeb Video Processing
Video Title
User Description
4CS848 Winter 2011
![Page 5: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/5.jpg)
Web Video Classification Problems
o Although text features and content features areo Although text features and content features are complementary but utilizing content features in video classification stage is computationally expensive.
o Text classification cannot use the rich information contained in video content.
o Text description characteristics limits the classification performance of semantic similarity based on WordNet(and / or) term co occurrence(and / or) term co-occurrence.
5CS848 Winter 2011
![Page 6: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/6.jpg)
Current ApproachesCurrent Approaches
6CS848 Winter 2011
![Page 7: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/7.jpg)
Proposed ApproachProposed Approach1- Content-Enriched Similarity:
Using visual clues of web videos to obtain more reasonablesemantic relations among words which called Content-EnrichedSimilarity (CES) between words.y ( )
2- Content-Enriched Nonlinear Classifier
At the training stage a nonlinear SVM classifier is built to– At the training stage, a nonlinear SVM classifier is built to explore the semantic similarity between words using CES.
– At the classification stage, this classifier classifies a new video g ,using its text features (but not its content features).
CS848 Winter 20117
![Page 8: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/8.jpg)
Proposed Approach (cont.)Proposed Approach (cont.)3- Semantic kernels will be computed using the following
f lformula:
4- Multi-Kernel Enhancement: Given several kernels created using different word pair-wise similarity matrices for multiple kernel optimization.
8CS848 Winter 2011
![Page 9: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/9.jpg)
Content-enriched ClassifierContent enriched ClassifierClassifier
Training Data(Test
Features)Content-Enriched Semantic KernelBuilding
Classifier
Features)
Content-enriched word
similarity Finding the hyperplane in
Content Enriched
ClassifierApplying Classifier
Extract CES
Testing Data
Content-Enriched Kernel Space
Training Data g(Test Features)
CES: Content-Enriched Similarity
(Test + Content Features)
9CS848 Winter 2011
![Page 10: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/10.jpg)
Content-Enriched SimilarityContent Enriched SimilarityGenerally, two words are similar if they appear in they, y ppsame cluster, within which the videos are similar in termsof content.
Extract Visual Content Features
VideoDatabase
K-means Cl t i
“K” clusters = 100Project ‘tf’ into cluster
spaceDatabase(5149 videos)
Clustering
‘VS’video-cluster
space
relation matrix
10CS848 Winter 2011
![Page 11: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/11.jpg)
CES Classifier AlgorithmCES Classifier Algorithm
11CS848 Winter 2011
![Page 12: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/12.jpg)
Experimental ResultsExperimental Results• Experimental Settings p g
– Datasets:• Two real-life datasets are collected from ‘YouTube’ between Sept 23
& 24 of 2009, YT923 (5149 videos) & YT924(4447 videos)., ( ) ( )• They categorized both datasets into 15 Categorize.
– Preprocessing : Feature Extraction:• Text features are extracted from videos include (video titles andText features are extracted from videos include (video titles and
descriptions).• Words are stemmed using WordNet stemmer.• Stop words are manually removed.p y• The following visual content features (color, texture & edges) are
extracted .
12CS848 Winter 2011
![Page 13: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/13.jpg)
Word Similarity ApproachWord Similarity Approach
• The relation discovered by CSE are meaningful and agree with common sense.
• The classification results reflect the superiority of proposed methods.
13CS848 Winter 2011
![Page 14: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/14.jpg)
Classification effectivenessClassification effectiveness
• Classification Performance on different frameworks.• F-score: accuracy measure for classification which can
be calculated usingbe calculated using . • Macro-F: average of F-score for each category.• Micro-F: average of F-score for all decisionsMicro F: average of F score for all decisions.
14CS848 Winter 2011
![Page 15: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/15.jpg)
Effectiveness Per CategoryEffectiveness Per Category
The scores of content classifier have been excluded because their performance is much worse than the text.
15CS848 Winter 2011
![Page 16: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/16.jpg)
Performance on Multi-kernelPerformance on Multi kernel
• This table shows the results on classification effectiveness with multi-kernel solutioneffectiveness with multi kernel solution
16CS848 Winter 2011
![Page 17: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/17.jpg)
ConclusionsConclusions• Novel Framework that exploits visual contentNovel Framework that exploits visual content
and text features to facilitate online videocategorization is presented.
• Content-enriched Semantic Kernel whichextracts word relationship by clustering the videowith visual content feature is proposed.
17CS848 Winter 2011
![Page 18: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/18.jpg)
PRESENTED APPROACH
EXTENDEXTEND
![Page 19: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/19.jpg)
Camera Motion ModelCamera Motion Model
To enhance the presented approach, we will study the feasibility ofi C M ti M d l id t t f t thusing Camera Motion Model as a video content feature on the
classification performance and efficiency using CC_WEB_VIDEO.
20CS848 Winter 2011
![Page 20: Content-enriched Classifier for Web Video Classification](https://reader034.vdocuments.mx/reader034/viewer/2022052522/554a1ee8b4c9058c5d8b56fc/html5/thumbnails/20.jpg)
Questions
Thank You21
CS848 Winter 2011