content-enriched classifier for web video classification
Post on 06-May-2015
134 Views
Preview:
DESCRIPTION
TRANSCRIPT
C t t i h d Cl ifiContent-enriched Classifier for Web Video Classificationo eb deo C ass cat o
ByBin Cui & Ce Zhang
Dept of CS Peking UniversityDept. of CS Peking University
Gao CongSchool of CE, Nanyang Technological
Presented by
University, Singapore
SIGIR 2010Presented by
Ahmed Ibrahim
OutlineOutline• Introduction• Current Approaches• Current Approaches• Proposed Approach
– Content -enriched Classifier– Content-enriched Similarity – CSE Classifier Algorithm
• Experimental ResultsExperimental Results• Conclusions & Critique• Proposed approach extension
IntroductionIntroduction
• In video sharing services the userIn video sharing services, the user browses the web by categories.
• Real time categorization plays a key roll for organizing, browsing, and retrieving online video.
3CS848 Winter 2011
Web Video ProcessingWeb Video Processing
Video Title
User Description
4CS848 Winter 2011
Web Video Classification Problems
o Although text features and content features areo Although text features and content features are complementary but utilizing content features in video classification stage is computationally expensive.
o Text classification cannot use the rich information contained in video content.
o Text description characteristics limits the classification performance of semantic similarity based on WordNet(and / or) term co occurrence(and / or) term co-occurrence.
5CS848 Winter 2011
Current ApproachesCurrent Approaches
6CS848 Winter 2011
Proposed ApproachProposed Approach1- Content-Enriched Similarity:
Using visual clues of web videos to obtain more reasonablesemantic relations among words which called Content-EnrichedSimilarity (CES) between words.y ( )
2- Content-Enriched Nonlinear Classifier
At the training stage a nonlinear SVM classifier is built to– At the training stage, a nonlinear SVM classifier is built to explore the semantic similarity between words using CES.
– At the classification stage, this classifier classifies a new video g ,using its text features (but not its content features).
CS848 Winter 20117
Proposed Approach (cont.)Proposed Approach (cont.)3- Semantic kernels will be computed using the following
f lformula:
4- Multi-Kernel Enhancement: Given several kernels created using different word pair-wise similarity matrices for multiple kernel optimization.
8CS848 Winter 2011
Content-enriched ClassifierContent enriched ClassifierClassifier
Training Data(Test
Features)Content-Enriched Semantic KernelBuilding
Classifier
Features)
Content-enriched word
similarity Finding the hyperplane in
Content Enriched
ClassifierApplying Classifier
Extract CES
Testing Data
Content-Enriched Kernel Space
Training Data g(Test Features)
CES: Content-Enriched Similarity
(Test + Content Features)
9CS848 Winter 2011
Content-Enriched SimilarityContent Enriched SimilarityGenerally, two words are similar if they appear in they, y ppsame cluster, within which the videos are similar in termsof content.
Extract Visual Content Features
VideoDatabase
K-means Cl t i
“K” clusters = 100Project ‘tf’ into cluster
spaceDatabase(5149 videos)
Clustering
‘VS’video-cluster
space
relation matrix
10CS848 Winter 2011
CES Classifier AlgorithmCES Classifier Algorithm
11CS848 Winter 2011
Experimental ResultsExperimental Results• Experimental Settings p g
– Datasets:• Two real-life datasets are collected from ‘YouTube’ between Sept 23
& 24 of 2009, YT923 (5149 videos) & YT924(4447 videos)., ( ) ( )• They categorized both datasets into 15 Categorize.
– Preprocessing : Feature Extraction:• Text features are extracted from videos include (video titles andText features are extracted from videos include (video titles and
descriptions).• Words are stemmed using WordNet stemmer.• Stop words are manually removed.p y• The following visual content features (color, texture & edges) are
extracted .
12CS848 Winter 2011
Word Similarity ApproachWord Similarity Approach
• The relation discovered by CSE are meaningful and agree with common sense.
• The classification results reflect the superiority of proposed methods.
13CS848 Winter 2011
Classification effectivenessClassification effectiveness
• Classification Performance on different frameworks.• F-score: accuracy measure for classification which can
be calculated usingbe calculated using . • Macro-F: average of F-score for each category.• Micro-F: average of F-score for all decisionsMicro F: average of F score for all decisions.
14CS848 Winter 2011
Effectiveness Per CategoryEffectiveness Per Category
The scores of content classifier have been excluded because their performance is much worse than the text.
15CS848 Winter 2011
Performance on Multi-kernelPerformance on Multi kernel
• This table shows the results on classification effectiveness with multi-kernel solutioneffectiveness with multi kernel solution
16CS848 Winter 2011
ConclusionsConclusions• Novel Framework that exploits visual contentNovel Framework that exploits visual content
and text features to facilitate online videocategorization is presented.
• Content-enriched Semantic Kernel whichextracts word relationship by clustering the videowith visual content feature is proposed.
17CS848 Winter 2011
PRESENTED APPROACH
EXTENDEXTEND
Camera Motion ModelCamera Motion Model
To enhance the presented approach, we will study the feasibility ofi C M ti M d l id t t f t thusing Camera Motion Model as a video content feature on the
classification performance and efficiency using CC_WEB_VIDEO.
20CS848 Winter 2011
Questions
Thank You21
CS848 Winter 2011
top related