a novel scheme for video similarity detection chu-hong hoi, steven march 5, 2003
Post on 18-Dec-2015
215 views
TRANSCRIPT
A Novel Scheme for Video Similarity Detection
Chu-Hong Hoi, Steven
March 5, 2003
OutlineIntroductionOverviewPhase I: Coarse Similarity Measure
Pyramid Partitioning & Density Histogram Naïve Pyramid Density Histogram (NPDH) Fuzzy Pyramid Density Histogram (FPDH)
Phase II: Fine Similarity Measure Near Feature Trajectory (NFT) Simplification Algorithm Similarity Measure Based on NFT
Experiments and ResultsConclusion & Future Work
Introduction Motivation
Huge volume of video data are distributed over the Web.
How to fast detect the similar video effectively? Applications
Copyright issues / watermarking Content-based video retrieval
Overview Challenging Issues
Efficiency Effectiveness
We propose a Two-Phase Similarity Detection Framework based on two kinds of signatures with different granularity.
Solutions by two kinds of signatures Coarse Signature
Pyramid Density Histogram Fine Signature
Nearest Feature Trajectory
Overview
A two-phase framework for video similarity detection
Phase I: Coarse Similarity Measure Video data indexing
A frame is considered as a feature point. A video sequence is formed by a series of feature points. It is hard to index and search the video dat in original data
space. Two partitions of data space
Regular partitioning (Fig.2 (a)) Pyramid partitioning (Fig.2 (b)) (S. Berchtold-SIGMOD 98)
Center Point at (0.5,0.5,…,0.5)
Phase I: Coarse Similarity Measure Pyramid Density Histogram (PDH)
Map the feature points to the pyramid data space, and statistically calculate the distribution of the feature points
Obtain a density histogram of feature points as the coarse signature
Two kinds of PDHs Naïve Pyramid Density Histogram Fuzzy Pyramid Density Histogram
Phase I: Coarse Similarity Measure Naïve Pyramid Density Histogram
2d-dimension NPDH vector u=(u1,u2,…,u2d) How to calculate the density histogram?
For a d-dimension feature point v=(v1,v2,…,vd)
Center Point at (0.5,0.5,…,0.5)
Phase I: Coarse Similarity Measure Fuzzy Pyramid Density Histogram
In NPDH, a given feature point is allocated to only 1 pyramid. It would loss the information of other dimensions.
In FPDH, we fuzzyly allocate a feature point v to d pyramids based on the value of its d dimensions.
Center Point at (0.5,0.5,…,0.5)j=1,2,…,d
Phase I: Coarse Similarity Measure Similarity Filtering Based on PDH
Given a query example q and a compared sample s from the video database.
Set a filtering threshold δ , then video s is filtered out if it satisfies the following condition:
Phase II: Fine Similarity Measure Conventional Similarity Measure
Nearest Neighbor (NN) or (k-NN) Nearest Center (NC) Disadvantage: ignore the temporal
information of video sequences Nearest Feature Trajectory (NFT)
A video sequence is considered as a series of feature trajectories rather than isolated key-frames.
Phase II: Fine Similarity Measure Nearest Feature Trajectory
A frame in a video sequence is considered as a feature point.
Two feature points form a feature line. A series of feature lines form a feature trajectory in a
video shot. A video sequence consists of a series of feature
trajectories. Each trajectory corresponds to a individual shot or a
gradual transition of shots. Similarity measure is based on the nearest average
distance of feature trajectories in two video sequences.
Phase II: Fine Similarity Measure Generation of Simplified Feature Trajectory
Formulate the procedure by Minimum Square Error approach
The minimum procedure of MSE is time-consuming!
Phase II: Fine Similarity Measure We propose an algorithm for efficient generate
the feature trajectories. Define a local similarity measure function to
approximate the deviation degree.
The larger the value of LR(vk) is, the larger the deviation degree at vk is.
Based on the LR(vk), we remove the point with the minimum value each time until there remains only feature points.
Phase II: Fine Similarity Measure Similarity Measure Based NFT
Phase II: Fine Similarity Measure
Distance Measure of Two Feature Trajectories
Similarity Measure of Two Video Sequences
Considering the boundary problem, if 0≦λ≦1, falls in the line segment; otherwise, it falls out of the line
Experiments and Results Ground Truth Data
About 300 video clips with different coding formats, resolutions and slight color modifications
Feature Extraction RGB Color Histogram 64 dimensions
Performance Evaluation Metric Average Precision Rate
Average Recall Rate
Experiments and Results Coarse Similarity Measure
FPDH vs. NPDH
Experiments and Results Fine Similarity Measure
NFT vs. NN
Conclusions We propose an effective two-phase framework
to achieve the video similarity detection. Different from the conventional way, our
similarity measurement scheme is based on different granular similarity measure.
In the coarse measurement phase, we suggest Fuzzy Pyramid Density Histogram.
In the fine measurement phase, we present the Nearest Feature Trajectory technique.
Experimental results show that our scheme is better than the conventional approach.
Future Work Engaging more effective features in our
scheme to improve the performance
Enlarging our database and testing more versatile data
Cost performance evaluation
Q & A
Thank you!