online detection of unusual events in videos via dynamic sparse coding
DESCRIPTION
Online Detection of Unusual Events in Videos via Dynamic Sparse Coding. Outline. Unusual Event Detection Video Representation Dynamic Sparse Coding Empirical Study Conclusions. Outline. Unusual Event Detection Video Representation Dynamic Sparse Coding Empirical Study Conclusions. - PowerPoint PPT PresentationTRANSCRIPT
Online Detection of Unusual Events in Videos via Dynamic Sparse Coding
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Unusual events: Incidences that occur very rarely in the entire video
Unusual Event Detection
• Easy-to-verify– Given a frame, fairly easy to decide if unusual
events occur• Hard-to-describe– Cannot enumerate all possible unusual events – Cannot model unusual events directly
• Solution: Model usual events instead, and claim anything different as unusual
Easy to model usual events?
Challenges
• Unsupervised learning– Only input is video itself
• Online detecting– In most cases, cannot afford multiple runs through
the video• Concept drift– Usual events change
• Truly unusual event vs. Noisy usual event
Previous Works
• Clustering Based Method (CVPR 2004)
– Finding spatially isolated clusters• Reconstruction (IJCV 2007)
• Space-time Markov Random Field (CVPR 2009)
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Video Features
• Static features based on edges and object shapes– Image-level information
• Dynamic features based on optical flow measurements– Motion information
• Spatio-Temporal Interest Points– Obtained from local video patches– Shown to be useful in human action categorization
Spatio-Temporal Interest Point
• Detection– Basic idea: generalize interest point detector from
spatial domain to spatio-temporal domain– Spatial (image): Laplacian, Hessian, Harris corner
detector, etc.– Spatio-temporal (video): spatio-temporal corners,
Laplacian on spatial and temporal axis– Output: small video patches extracted from each
interest point
Spatio-Temporal Interest Point (Cont.)
• Description– Similar to detection, generalization of spatial
method to spatio-temporal domain– Spatial: histogram of directional gradients – SIFT,
HOG– Spatio-Temporal: gradients on x, y, and time
directions
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Motivation of the Approach
• Sparse Reconstruction– Reconstruct an event with other events in the
video• Usual events: multiple appearances could find a few
events to reconstruct it SPARSE• Unusual events: rare appearance need large amount
of events for reconstruction DENSE
• Concisely represent the knowledge of usual events
The Proposed Approach• Define events in the video
– Sliding window runs through the video– Spatio-temporal interest points within the same window define
an event
• Knowledge of what are usual events– Store in the learned dictionary D
• Abnormality of an event
• Update dictionary D
(Ab)Normality
• Reconstruction error
• Sparsity regularization
• Smoothness regularization
(Ab)Normality (Cont.)
• Empirical demonstration
Work-flow
Optimization
• Learning with Fixed 𝜶 D
• Learning D with Fixed 𝜶
• Online Dictionary Update
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Video Data• Subway Surveillance Videos
– Subway exit: 43 minutes, 65K frames• Usual events: people exiting subway• Unusual events: entering subway, loitering, etc.
– Subway entrance: 1 hour 36 minutes, 144K frames• Usual events: people entering subway• Unusual events: exiting subway, no payment, loitering
• Youtube Videos: 8 short videos– Different camera motion (rotation, zoom in/out, fast tracking, slow
motion, etc.)– Different categories of targets (human, vehicles, animals, etc.)– Wide variety of activities and environmental conditions (indoor, outdoor).
Learned Dictionary
• Subway exit surveillance video
• Subway entrance surveillance video
Quantitative Comparison
• Subway exit surveillance video
• Subway entrance surveillance video
Analysis Experiment
• Online Update of the Learned Dictionary– Our approach: update learned dictionary after
observing new event– Comparing method: fixed dictionary
Detected Unusual Events
• Subway Exit
Detected Unusual Events (Cont.)
• Subway entrance
Detected Unusual Events (Cont.)
• Youtube Videos– For each video, approximately the first 1/5 of
video data is used to learn initial dictionary– Unusual event detection is carried out in the
remaining video– Red boxes represent sliding windows that result in
unusual event detection
Outline
• Unusual Event Detection• Video Representation• Dynamic Sparse Coding• Empirical Study• Conclusions
Conclusions
• Fully unsupervised dynamic sparse coding approach for detecting unusual events in videos
• Bases dictionary is updated in an online fashion as the algorithm observes more data, avoiding any issues with concept drift.