Comparison of Matrix Completion Algorithms for Background Initialization in Videos

Andrews Sobral, Thierry Bouwmans and El-hadi ZahZahPh.D. Student, Computer Vision

Lab. L3I/MIA – University of La Rochelle, France

Summary

▪ Context

▪ Understanding an Intelligent Video Surveillance Framework

▪ Introduction to Background Subtraction

▪ Background Model Initialization Problem

▪ Matrix Completion

▪ Proposed Approach

▪ Experimental results

▪ Conclusions

!

Video Content Analysis(VCA) or Video Analytics

Behavior Analysis

Image acquisition and preprocessing

ObjectDetection

ObjectTracking

event location

Intrusion detection Collision

prevention

Target detection and tracking

Anomaly detection

Target behavior analysis

Traffic data collection and

analysis

activity report

Understanding an Intelligent Video Surveillance Framework

supervisor

Behind the Scenes of an Intelligent Video Surveillance Framework

!

Example of automatic incident detection

our focus

Introduction to Background Subtraction

Initialize Background Model

frame modelForegroundDetection

Background Model Maintenance

our focus

Background subtraction methods

Traditional methods:• Basic methods, mean and variance over time• Fuzzy based methods• Statistical methods • Non-parametric methods• Neural and neuro-fuzzy methods

Matrix and Tensor Factorization methods:• Eigenspace-based methods (PCA / SVD)• RPCA, LRR, NMF, MC, ST, etc.• Tensor Decomposition, NTF, etc.

BGSLibrary (C++)https://github.com/andrewssobral/bgslibrary

A large number of algorithms have been proposed for background subtraction over the last few years:

LRSLibrary (MatLab)https://github.com/andrewssobral/lrslibrary

our focus

Andrews Sobral and Antoine Vacavant. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding (CVIU), 2014. http://dx.doi.org/10.1016/j.cviu.2013.12.005

Introduction to Matrix Completion (MC)

▪ MC can be formulated as the problem of o recover a low rank matrix (L) from the partial observations of its entries:

L

Underlying low-rank matrix

A

Matrix of partial observations

http://perception.csl.illinois.edu/matrix-rank/home.html

Applying MC for Background Model Initialization

▪ Proposed approach:

Motion Detection and Frame Selection

92 relevant frames are selected from a total of 296 frames (68,92% of reduction).

Matrix Completion Process

Illustration of the matrix completion process. From the left to the right: a) the selected frames in vectorized form (our observation matrix), b) the moving regions are represented by non-observed entries (black pixels), c) the moving regions filled with zeros (modified version of the observation matrix), and d) the recovered matrix after the matrix completion process.

Experimental results

Scene Background Initialization (SBI) data set

SBI data set provide 7 image sequences and corresponding ground truth backgrounds. Matlab scripts are provided for evaluating results in terms of eight metrics that include those used in the literature for background estimation.

Lucia Maddalena and Alfredo Petrosino, Towards Benchmarking Scene Background Initialization, arXiv:1506.04051http://sbmi2015.na.icar.cnr.it/

Matrix Completion Algorithms

The algorithms were implemented in MATLAB (R2014a) running on a laptop computer with Windows 7 Professional 64 bits, 2.7 GHz Core i7-3740QM

processor and 32Gb of RAM.

Quantitative results on SBI data set

Quantitative results on SBI data set

Visual results on SBI data set

Conclusions

▪ The key idea is to eliminate the redundant frames, and consider its moving regions as non-observed values. This approach results in a matrix completion problem, and the background model can be recovered even with the presence of missing entries.

▪ The experimental results on the SBI data set shows the comparative evaluation of these recent methods, and highlights the good performance of LRGeomCG method over its direct competitors.

▪ Future research may concern to evaluate incremental and real-time approaches of matrix completion algorithm applied in streaming videos.

▪ MATLAB codes available: https://sites.google.com/site/mc4bmi/