robust multi-pedestrian tracking in thermal-visible surveillance videos alex leykin, yang ran, and...
TRANSCRIPT
Robust Multi-Pedestrian Tracking in Thermal-Visible
Surveillance Videos
Alex Leykin, Yang Ran, and Riad Hammoud
GoalCreate a pedestrian tracker that operates in:
1. Varying illumination conditions2. Crowded environment
To achieve it we create a fusion pedestrian tracker that uses input from:
3. IR camera4. RGB camera
Our approach consists of three stages:
BG Subtraction Bayesian tracker Pedestrian Classifier
Background Model
Two stacks of codeword values (codebooks)
Color• μRGB
• Ilow • Ihi
Thermal• thigh
• tlow
codeword
codebook
Adaptive Background Update
If there is no match create new codeword
Else update the codeword with new pixel information
If >1 matches then merge matching codewords
I(p) > Ilow
I(p) < Ihigh
(RGB(p)∙ μRGB) < TRGB
t(p)/thigh > Tt1
t(p)/tlow > Tt2
Match pixel p to the codebook b
Tracking
Location of each pedestrian is estimated probabilistically based on:
Current image Model of pedestrians Model of obstacles
The goal of our tracking system is to find the candidate state x` (a set of bodies along with their parameters) which, given the last known state x, will best fit the current observation z
P(x’| z, x) = P(z|x’) · P(x’|x)
observation likelihood state prior probability
Tracking – Accepting the State
))|'(
)'|(
)(
)(,1min(),(
xxtm
xxtm
xP
xPxx
x’ and x candidate and current states
P(x) stationary distribution of Markov chain
mt proposal distribution
Candidate proposal state x’ is drawn with probability mt(x’|x) and then accept it with the probability α(x, x’)
Tracking: Priors
N(hμ, hσ2) and N(wμ,wσ
2) body width and height
U(x)R and U(y)R body coordinates are weighted uniformly within the rectangular region R of the floor map.
d(wt, wt−1) and d(ht, ht−1) variation from the previous size
d(xt, x’t−1) and d(y, y’t−1) variation from Kalman predicted position
N(μdoor, σdoor) distance to the closest door (for new bodies)
Constraints on the body parameters:
Temporal continuity:
Tracking Likelihoods: Distance weight plane
2hPz
Problem: blob trackers ignore blob position in 3D (see Zhao and Nevatia CVPR 2004)
Solution: employ “distance weight plane” Dxy = |Pxyz, Cxyz| where P and C are world coordinates of the camera and reference point correspondingly and
Tracking: Likelihoods
),(11 1 ttcolorcolor ccBwP
I
DZOIP xyZ
)( )0(
O
DIZOP xyZ
)( )0(
Implementation of z-buffer (Z) and distance weight plane (D) allows to compute multiple-body configuration with one computationally efficient step.
Let I - set of all blob pixels O - set of body pixels
Then
Color observation likelihood is based on the Bhattacharya distance between candidate and observed color histograms
Tracking: Jump-Diffuse Transitions
Add a new body Delete a body Recover a recently deleted body Change body dimensions
Change body position (optimize with mean shift)
Tracking Results
Sequence
number
Frames
People
People
missed
False hits
Identity switches
1 1054
15 3 1 3
2 0601
8 0 0 0
3 1700
16 5 1 2
4 1506
3 0 0 0
5 2031
2 0 0 0
6 1652
4 0 0 0
%% 8544
48 12.5
4.1 10.4
Finding Gait in Spatio-temporal Space
Periodic Pattern Grouping Theory: A two-dimensional pattern that
repeats along one dimension is called a frieze pattern in the mathematics and geometry literature
Group theory provides a powerful tool for analyzing such patterns
Mapping gait into repetitive texture Translational symmetry: Class P4 Detection: verifying spatio-temporal
texture Localization: extract orientation
(trajectory), frequency (period), representative motif (signature)
Symmetries of the gait patterns
Classifying PedestriansX-t Image Extract Lattice Signature Results
Finding Gait in Spatio-temporal Space
Details in Y. Ran, I. Weiss, Q. Zheng, and L. S. Davis. Pedestrian detectionvia periodic motion analysis. IJCV 2007
Contributions
Robust to illumination changes
Resolving track initialization ambiguity with MCMC
Non-unique body-blob correspondence
Gait detector runs in real time
Future Work Extend binary background mask with
foreground probability values
Incorporate these probabilities into appearance-based fitness equation for particle filter-based tracker
Utilize tracklet stitching (via particle tracker) to decrease the number of broken paths
Aknowledgements
Organizers of OTCBVS Benchmark Dataset Collection
http://www.cse.ohio-state.edu/otcbvs-bench