Multiple Human Objects Tracking in Crowded Scenes

Yao-Te Tsai, Huang-Chia Shih,and Chung-Lin HuangDept. of EE, NTHU

International Conference on Pattern Recognition (ICPR’06)

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Introduction human objects tracking systems

Pfinder Utilize stochastic, region-based feature

McKenna et al. Adaptive Gaussian mixture to model color distributi

on M2Tracker

Combine presence probability with color model to classify each pixel

Tsutsui et al. Exchange the optical flow information within multip

le cameras

Multiple human objects tracking system

System consist of Model-based object segmentation Remove noise of segmented region Optical flow-based position estimation Occlusion detection Object separation from occlusion

Contribution Track occlusion, separate object and

track it individually afterwards

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Gaussian Mixture Color Model Condition probability for pixel i belong to

object O is

Parameters:mean , and covariance matrix

Expectation-maximization (EM) algorithm To determine the maximum likelihood

parameters of a mixture of m Gaussian

1)(,0)( jj j

m

j

ii

j

jeOipjj

Tj

1

)()(2

1

)(2

1)(

1

Color Model Use HIS space to reduce ambient illumination change

Each pixel i has 2-D feature vectorwhere hi is the hue, si is the saturation

Likelihood pixel i belonging to torso (n=0) or the bottom (n=1) of a person O is

),( iii shv

m

j

vv

j

n jeOipjij

Tji

1

)()(2

1

)(2

1)(

1

Color similarity

The color of the torso of object 1 is similar to the color of the bottom of object 2

(b) is the result of applying the torso color model of object 2 for all pixels

Initialize Presence Map Presence map

The set of presence probabilities of the pixels inside the object

Head line Scan the torso projection profile H0(yi) yHL=arg minyiH0(yi)

Torso line Central vertical axis

Probabilities of the pixels will be larger

Bayesian Classification Only consider pixels in the neighborhood of an object

Pposteriori(Ok|i) : posterior prob. of pixel i belong to object Ok P(i|Ok) : probability defined by torso or bottom model Ppriori(Ok) : presence probability of Ok

Relative coordinate Defined by the head line and central axis

Color model selection for torso or bottom If Pposteriori(Ok|i) >=0.05, then i classified to Ok

)()()( ),(,nkkyxprior

nkposterior OiPOPiOP

rr

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Single Object Tracking Flow chart of single object tracking

Newcomer detection By using background subtraction

Tracking process Calculate angles and magnitudes of the flow vectors in the

neighborhood of window Quantize the direction into 12 bins (30 degree/bin) and det

ermine which bin object belong to Find the most significant bin and calculate average flow Shift object window by average flow

Update presence map Size and shape of a moving object change over ti

me Need to update the presence map

If pixel at (xr, yr) classified correctly, increase the corresponding priori prob. for every 10 frames

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Tracking Occluded Objects

Optical flow and presence probability are unreliable

Only use color models to estimate object’s central vertical axis

Use distance between central axes to determine object becomes separable

Occlusion detection Each individual object has five attributes

based on its activity

Two object windows touch and form an occlusion window

Two object windows overlap and form an occlusion window

A single object joins an occlusion and form a new occlusion window

Occlusion group separation Compute distance between every two objects i

n an occlusion group as

If Two extreme objects Oi and Oi+1

If |di| > threshold, then determine Oi or Oi+1 can be separate from the original occlusion

kc

kck xxd 1

ki dd max

Object separation example

separate separate

Resume tracking One an object separate from occlusion, we

need to update: Object window location, head line, and torso line Central vertical axis

Left and right boundary Scan the vertical projection profile of Ok From the central vertical axis leftward and then

rightward Head line and torso line

Analyze the horizontal projection profile

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Tracking example 1 Format:

Image frame is 160x120x24 bits, 15 frames/sec

Occlusion 1

Occlusion 2

Object 2 separate and join occlusion 1 single object 4

occlusion 1

Tracking example 2 Two occlusion groups merge as one and then separate

to another two occlusion groups

System evaluation and error analysis Three separation events:

2-object, 3-object, and 4-object separation event Define separation occurs’ accuracy based on:

A single object leaves an occlusion and track him correctly afterward

If an occlusion splits into two, system identify the correct objects in the two pairs.

More 2-object separation events

Outline

Introduction Initialization and Pixel Classification Single Object Tracking Tracking Occluded Objects Experimental Results Conclusion

Conclusion Object tracking consists of:

Gaussian mixture model Presence probability Optical flow

Objects under occlusion Use color model to distinguish each object and

locate central vertical axes Object separation

Determine by distances between the central vertical axes of objects