Target Tracking Based on Mean Shift and Improved Kalman Filtering Algorithm

Hongxia Chu1,2, Kejun Wang1

College of automation Department of Electronic Engineering Harbin Engineering University1 Heilongjiang Institute of Technology2

Harbin 150001, Heilongjiang Province, China Harbin 150001, Heilongjiang Province, China chx0420@163.com chx0420@163.com

Abstract - A novel real-time image target tracking algorithm which is based on Mean Shift and improved Kalman filtering algorithm is studied. In the cases of known initial information(position and velocity), measuring point is integrated in tracking window by applying the method of maximum fuzzy entropy Gaussian clustering. The point which has been integrated is inputted to the Kalman filter, and Kalman filter is used to predict the next state’s position of the target point. At last, the fast tracking of target is realized by using the combination of Mean Shift algorithm and improved Kalman filter. Result of theory and experiment indicates that the algorithm could keep tracking’s real-time performance in condition of image sequences. Accuracy of the target tracking is guaranteed as the target’s alternating problem and occlusion problem is improved.

Index Terms - Target tracking. Maximum fuzzy entropy Gaussian clustering. Mean Shift. Kalman filter.

I. INTRODUCTION

Target tracking technology is an important subject of pattern recognition and image processing and computer vision and weapon guidance fields etc. It combines image processing and automatic control and information science and probability and statistics reasoning. The target tracking has developed being as an advanced technology, which can automatically distinguish objects from the video image, acquire information of the target position velocity etc, and track the target movement. Target tracking technology has been widely used in military and civil field[1][2][3][4]. As an efficient pattern matching algorithm, Mean Shift algorithm has been used successfully in target tracking field[5][6][7]. It uses gradient optimum algorithm to quickly realize target location and it can track non-rigid target on real-time, it also has fairly good applicability for deformation and rotation of the motion target. But in complicated background, because Mean Shift algorithm only uses color information and don’t use the target’s motion direction and velocity information in spatial and did not receive any prediction for moving target, its anti-interference ability is bad. Especially when the target in big-size is kept out, it leads to the poor tracking ability. So problems of mutual occlusion in context of targets and target suspension etc need to be processed. Therefore, estimator is introduced to predict the motion parameters. The paper adopts Mean Shift tracking algorithm based on color histogram. At the same time, it combines the prediction of Kalman filtering for target space motion position reasonably so that consistency and coherence of target motion is guaranteed. Kalman filter is a

linear sequence of the dynamic systems and the state minimum variance estimation algorithm. It is a small amount with the characteristics of real-time computation. It can predict exactly the position and velocity of the target [8].

However, in the process of target tracking, because the target is too small or has low signal noise ratio, it makes more false alarm target existing in the single-frame tracking window. Therefore, the paper is applying the method of maximum fuzzy entropy Gaussian clustering to check the detecting points. The point which has been integrated is inputted to the Kalman filter to be predicting. At last, fast tracking of target is realized by using the combination of mean shift algorithm and Kalman filter. Experiment results demonstrate that tracking effect is good.

II. IMPROVED KALMAN FILTER ALGORITHM

A. Maximum Fuzzy Entropy Gaussian Clustering After target is searched and detected in full-image spatial

scope, target’s approximate position has been determined. Therefore, in process of tracking, detecting target can only in tracking window, which makes it possible for single-frame detection. However, because the target is too small or low signal noise ratio, its makes more false alarm target still exist in the single-frame tracking window. Tracker exits uncertainty for these detection results. If uncertainty can be solved, tracker can finish forecast tasks for target’s next state exactly through adopting Kalman filter. In the methods of eliminating the uncertainty, the paper references the principle of maximum fuzzy entropy clustering

In order to use the theory of maximum fuzzy entropy Gaussian clustering in the target tracking, here, supposing in a n-dimensional space, mk observation data (z1,z2,…

kmz ) and a target T is received. Thereupon, clustering process can be described as the following optimization process. Corresponding cost function is:

∑1=

),(=km

iii cxduE (1)

Thereinto, ),( cxd i represents euclidean distance

between observation zi and clustering c, and also iu obeys the following restriction:

808

Proceedings of the IEEE

International Conference on Automation and Logistics Shenyang, China August 2009

978-1-4244-4795-4/09/$25.00 © 2009 IEEE

]1,0[∈∀1=∑1=

i

c

ii uu (2)

According to the information theory, in order to describe data points and membership degree of class-center for minimum unbiased, maximum entropy principle is adopted to make the entropy maximum. By shannon entropy principle, expression is:

i

m

iii uuuHH

k

∑1=

ln-=)(= (3)

Under the restriction of Equation (1) and (2), by using Lagrange multiplier method, optimization target function can be defineded as:

∑ ∑∑1= 1=1=

)1-(+),(--ln-=),(k kk m

i

m

iiiii

m

ii uλcxduauuCUJ (4)

λα, is lagrange multiplier. From maximum formula (4) can get probability is:

km

i

cxad

cxad

i mie

eu

k

i

i

,...2,1==∑

1=

),(-

),(-

(5)

In Equation (5), membership degree ui, i=1,2,…, mk is a minimum bias estimate.

B. Improved Kalman Filtering Algorithm The step of improved Kalman filtering algorithm is as

following: Step 1: Supposing system state of present is k+1,

according to the system model, state of present can be predicted based on the former state.

)|(=)|1+(^^

kkXAkkX (6)

In formula (6), )|1(^

kkX + is the prediction results of

the former state, )|(^

kkX is the optimal result of the former state.

Step 2: Calculating “innovation”.

],1[∈)|1+(-)1+(=)1+(^

kjj mjkkXHkzkv (7)

Step 3: Renewing covariance of X(k|k-1).

QAkkAPkkP +)|(=)|1+( ' … …(8)

Step 4: Covariance of renewing “innovation”.

RHkkHPkS T +)|1+(=)1+( (9)

Step 5: Using formula (5) to calculating membership degree uj.

Step 6: Fusion observation information.

∑1=

)1+()1+(=1)+(km

iii kzkukZ (10)

Step 7: Renewing “innovation” and renewing target state.

)|1+(-1)+(=1)+(^

kkXHkZkV (11)

)1+()1+(+)|1+(=)1+|1+(^^

kVkKgkkXkkX (12)

In formula (12), )1+()|1+(=)1+( 1- kSHkkPkKg T is Kalman gain.

Step 8: Renewing prediction covariance, that is:

TT

Tj

m

iji

kKgkVkV

kvkvkukkg

kkPHkKgIkkPk

)1+(])1+()1+(-

)1+()1+()1+()[1+(

+)|1+(])1+(-[=)1+|1+(

∑1=

(13)

Step 9: Tracker receives the next frame and returns to the first step.

Repeating the process until all the images is tracked.

III. MEAN SHIFT TARGET TRACKING ALGORITHM

Histogram of motion target can’t be affected by the shape change of target. Therefore, taking histogram as target’s pattern and basing for color distribution to matching has good stability. But it is simple in calculation. Therefore, when pattern is matching, it needs less calculation. Through incessant vector iterating, the algorithm can obtain optimal solution in local. It also has the characteristics of high-speedy and effective, so it can satisfy the requirement of real time. A. Model Foundation 1) Initializing Target Model

The center of target regions is x0, supposing there is n positive pixel expressed as {xi}i=1,2,…n. The number of eigenvalue bin is m. So estimated probability density of the eigenvalue of target model (u=1,2,…m) is:

∑1=

20

^]-)([)

-(=

n

ii

iu uxbδ

hxx

kCq

In which, k(x) is profile function of kernel function. Because of the affect of occlusion or background, the pixels near the center of target model is more reliable than that near outside. The role of k(x) is weighting for each point and giving a big weight for center pixels. The more distant is far away from the point to the center, weight is the little.

In the function k(x), the effect of h

xx i-0 is to eliminate

the effect of targets which are different sizes when calculating.

809

It makes the target which is expressed by elliptic normalize as a unit circle. The main function of ]-)([ uxbδ i is judging if the color value of pixel xi in target area belongs to bin of u-th. Value is 1 when is equal, or is zero. C is a standardized

constant coefficient which it makes∑1=

1=m

uuq , so:

∑1=

20 )

-(

1= n

i

i

hxx

kC

2) Candidate Target Model Candidated target takes y as the center in current frame. It

selects the same kernel function k(x) and window radius h. Then the kernel-based density estimation of the eigenvalue of candidate target u=1,2,…m can be described as:

∑1=

2^]-)([)

-(=)(

hn

ii

ihu uxbδ

hxy

kCyp

In which

∑1=

2

)-

(

1=

hn

i

ih

hxy

kC is standardized constant

coefficient.

B. Similarity Function Similarity function describes similarity measure of initial

frame target model and candidate target model. Defined as:

∑1=

^^

u

^^^),(=)),((=)(ρ

m

uuqypqypy ρ

Its value is between 0 and 1. The more large of )(^

yρ , the more similar of two models. Candidate model which is got by calculating different candidate regions in current frame, which

making )(^

yρ the largest candidate region is target position of this frame.

C. Target Positioning The center of current frame target is located as position

y0 which is the center of previous frame target. The target of optimal matching is searched from the point, in which the central is y. Maximum is got through similarity function is carried out taylor expansion. Then, the vector which moves from the center of candidate region to reality region y is found.

0

1=

2

0

^

1=

2

0

^

01, -

)-

(

)-

(

-=)(

∑

∑

y

hxy

gw

hxy

gwx

yyymh

h

n

i

ii

n

i

iii

Gh

D. Whole Algorithm of Mean Shift Step 1: Setting the whole image as searching area,

adopting target detection to decide automatically tracking target, initializing search window’s size and location.

Step 2: Calculating the probability distribution u

q^

of

target feature in the initializing search window. Step 3: Estimating the feature

muu yp ,...2,1=0

^)}({ which is the

candidate target of y0 in the current frame, and calculating the

coefficient of Bhattacharyya )( 0

^yp .

Step 4: Calculating {wi}i=1,2,…,m. Step 5: Using Mean Shift vector to calculate target’s new

position y1.

Step 6: Renewing muu yp ,...2,1=1

^)}({ , and calculating

)( 1

^yp .

Step 7: If )( 0

^yρ > )( 1

^yρ , then )+(

21

← 101 yyy , or

jumping to step 6.

Step 8: If ε<- 01 yy , then stop. Or else 10 ← yy , jumping to step 4. Continuing to search the best position

until )( 0

^yρ < )( 1

^yρ .

Iterating a few times like this multiple, target area shifts gradually from initial position to the real target position.

IV. COMBINED IMPROVED KALMAN FILTERING WITH MEAN SHIFT ALGORITHM

A. Combining Kalman Filtering with Mean Shift Algorithm

1)Kalman pre-estimating gets )1(^

+kX Setting the whole image as searching area, tracking target

is ensured automatically by adopting GCM detection. Search window’s size and location is initializied. The Measured value is transmited to the Kalman filter, which is estimating the next state on the basis of current state at time k .

)|(=)|1+(^^

kkXAkkX

2) Searching to get Y(k) through Mean Shift State )1(

^+kX is pre-estimated in the

)|1(^

kkx + and )|1(^

kky + representatives the center

810

coordinate of pre-estimating target. Component )|1(^

kkx + ,

)|1(^

kky + is setted as the initial position of search window which is based on Mean Shift algorithm. Optimal target position is searched in the neighborhood of center position.by starting up Mean Shift algorithm 3)Tracking feature is updated according to the characteristics of the current and the previous frame target area.

Thus, the combination of Kalman filtering with Mean Shift algorithm is completed B. The Processing of Occlusion Problem

The difference between target component )1+(^

kx and

)1+(^

ky of current frame target pro-estimated value )1+(^

kX and the value of measurement vector Y(k) is expressed as filter's residual, denoted as:

2^

22

^

1 ))1+(-)((+))1+(-)((=)( kykykxkykr

In order to judge the occlusion, setting a threshold a=10. If r(k)>a, Kalman filter stops working. At this time, the starting point of the next frame’s target is predicting by using the center position of the previous frames target. Then, target’s real position of current frame is searched by using Mean Shift. At the same time, still calculating the value of r(k), compared with a, if r(k)<a, Kalman filter restarts in the next frame. C. The Process of target Tracking Algorithm

V. CONCLUSION AND ANALYSE

The sequences are available at http://www. elec.qmul.ac.uk/staffinfo/andrea/multi-feature.html. They are Targets H5 (Toni) in the experiment. Tracking target is face of Toni. Image size is 720×576 and total frame is 910. In the movement, Toni has the effect of light. Occlusion and Rapid movement exists when he is moving. The comparison between traditional Mean Shift and improved Mean Shift is showed in Figure 5.1. Red line is improved algorithm, and black line is common algorithm. It can be seen from the chart that Bhattcharyya coefficient value of improved algorithm is close 1. Thus tracking performance is better.

Figure 5.2 and Figure 5.3 is face of Toni tracking effect which adopts respectively Mean Shift algorithm and improved algorithm. Frame 210, 390, 394, 400, 425, 481, 497 and 510 is respectively typical representative. Frame 210 shows the case of the rapid movement. Frame 390, 394, 400 and 425 represents the rotation of the body. Frame 481 and 497 represents the situation of face which blocked by right-hand and left-hand. Frame 510 represents the impact of light.

In the contrast experiment of Figure 5.2 and Figure 5.3, the improved algorithm correctly tracks target for the face’s fast moving in the case of light and occluded. Because tracking region has relativity in the former and last frame, it induces predictive position in the last frame is inside the neighborhood range of tracking area’s measured value. So the local maximum of searched is correct face’s position.

150 200 250 300 350 400 450 500 550

0.74

0.76

0.78

0.80

0.82

0.84

0.86

0.88

0.90

0.92

0.94

Bha

ttcha

ryya

coe

ffic

ient

Frame

MF MF+KL

Fig. 5.1 Comparison of the tracking performance about two algorithms

Original feature extracting

Whether is several frames

of initial

Feature refreshing

Using kalman to position prediction

Using mean shift algorithm to tracking

Using mean shift algorithm to tracking

Feature refreshing

yes no

Moving target positioning

collecting image sequence

GCM fused measured value

Fig. 4.1 The Process of -target Tracking Algorithm

811

Fig. 5.2 The tracking result based on the algorithm of Mean Shift

(Frames 210,390,394,400,425,481,497and 510)

Fig. 5.3 The tracking result based on Mean Shift and improved Kalman filtering algorithm (Frames 210,390,394,400,425,481,497and 510)

REFERENCES [1] Wang S, Ai H. z, He K. Z, Difference-image-based

multiple motion targets detection and tracking. Chinese

Journal of Image and Graphics, 1999, 4 (6):470~175.

[2] Tai J, Tsang S, Lin C, Song K. Real-time image tracking

for automatic traffic monitoring and enforcement

application. Image and Vision Computing, 2004, 22(6):

485~501.

[3] Zhu Z, Xu G, Yang B, Shi D, Lin X. VISATRAM: A

real-time vision system for automatic traffic monitoring.

Image and Vision Computing, 2000, 18(10): 781~794.

[4] Pai C, Tyan H, Liang Y, Liao H M, Chen S. Pedestrian

detection and tracking at crossroads. Pattern Recognition,

2004, 37(5):1025~1034.

[5] D Comaniciu, P Meer. Mean shift: A robust approach

toward feature space analysis [J]. IEEE Transaction on

Pattern Analysis and Machine Intelligence (S0162-8828),

2002, 24(5): 603 - 619.

[6] Yilmaz A. Object Tracking by Asymmetric Kernel Mean

Shift with Automatic Scale and Orientation Selection[C].

//IEEE Conference Computer Vision and Pattern

Recognition. Minneapolis, MN, USA: IEEE, 2007: 1

[7] Collins R. Mean- shift blob tracking through scale space

[C]//IEEE Conference Computer Vision and Pattern

Recognition, 2003, 2:234- 240.

[8] Feng Zhiquan, Meng Xiangxu. A Strong Tracking

Extended Kalman Filter and Its Applications in Moving

Human Hand Tracking. 2006, 18(7): 1060~1066.

812