Multisensor Data Fusion in Object Tracking Applications

S.A.Quadri and Othman Sidek Collaborative µ-electronic Design Excellence CentreUniversiti Sains Malaysia

Presentation Overview

1)Introduction to Data fusion• Definition• Applications

2) Object tracking • Definition • Decentralized Kalman filter 3)Simulation and results

DATA FUSION

•Data-fusion is a problem-solving technique based on the idea of integrating many answers to a question into a single; best answer.

•Process of combining inputs from various sensors to provide a robust and complete description of an environment or process of interest.

•Multilevel, multifaceted process dealing with the automatic detection, association, correlation , estimation, and combination of data and information from single and multiple sources.

“Properly said, fusion is neither a theory nor a technology in its own. It is a concept which uses various techniques pertaining to information theory, artificial intelligence and statistics [1]”

[1] David L. Hall & James Llinas, “Introduction to Multisensor Data Fusion”, IEEE , Vol. 85, No. 1, pp. 6 – 23, Jan 1997.

Multisensor data fusion Applications :Military application :•Location and characterization of enemy units and weapons•High level inferences about enemy situation•Air to air or surface to air defense•Ocean monitoring•Battlefield intelligence•Strategic warning

Non military applications:•Condition based maintenance•Detection of system faults•Robotics•Medical•Environmental monitoring•Location and identification of natural phenomena

DATA FUSION APPLICATION IN ESTIMATION PROBLEMS

Application Dynamic system Sensor Types

Process control Chemical plant Pressure

Temperature

Flow rate

Gas analyzer

Flood predication River system Water level

Rain gauge

Weather radar

Tracking Spacecraft Radar

Imaging system

Navigation Ship Sextant

Log

Gyroscope

Accelerometer

Global Navigation satellite system

Multisensor data fusion provides significant advantages over single source data•Improves accuracy •Improves precision •Supports effective decision making

Schematic diagram of data fusion

Object tracking

Object tracking can be defined as the problem of estimating the trajectory

of an object in the image plane as it moves around a scene.

•A tracker can also provide object-centric information, such as

• Orientation

• Area

• Shape of an object

Tracking objects can be complex due to many reasons like

• Loss of information

•Introduction of noise

•Complex object motion

•Non-rigid or articulated nature of objects

•Complex object shapes

• Lack of real-time processing requirements.

SIMULATION OF TARGET TRACKING AND ESTIMATION USING DATA FUSION

Objective: Target tracking and estimation of a moving object

Sensors used: Multiple sensors

=> Position estimation sensors

=> Velocity estimation sensors

Need for heterogeneous multi sensors ?

=>It is not possible to deduce a comprehensive picture about the entire target scenario from each of the pieces of

evidence alone.

=>Due to the inherent limitations of technical features characterizing each sensor.

Coordinate system Selected : Cartesian coordinate system

Technique applied : Multisensor data fusion using Kalman filter

The need of Kalman filter ?

• System state cannot be measured directly• Need to estimate “optimally” from measurements

Measuring Devices Estimator

MeasurementError Sources

System State (desired but not known)

External Controls

Observed Measurements

Optimal Estimate of

System State

SystemError Sources

System

Black Box

Kalman filter

The Kalman filter produces estimates of the true values of measurements and their associated calculated values by

•Predicting a value,

•Estimating the uncertainty of the predicted value,

•and computing a weighted average of the predicted value and the measured value.

•The most weight is given to the value with the least uncertainty.

•The estimates produced by the method tend to be closer to the true values than the original measurements .

•Weighted average has a better-estimated uncertainty than either of the values that went into the weighted average.

Kalman filter uses

•System's dynamics model (i.e., physical laws of motion),

• Known control inputs to that system,

•Measurements from sensors to form an estimate of the system's varying quantities (its state)

that is better than the estimate obtained by using any one measurement alone.

Kalman Filtering Equations

ŷ-k = Ayk-1 + Buk

P-k = APk-1AT + Q

Prediction (Time Update)

(1) Project the state ahead

(2) Project the error covariance ahead

Correction (Measurement Update)

(1) Compute the Kalman Gain

(2) Update estimate with measurement zk

(3) Update Error Covariance

ŷk = ŷ-k + K(zk - H ŷ-

k )

K = P-kHT(HP-

kHT + R)-1

Pk = (I - KH)P-k

A – Dynamic coefficient matrix of a continuous linear differential equation defining dynamic systemB – Coupling matrix between random process noise and state of a linear dynamic system Q- Covariance matrix of process noise in the system state dynamics R - Covariance matrix of measurement noise in the system state dynamicsP- Covariance matrix of state estimation uncertainty K – Kalman gain ; H – Measurement sensitivity ; z- measurement vector :u – control input

Decentralized Kalman filter

Simulation has been carried out by with two-dimensional state model of the moving object along x; y and z directions.The program is executed in Matlab environment .

Result and conclusion As shown in figure estimation using state vector fusion method using Kalman filter is more close and accurate to actual track.

Future work:

•Finding the parameters which effect the data fusion performance •Finding the effect of process noise and measurement noise

Sample code% Missile_Launcher tracking Moving_Object using kalman filterclear all%% define our meta-variables (i.e. how long and often we will sample)duration = 10 %how long the Moving_Object fliesdt = .1; %The Missile_Launcher continuously looks for the Object-in-motion ,%but we'll assume he's just repeatedly sampling over time at a fixed interval%% Define update equations (Coefficent matrices): A physics based model for A = [1 dt; 0 1] ; % state transition matrix: expected flight of the Moving_Object (state prediction)B = [dt^2/2; dt]; %input control matrix: expected effect of the input accceleration on the state.C = [1 0]; % measurement matrix: the expected measurement given%% define main variablesu = 1.5; % define acceleration magnitudeQ= [0; 0]; %initized state--it has two components: [position; velocity] of the Moving_Object Q_estimate = Q; %x_estimate of initial location estimation of where the Moving_Object Moving_ObjectAccel_noise_mag = 0.05; %process noise: the variability in Q_loc = []; % ACTUAL Moving_Object flight pathvel = []; % ACTUAL Moving_Object velocityQ_loc_meas = []; % Moving_Object path that the Missile_Launcher sees%% simulate what the Missile_Launcher sees over timefigure(2);clf figure(1);clf % Generate the Moving_Object flight Moving_ObjectAccel_noise = Moving_ObjectAccel_noise_mag * [(dt^2/2)*randn; dt*randn]; Q= A * Q+ B * u + Moving_ObjectAccel_noise; ......................... pauseend%plot theoretical path of Missile_Launcher that doesn't use kalman plot(0:dt:t, smooth(Q_loc_meas), '-g.')%plot(0:dt:t, vel, '-b.')%% Do kalman filtering%initize estimation variables......................... % Plot the resultsfigure(2);plot(tt,Q_loc,'-r.',tt,Q_loc_meas,'-k.', tt,Q_loc_estimate,'-g.');

%data measured by the Missile_Launcher ……………………….. %combined position estimate mu = Q_loc_estimate(T); % mean sigma = P_mag_estimate(T); % standard deviation y = normpdf(x,mu,sigma); % pdf y = y/(max(y)); hl = line(x,y, 'Color','g'); % or use hold on and normal plot axis([Q_loc_estimate(T)-5 Q_loc_estimate(T)+5 0 1]); %actual position of the Moving_Object plot(Q_loc(T)); ylim=get(gca,'ylim'); line([Q_loc(T);Q_loc(T)],ylim.','linewidth',2,'color','b'); legend('state predicted','measurement','state estimate','actual Moving_Object position') pauseend

?

Thank You