oklahoma state university generative graphical models for maneuvering object tracking and dynamics...
TRANSCRIPT
Oklahoma State University
Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis
Xin Fan and Guoliang Fan
Visual Computing and Image Processing Lab
School of Electrical and Computer Engineering
Oklahoma State University
4th Joint IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum(OTCBVS'07)Minneapolis, MN, USA, June 22, 2007
2
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Problem StatementIntroduction Problem Statement
Related Work
Generative model
Experimental Results
Conclusions
Motion models Deal with object movements Why important?
complex motion patterns, e.g., maneuvering no good appearance model, e.g., low SNR provide good prediction for robust and efficient tracking
Challenges Hardly predict maneuvering actions Model constraints
A motion model that incorporates constraints
3
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Our observations Maneuvering actions are due to forces and torques.
forces and torques cause kinematic changes
Newton equations for rigid body motion Rigid body motion VS point motion Newton equations
Forces are dependent on kinematics Limited output power of engines.
Uncertainties exist, e.g., air resistance, road friction, mechanical instability, etc.
Introduction Problem Statement
Related Work
Generative model
Experimental Results
Conclusions z
4
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Problem FormulationIntroduction Problem Statement Problem Formulation
Related Work
Generative model
Experimental Results
Conclusions
Switching statistical models for maneuvering variables (forces and torques)
Maneuvering actions are due to forces and torques.
Newton equations to define kinematics evolution densities
Newton equations of rigid body motion
Rayleigh distribution to model velocity-force constraints
Physical constraints reveal how forces are dependent on kinematics.
Organize these dependencies with a probabilistic graphical model
5
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Related work
White Gaussian noise acceleration (WGNA) Point target assumption
Miller’s condition mean estimation Rigid Newton dynamics Jump-diffusion process, not sequentially
Switching Linear dynamic system (SLDS) or Jump Markov linear system (JMLS)
Discrete switching variables for maneuvering actions No explicit physical dynamics
Inference algorithms IMM works for Gaussian densities BP works for tree structures Sampling based approximation
Introduction Problem Statement Problem Formulation
Related Work
Generative model
Experimental Results
Conclusions
6
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative Model - StructureIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure
Experimental Results
Conclusions
Generative model How forces and torques generate kinematics changes how kinematics generate observations
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations Frames
Velocity
Position
Orientation
Forces
Torques
7
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model-Cause variablesIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables
Experimental Results
Conclusions
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations
Switching continuous probabilistic models Specify three switching normal distributions for forces. Ternary uniform mixture for torques (angular velocity)
8
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints
Experimental Results
Conclusions
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations
Newton equations Investigate the dynamics of 3D rigid motion Define the kinematic dependence by Newton equ
ations of rigid body motion.
9
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints
Experimental Results
Conclusions
Newton equations for 3D rigid motion
τh
fp
Simplified for ground vehicles
x y
z
f
xy
x
z
z
y
x fv
v
v
v
0
1
0
0
z
y
x
v
v
y
x
cossin
sincos
zzzJ
p-linear momentum and f- force
h - angular momentum and τ- torque
10
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model – Temporal constraintsIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints
Experimental Results
Conclusions
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
11,1 ttztt nT
vttttztt nfT 1111,1
~ vωvv
rtttt nvT 111 )( Arr
kinematics dependency via Newton equations
Velocity
Orientation
Position
11
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model- VF constraintsIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints
Experimental Results
Conclusions
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations
Rayleigh distribution for velocity-force constraints
Driving force conditional on velocities
Resistance force conditional on velocities
12
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model - Likelihood
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations
Simple template matching to define likelihood
Introduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood
Experimental Results
Conclusions
13
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Generative model - InferenceIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results
Conclusions
ft-1 ft+1ft
rt-1 rt+1rt
vt-1 vt-1vt
Zt-1 Zt+1Zt
Cause variables
Effect variables
Effect variables
Cause variables
Observations
Predict with temporal densities
Evaluate weights with likelihood
MCMC to generate samples of forces
SMC based inference algorithm
14
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Experiments – Simulated data
Compared with a particle filter (PF) for JMLS Tracking with coupled linear and angular motion
No coupling
Introduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results Simulated data
Conclusions
Ours
JMPF
0 10 20 30 40 50 60
Time
Orie
ntat
ion
Ground Truth
GMJMPF
0 10 20 30 40 50 60
Time
For
ce
Ground Truth
GM
JMPF
0 10 20 30 40 50 60Time
Vel
ocity
Ground Truth
GM
JMPF
-10 0 10 20 30 40 50 60 70-20
0
20
40
60
80
100
120
140
X Coordinate
Y C
oord
inat
e
Ground Truth
GM
JMPF
15
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Experiments – Simulated data
Compared with a particle filter (PF) for JMLS Tracking with coupled linear and angular motion
Has coupling
Introduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results Simulated data
Conclusions
JMPF
Ours-20 0 20 40 60 80 100 120 140 160 180
-10
0
10
20
30
40
50
60
70
80
90
X Coordinate
Y-c
oord
inat
e
Ground Truth
GM
JMPF
0 10 20 30 40 50 60-5
0
5
10
Time
Vel
ocity
Ground Truth
GMJMPF
0 10 20 30 40 50 60Time
For
ce
Ground Truth
GM
JMPF
0 10 20 30 40 50 60
Time
Orie
ntat
ion
Ground Truth
GMJMPF
16
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Experiments – Simulated data
Compared with a particle filter (PF) for JMLS Tracking with velocity-force constraints
Introduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results Simulated data
Conclusions
0 10 20 30 40 50 60Time
Orie
ntat
ion
Ground truth
Proposed method
JMPF
0 10 20 30 40 50 60Time
Forc
e
Ground truth
Proposed method
JMPF
0 10 20 30 40 50 60Time
Vel
ocity
Ground Truth
GM
JMPF
0 10 20 30 40 50 60X-Coordinate
Y C
oord
inat
e
Ground Truth
GM
JMPF
17
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Experiments – Real world video
Compared with constant velocity constant turn (CVCT) model
Introduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results Simulated data Real world video
Conclusions
OursCVCT
0 50 100 150 200 250 300
Time
Orie
ntat
ion
GMCVCT
250 300 350 400 450 5000
50
100
150
200
250
300
X Coordinate
Y C
oo
rdin
ate
GMCVCT180th Frame
0 50 100 150 200 250 300
Time
Ve
loci
ty
GM
CVCT
18
X. Fan and G. Fan, Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis , OTCBVS'07.
Conclusions and future workIntroduction Problem Statement Problem Formulation
Related Work
Generative model Structure Cause variables Temporal constraints Velocity-force constraints Likelihood Inference
Experimental Results Simulated data Real world video
Conclusions
Conclusions Graphical model for maneuvering targets, which encode
the Newtonian dynamics in a probabilistic framework. Explicitly and directly build the cause-effect relationship Feedback constraint from velocity to the forces
Future work Handle multiple views. Multiple targets with data association