muri annual review meeting john fisher, al hero, alan willsky november 3, 2008
DESCRIPTION
Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation Graphical Models for Resource-Constrained Hypothesis Testing and Multi-Modal Data Fusion. MURI Annual Review Meeting John Fisher, Al Hero, Alan Willsky November 3, 2008. Topics. - PowerPoint PPT PresentationTRANSCRIPT
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 1
Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation
Graphical Models for Resource-Constrained Hypothesis Testing and Multi-Modal Data Fusion
MURI Annual Review Meeting
John Fisher, Al Hero, Alan Willsky
November 3, 2008
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 2
Topics
Design of Discriminative Sensor Models Under Resource Constraints Discriminative tree models
Distributed PCA Multi-modal Data Fusion
LIDAR/EO registration
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 3
Common Thread
These problems are formulated as inference in graphical models in which:
Assumptions or partial specifications of the structures and associated parameters of the latent variables lead to approximate inference methods
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 4
Max-weight Discriminative Forests
Discriminative Generalization of Chow-Liu Method for Learning Discriminative Trees Kruskal’s Algorithm Use of J-Divergence
Interpretation & Performance bounds
Algorithm, Proof Sketch Empirical Results
Small Sparse & Non-Sparse Models
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 5
Trees admit a simple factorization
Tree models described by marginal properties on the vertex set pair-wise relationships on the edge set
Entropy has a similarly simple decomposition
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 6
The closest tree model in a K-L divergence sense to any distribution can be found by solving a max-weight spanning-tree (MWST) problem over pair-wise mutual information terms, (Chow-Liu ‘68).
Information Projection Result
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 7
Many algorithms for solving the max-weight spanning (Prim, Kruskal, reverse-delete, Chazelle, etc.)
Kruskal’s is of particular interest Greedy Yields a sequence of optimal k-edge forests
Solving the MWST Problem
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 8
Proof Sketch of Kruskal’s Algorithm
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 9
J-Divergence
This is a well studied information measure (Jeffreys ’46)
Difference between the expected value of the log-likelihood ratio under each hypotheses for binary hypothesis
Upper and lower bound on probability of error (Hoeffding & Wolfowitz ’58, Kailath ’67)
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 10
J-Divergence
We’ll consider an alternative measure
where
If we restrict , to be trees then it turns there is an efficient MWST algorithm for jointly learning tree models for p and q which optimizes
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 11
Multi-valued edge weights
If pA and qB are constrained to be trees:
where
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 12
Multi-valued edge weights
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 13
On the use Kruskal’s Algorithm
The sequence of forests are optimal. At each iteration, the number of edges in the
respective models may be different Early termination possible (i.e. pA and/or qB may
not be spanning trees upon termination) The proof optimality is similar to the generative
case, the primary difference is that some edges may be precluded in one tree, but not the other.
The ws,t become single-valued The max is always reduced
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 14
Empirical Results
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 15
Empirical Results
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 16
A somewhat contrived example
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 17
A somewhat contrived example
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 18
Comments
We have presented a discriminative version of the Chow-Liu method for learning trees using an approximation to J-divergence
This was enabled by defining a multi-valued weight function where elements map to edge choices in one or both tree models.
The method results in structures that differ from the generative models.
Early termination is possible resulting in a forest or non-spanning tree.
Empirical Results Marginal improvements for small graphs which are already sparse. Illustrative example demonstrated that improvement over
generatively learned trees can be significant. Open Questions/Directions
Can one develop a Hoeffding-Wolfowitz type bound using the approximate measure?
What is the behavior on sparse graphs as the size of the graph grows?
Evaluate performance with empirical distributions.
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 19
Outline
Progress in front-end processing and fusion New: graphical models for distributed
decomposable PCA New: graphical models for
hyperspectral image unmixing
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 20
Part III: High level fusion
Application of hierarchical graphical models Distributed decomposable PCA Hyperspectral imaging and unmixing
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 21
Decomposable PCA
Principle components analysis (PCA) is a model-free dimensionality reduction technique used for high level data fusion (variable importance, regression, variable selection)
Deficiencies: PCA does not naturally incorporate priors on
Dependency structure (graphical model) Matrix patterning (decomposability)
Scalability problem: complexity is O(N^3) Unreliable/unimplementable for high dimensional data Ill-suited for distributed implementation, e.g., in
sensor networks
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 22
Networked PCA
Network model: measure sensor outputs Xa, Xb, Xc
Two cliques {a,c} and {b,c}
Separator {c} Decomposable model:
covariance matrix R unknown but conditional independence structure is known.
PCA of covariance matrix R finds linear combinations y=UTX that have maximum or minimum variance
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 23
DPCA formulation
Precision matrix K=R-1 For decomposable model K has
structure
General Representation
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 24
1-dimensional DPCA
PCA for minimum eigenvector/eigenvalue solves
Key observation: This constraint is equivalent to
where
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 25
Extension to k-dimensional DPCA
k-dimensional PCA solves sequence of eigenvalue problems
Dual optimization
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 26
k-dimensional DPCA (ctd)
Dual maximization splits into local minimization with message passing
Message passing
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 27
Tracking illustration of DPCA
Scenario: Network with 305 nodes representing three fully connected networks with only 5 coupling nodes
C1 = {1, · · · , 100, 301, · · · , 305}, C2 = {101, · · · , 200, 301, · · · , 305}, and C3 = {201, · · · , 300, 301, · · · , 305}.
Local MLEs computed over sliding time windows of length n = 500 400 samples overlap.
Centralized PCA computation: EVD O(305)^3 flops DPCA computation: EVD O(105)^3 flops + message
passing of a 5x5 matrix M
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 28
DPCA min-eigenvalue tracker
Iteration 1
Iteration 2
Iteration 3
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 29
DPCA network anomaly detection
SNVA
STTL
LOSA KSCY
HSTN
DNVRCHIN
IPLS
ATLA
WASH
NYCM
Multiple measurement sites (Abilene)
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 30
DPCA anomaly detection
PCA (centralized)
DPCA (E-W decomp)
DPCA (E-W-S decomp)
DPCA (Random decomp)
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 31
Discussion
Take home message: Combination of model-free dimensionality reduction and model-based graphical model can significantly reduce computational complexity of PCA-based high-level fusion
Complexity scales polynomially in clique size not in overall size of problem. Example: 100,000 variables with 500 cliques each of size 200
Centralized PCA: complexity is of order 1015 DPCA: complexity is of order 106
If one can impose similar decomposability constraints on graph Laplacian matrix, be extended to non-linear dimensionality reduction: ISOMAP, Laplacian eigenmaps, dwMDS.
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 32
Outline
Laser Radar 3D Model Generation
Delaunay mesh formation Model Demo
Camera Model Statistical Registration Methods Results
Probing Experiments Registration Demo
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 33
Laser Radar
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 34
Laser Radar
3D point cloud Not gridded Typical lateral resolution
of 0.5 – 1.0 meters
Linear mode sensors: Slower collects Probability of detection
(pdet) values Geiger mode sensors:
Faster collects Geometry only
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 35
3D Model Generation
Ladar point cloud
Inferred surface
Registered optical image
Projective texture mapping
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 36
3D Model Generation
Point cloud Mesh 3D Model
Optical image
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 37
Mesh Generation: Delaunay Triangulation
Maximizes the minimum angle of all the angles of the triangles in the triangulation
Circle circumscribed on any triangle does not contain any other points
Dual graph of Voronoi Tessellation
Provides rough estimate of underlying structure
http://en.wikipedia.org/wiki/Delaunay_triangulation
x
z
y
x
z
y
x
z
y
z
x
y
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 38
Automatic Registration Challenges
GPS/INS has low precision Matching colors with geometry (inherently low
mutual information) Occlusion reasoning with point cloud projection 3D rendering Resolution differences Previous work
Line correspondences (Frueh, Sammon, Zakhor 2004)
Vanishing points (Ding, Lyngbaek, Zakhor 2008)
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 39
Homogeneous Coordinate System
w
v
u
X
wv
wu
v
u~
~
t
z
y
x
X
tz
ty
tx
z
y
x
~
~
~
2D (optical)
3D (ladar)
Homogeneous Euclidean
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 40
Projective Camera
t
z
y
x
pppp
pppp
pppp
w
v
u
T
34333231
24232221
14131211
z
y
x
333231
232221
131211
100
010
001
100
00
00
|
C
C
C
rrr
rrr
rrr
f
f
CIKRT
view)of field s(determinelength Focal :
position projection ofcenter Camera :
angles)Euler 3by ized(parametermatrix Rotation :
matrixn calibratio Camera :
matrix projection Camera :
z
y
x
f
C
C
C
R
K
T
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 41
Projective Camera
x
z
y
Field of view
(determined by focal length)
Image plane
View direction
(characterized by 3 Euler angles)
Center of projection
z
y
x
C
C
C
f
Parameters to estimate:
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 42
Initial Registration
Manual calibration is not tractable User selected correspondence points Minimization of sum of algebraic error
T is parameterized by f, Cx, Cy, Cz, α, β, γ Levenberg-Marquardt iterative
optimization (derivative free)
2alg ),(min i
ii
TTd XX
22algalg )ˆˆ()ˆˆ(ˆ,(),( vwwvwuuwdTd )XXXX
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 43
Statistical Registration Methods
Commonly used for registration of multi-modal medical imagery
Information theoretic similarity measure with optimization algorithm
Ladar Image
Optical Image
Feature detection / classification
Feature detection / classification
Registration
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 44
Example: Joint Entropy
Pdet ValuesLuminance
Optical imageiTi
iT
TJE vupvupT ]),([log)],([minarg
T: camera projection matrixu: optical image features/labelsv: ladar image features/labelsp(·): probability mass function
JE Registration
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 45
Probing Experiments
-100 -80 -60 -40 -20 0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x displacement (meters)
Nor
mal
ized
join
t en
trop
y
-100 -80 -60 -40 -20 0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
y displacement (meters)
Nor
mal
ized
join
t en
trop
y
-100 -80 -60 -40 -20 0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
z displacement (meters)
Nor
mal
ized
join
t en
trop
y
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Field of view displacement (degrees)
Nor
mal
ized
join
t en
trop
y
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 46
Probing Experiments
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
a displacement (degrees)
Nor
mal
ized
join
t en
trop
y
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
b displacement (degrees)
Nor
mal
ized
join
t en
trop
y
-20 -15 -10 -5 0 5 10 15 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
g displacement (degrees)
Nor
mal
ized
join
t en
trop
y
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 47
Automatic Registration Demo
Minimum joint entropy with optical luminance and ladar elevation
Downhill simplex optimization (derivative free)
Ladar rendered using OpenGL Entire ladar data set in graphics
memory Approximate initial registration
MURI: Integrated Fusion, Performance Prediction, and Sensor Management for Automatic Target Exploitation 48
Unsupervised LIDAR/Optical Registration