alan steinberg and robert pack space dynamics laboratory/utah state university presented at
DESCRIPTION
Pixel-Level Fusion of Active/Passive Data for Real-Time Composite Feature Extraction and Visualization. Alan Steinberg and Robert Pack Space Dynamics Laboratory/Utah State University Presented at NATO IST-036/RWS-005 Halden, Norway 10-13 September 2002. Concept of Pixel-Level Fusion. - PowerPoint PPT PresentationTRANSCRIPT
[1]
Pixel-Level Fusionof Active/Passive Data
for Real-Time Composite Feature Extraction and Visualization
Alan Steinberg and Robert Pack
Space Dynamics Laboratory/Utah State University
Presented atNATO IST-036/RWS-005
Halden, Norway
10-13 September 2002
[2]
Concept of Pixel-Level Fusion
• Active Sensor – LADAR
• Passive Sensor – Electro-Optic (EO) Radiation
• Fused into One Data Set
RANGE DATA FROM MULTI-
CHANNEL LADAR
PASSIVE IR IMAGE DATA
+
FUSED RANGE IMAGE
[3]
Enables Observation Through Partial Obscuration
[4]
Employment Concept
IndividualPixel
r2
r3
r4
r1
IndividualPixel
r1
r2
r3r4
r5
Cloud Layers Cloud Layers
Ground Clutter
Upper Canopy
Lower Canopy
[5]
Fused LADAR/EO Image 1(Co-Registered and Georeferenced at the Pixel Level)
LADAR Image EO Image
[6]
LADAR Image EO Image
Fused LADAR/EO Image 2(Co-Registered and Georeferenced at the Pixel Level)
[7]
Orthoprojection of the Two 3D Images Combined in Geographic Space
First Image
Second Image
[8]
Profile Views in 3D Viewer
First LADAR/EO Image
First and Second LADAR/EO Images Combined
[9]
Visualization Concept -
Data Volume Exploration and Clutter Removal
[10]
3D Images of Cluttered Environment
Image Plane
Vegetation Canopy
Ground Plane
Terrain (per DTED or detected ground)
[11]
Predict Ground
Image Plane
Bottom of Canopy Layer(Terrain + 3m)
Ground Plane
Terrain (per DTED or detected ground)
[12]
Slice Volume at Bottom of Canopy
Image Plane
Bottom of Canopy Layer(Terrain + 3m)
Ground Plane
Terrain (per DTED or detected ground)
Residual Shadowed Regions: PD 0
[13]
Example of Canopy Slicing
Orthoprojected Using LADAR – Trees RemovedOblique EO Scene
Residual Shadowed Region: PD 0
Ground with Trees Removed
[14]
Two Shots Combined – Trees Removed
Residual Shadowed Region: PD 0
Corresponding EO Image
[15]
Target Hidden Under Tree
Hidden Target
[16]
Target Exposed After Tree Removal
Exposed Target
[17]
Conclusions (1):Pixel-Level Fusion of Active/Passive Data for Real-Time
Composite Feature Extraction & Visualization
• Pixel-Level fusion of LADAR + EO data facilitates target detection & recognition in highly occluded environments
• Clutter
– tree canopies, clouds, etc. –
can be removed through geometric filters
[18]
Visualization Implication
[19]
Some Visualization Implications
• Target Detection & Recognition
> Probabilities of Detection
> Probabilities of Target Presence (prior & posterior)
> Exploitation of Negative Data
• Temporal Evolution of Estimation and Expectations
> Out-of Sequence Data
> Situation Projection
> Expected Updates
[20]
Prior & Posterior Detection Probabilities; e.g. Presentation of Negative Data
Target Detectability
Map
0.75 - 1.00
0 - 0.25
0.25 – 0.50
0.50 – 0.75
pD(x,t+n|t,A)
Prior or Posterior Probability Density
(e.g. Track Projection Uncertainty)
x = Target state (type & location, etc.)
t, t+n = Update times
A = Action plan (sensor route, sensor controls, processing controls, etc.)
p(x,t+n|t)
ExpectationsMap
Likely false alarm
Likely missed
detections
p(x,t+n|t)pD(x,t+n|t,A)
[21]
Some Visualization Implications
• Target Detection & Recognition
> Probabilities of Detection
> Probabilities of Target Presence (prior & posterior)
> Exploitation of Negative Data
• Temporal Evolution of Estimation and Expectations
> Out-of Sequence Data
> Situation Projection
> Expected Updates
[22]
Revised Estimates of Track Histories (MTI + Imagery)
TimeELEMENTARY SCHOOL
MISSILE ASSEMBLY FACILITY
MTI Reports
Imagery Reports
Latency Issues:Out of Sequence Data
Initial Estimates of Track Histories (MTI only)
TimeELEMENTARY SCHOOL
MISSILE ASSEMBLY FACILITY
[23]
Adaptive Information ExploitationProcess
• Acknowledge uncertainties
• Estimate length of time for which decision can be deferred
• Estimate probability of resolving ambiguity in time
> Given current collection plan
> By redirecting current resources
> By adding resource
• Estimate Cost & Net Utility of Candidate Action Plans
• Select & Implement Revised Plan
[24]
Estimated History Estimated Present Situation Projection Situation Estimated History Estimated Present Situation Projection Situation Situation Time
-40 -30 -20 -10 0 +10 +20 +30 +40
Received Reports Expected CoverageReceived Reports Expected Coverage
Tgt Selection Decision Tgt Engagement Decision
Visualizing Time-Varying Situation Estimation & Expectation (1 of 3)
Current Estimate of Present Situation
Current Estimate of Present Situation
Report Time (Plan A)
[25]
Estimated History Estimated Present Situation Projection Situation Estimated History Estimated Present Situation Projection Situation Situation Time
-40 -30 -20 -10 0 +10 +20 +30 +40
Received Reports Expected CoverageReceived Reports Expected Coverage
Tgt Selection Decision Tgt Engagement Decision
Visualizing Time-Varying Situation Estimation & Expectation (2 of 3)
Current Projection of Future Situation
Current Projection of Future Situation
Report Time (Plan A)
[26]
Visualizing Time-Varying Situation Estimation & Expectation (3 of 3)
Report Time (Plan A)Estimated History Estimated Present Situation Projection Situation
Estimated History Estimated Present Situation Projection Situation Situation Time
-40 -30 -20 -10 0 +10 +20 +30 +40
Received Reports Expected CoverageReceived Reports Expected Coverage
Tgt Selection Decision Tgt Engagement Decision
Expected Update of Present Situation
Expected Update of Present Situation
[27]
Conclusions
• Pixel-Level fusion of LADAR + EO data facilitates target detection & recognition in highly occluded environments> Clutter – tree canopies, clouds, etc. – can be removed over
time through geometric filters
• Enables visualization of time-varying situation estimation & expectation> Presentation of Detection Probabilities (prior & posterior):
e.g. Presenting Negative Data
> Presentation of Temporal Evolution of Estimates and of Expectations
– Out-of Sequence Data
– Situation Projection
– Expected Updates & Information Evolution
[28]
Thank You
Mange Takk!