Download - Topic 4: Sensor Processing
Topic 4: Sensor Processing
David L. Hall
Topic Objectives
• Introduce the input side of data fusion processing
• Provide a brief survey of sensor types
• Describe how a generic sensor works• Provide a basis for analysis of
sensors for your selected application
Sensors: The Input Side of Data Fusion
Classes of Input • Traditional “hard” sensors
– Observation of physical phenomena via physical sensors
• “Soft” sensors– Humans as observer/reporters– Direct input (e.g., via cell phone)– Indirect input (web logs, etc.)
• Emerging sensors– Emulation of biological sensors via new nano-scale
technologies (e.g., electronic noses)
The rapid evolution of sensor technology provides opportunities and challenges:Nano and micro-scale sensors include the capability for self-location, wireless communication in self-organizing networks, and web-based serves; sensors maybe mobile and provide extensive on-sensor computations
Examples of New Sensors: Intel Mote
• 32 bit processor• Blue tooth wireless comm.
http://www.intel.com/research/exploratory/motes.htm
Smart Dust Vision
www.sice.umkc.edu/~leeyu/Udic/Groups/SensorNetworksPresentationFINAL_0909.ppt
Examples of New Platforms for Sensor Deployment
> 2.7 B Cell Phones
New space-based environmental sensors
Military platforms
10,000 surveillance cameras in London
Smart factories & buildings
Examples of Military Sensor Platforms
MILITARYPLATFORM
ROLE SENSOR FUNCTIONS TYPICAL SENSORS
Air-to-Air Combat Detect, track and I/D aircraft(friend-foe and type I/D)
Engage hostile aircraft andverify kill
Multi-mode radar IRST, TV IFF ESM
Ground Attack Search, acquire, I/D hostileground targets (mobile, fixed)
Engage targets: hand-offaircraft-to-weapon sensors
Terrain-following radar Imaging/mapping radar Forward looking IR ESM
Anti-Air Warfare Conduct surveillance formilitary ATC hostile targetdetection, I/D
I/D, track and engage hostileaircraft (CAP/SAM)
Air search radars Fire control radars ESM IRST IFF
Surface, Sub-Surface Warfare
Conduct surface/sub-surfacesurveillance for hostile ship-sub-detection, I/D
Coordinate air, surface, sub-surface engagements
Surface search radar Hull-mounted sonar Towed-array sonar ESM
Examples continued
Representative Observable-Sensor PairingsDetectable/measureable emission characteristics
Representative sensors
Radio Frequency Radar warning receivers; Electronic intelligence; Communications intelligence
Infra-red (IR) emissions & contrast Infra-red imagery
Acoustic/seismic Acoustic sensors; Seismometers; Sonar
Optical contrast TV imagery; Direct view optics; Optical augmentation sensors
Radar cross section (RCS) RADAR; Millimeter Wave Radar
Electro-optical (E-O) E-O sensors; LIDAR
Mechanical/structural vibrations Laser-based sensors; accelerometers
The Frequency Spectrum of Sensors
Representative Sensor Characteristics
SENSOR CHARACTERISTIC DESCRIPTION
Detection Performance Detection characteristics (false alarm rate, detection probabilities and ranges) for acalibrated target characteristic in a given noise background
Spatial/Temporal Resolution Ability to distinguish between two or more targets in space or time
Spatial Coverage Spatial volume covered by the sensor, for scanning sensors ( this may be describedby the instantaneous field-of-view, the scan pattern volume and the total field-of-regard achievable by moving the scan pattern)
Detection/Tracking Modes Search and tracking modes performed: Staring or scanning Single or multiple target tracking Single- or multi-mode (track-while-scan/stare)
Target Revisit Rate Rate at which a given target is revisited by the sensor to perform a samplemeasurement (staring sensors are continuous)
Measurement Accuracy Statistical accuracy of sensor measurements
Measurement Dimensionality Number of measurement variables (range, range rate, and spectral features)between target categories
Hard/Soft Data Reporting Sensor outputs are provided either as hard-decision (threshold) reports or as pre-processed reports with quantitative measures of evidence for possible decisionhypothesis
Detection/Track Reporting Sensor reports each individual target detection or maintains a time-sequencerepresentation (track) of the target’s behavior
Inside a Generic Sensor
Steering Control
Adaptive Control
Heuristic Data
Signal Conditioning
- Translation- Analog/Digital- Digital/Analog- Detection
Signal Processing
- Filtering- Transforms- Thresholding- Storage- Special Algorithms
Information Processing/
Decision Making
- Look-up Tables- Bit Map- Heuristics- Declaration Matrix Map
Output Processing
- Buffering- Data Conversion - Coordinate Transforms- Smoothing- Filtering
ENERGY EMISSION SENSOR ELEMENTS
SENSOR GUIDANCE/CONTROL
image
y(t)
• tasking• control data• tip-off information• environmental data
Sensor Input
Time FunctionEnergy
Emission
Input Energy• intentional• unintentional• jamming
Single Sensor Target Detection and Parameter Estimation
SIGNAL PARAMETRIZATION and
DETECTION
determine a presence of signal and represent via
attribute vectorExamples
• Feature extraction - Peak detection - Shape characterization - Statistic summaries• Detection logic - Hypothesis testing - Thresholding• Tracking/geolocation - Position estimation - Kinematic estimation• Attribute estimation
Image
Time
X(t) Time Series Processing
CANONICALTRANSFORMATION
natural Hilbert space representation of
signals
Examples• Fourier• Wavelet• Gabor• Walsh
etc.
• • •
SIGNAL INFORMATION
ANALYSIS
characterize and analyze a signal
Examples• Time series - State space - ARMA - HOS• Matched filter• Range-Doppler• Ambiguity function
DATABASE
• Target characteristics• Environment/sensor models• Signal models
Sensor Selection Matrix (Steinberg)
Radar• Direct Beam• SAR
IR Imagery• Passive• Augmented
TV Imagery• Passive• Augmented
EW Sensors• RWR• ESM
Acoustic/ Seismic
Direct View Optics• Passive• Augmented
Optical Augmentation Sensors
E-O Tracking• IRSTR• LADAR
SENSORTYPE
DetectionRange
DetectionTime
TargetID
RangeMeasurement
Probability of
Detection
Vulnerability to
Detection
Vulnerability to
CM
RELATIVEPERFORMANCE
Poor
Fair
Good
* Steinberg, DFS-87.
Radar: Radio Detection & Ranging
Basic Measurements:Basic Measurements:• Radar cross-section• Frequency • Time
Derived Measurements:Derived Measurements:• Range• Azimuth• Elevation• Velocity• Target Size/Shape• Signature
State DeterminationState Determination• Position, velocity, identity
– Limited target I/D from radar cross-section signature– Target size/shape may be obtained in limited cases
1. Skolnick, M.I., Ed., Radar Handbook , McGraw-Hill, NY, 1970.
2. Barton, D.L., Modern Radar System Analysis , Artech House, Norwood, MA, 1988.
3. Stein, A., "Bistatic Radar Applications in Passive Systems," Supplement to the January 1990 Journal
of Electronic Defense , Horizon House Microwave, Norwood, Massachusetts.
SPY-1 Phased Array Radar
LIDAR: Light Detection and Ranging
Basic Measurements:Basic Measurements:• Optical intensity• Range• Angular information (direction)
Derived Measurements:Derived Measurements:• Visual signature
State DeterminationState Determination• Location
LIDAR is the visual light equivalent of RADAR.
Cruickshank, J.M., and R.C. Harney, Eds., Laser Radar Technology and Applications ,
SPIE Proceedings, Vol. 663, Quebec City, Canada, June 1986.
ELINT: Electronic Intelligence
ELINTDEFINITION: Electronic intelligence – derived from observations of
radar, other non-communication emitters (e.g., NAVequipment)
TACTICAL ELINT: Type Radar/SEI Operations Timeline Location
TECHNICAL ELINT: Reverse Engineering of Design Level of technology Weaknesses to ECM Parametric data for database development
- Power- Antenna- Modulation
PROBLEMS: Collection Series Separating Signal-of-Interest (SOI) Statistical Analysis
Electronic Intelligence Sensors
Basic Measurements:Basic Measurements:• Amplitude• Frequency • Time
Derived Measurements:Derived Measurements:• Received signal-to-noise ratio• Polarization• Pulse shape• Pulse repetition interval• Radio frequency
State DeterminationState Determination• Identity
– General and specific emitter identity– Analysis yields radar design characteristics
Wiley, R.G., Electronic Intelligence: The Analysis of Radar Signals , Artech House, Norwood, MA, 1988
ESM: Electronic Support Measures
Basic Measurements:Basic Measurements:• RF Intensity/Amplitude• Frequency• Range/Time of Arrival• Angular information (direction)
Derived Measurements:Derived Measurements:• Signal signature
State DeterminationState Determination• Location• Type of emitter
.
EW Design Engineers Handbook 1989/1990 , Supplement to the January 1990 Journal of Electronic
Defense , Horizon House Microwave, Norwood, Massachusetts.
AN/ALR-606 Electronic Support Measures (ESM) Receiver (Northrop Grumman)
Communications Intelligence
Human Skills:• Equipment operator• Linguistics• Gisting
Analysis Tools:• Transforms• Graphics• Statistics• Database
Hardware:• Receivers• Turners• Demodulation• Filters
Tactical IntelligenceTechnical Intelligence
IR: Infrared Warning
Basic Measurements:Basic Measurements:• IR intensity• Detection
Derived Measurements:Derived Measurements:• Direction• Intensity
• Spectral Characteristics
State DeterminationState Determination• Identity• Location
Warning of heat sources approaching.Buser, R.G., and F.B. Warren, Eds., Sensors and Sensor Fusion,
Proceedings of the SPIE , Vol. 782, Orlando, Florida, May 1987.
Word to the wise
Sutton Coldfield Observer,Sutton Coldfield Observer, SUTTON COLDFIELD, ENGLAND: SUTTON COLDFIELD, ENGLAND:
“Even when the helicopter is flying between 800 and 1,000 feet”, say police intelligence, their heat seeking sensors can differentiate between a fleeing suspect and a heat producing device. The officers who rushed to the scene of the robbery at a builders site called for helicopter assistance, and the high tech sensor led them to the door of local, Barry Silvester. Detectives burst into the house to discover that the sensor had led them straight to a steaming hot compost heap in the back garden.
Synthetic Aperture Radar
http://www.nnsa.doe.gov/na-20/synthetic_aperture.shtmlhttp://www.nnsa.doe.gov/na-20/synthetic_aperture.shtml
Basic Measurements:Basic Measurements:• Coherent radar cross-section
Derived Measurements:Derived Measurements:• Target/platform shape• Target size• Aim point (direction)
State DeterminationState Determination• Identity
Enhanced radar resolution via coherent processing ofdata collected by a stable moving antenna.
1. Hovanessian, S.A., Introduction to Synthetic Array and Imaging Radars , Artech House, Norwood,
Massachusetts, 1980.
2. Fitch, J.P., Synthetic Aperture Radar, Springer-Verlag, NY, 1988.
3. Hovanessian, S.A., Introduction to Sensor Systems , Artech House, Norwood, Massachusetts, 1988.
Electro-Optical
1. Blouke, M.N., and D. Pophal, Eds., Optical Sensors and Electronic Photography,
Proceedings of the SPIE , Vol. 1071, January 16-18 1989, Los Angeles, California.
Basic Measurements:Basic Measurements:• Picture elements (pixel)• Picture intensity and color
Derived Measurements:Derived Measurements:• Location• Size• Shape
State DeterminationState Determination• Position• Identity
- Can be used to obtain environmental data
Good for size/shape determination, but requires significant processing time.
Representative Classification Results for FLIR and TV Imagery
TRACKED TRUCK CLUTTER CLUTTER TRACKED
TRUCK CLUTTERTRACKED
FLIR IMAGE: TRUE CLASSES FLIR CLASSIFICATIONS
PIXEL-LEVEL FUSION
TRUCK CLUTTERTRACKED TRACKED
CLUTTER TRACKED
TV IMAGE: TRUE CLASSES
TV CLASSIFICATIONS
FEATURE-LEVEL FUSION
Example: Self-Locating/
Self-Calibrating Acoustic SensorNODE PROTOTYPE
BreadboardProcessor
BatterySensor Platform
Node ComponentsNode Components• Acoustic array
• Seismic/tiltmeter array
• GPS, flux-gate compass/magnetometer
• Temp/humidity, solar, soil sensors
• PC/DSP breadboard processor
• Wireless Ethernet networking
FunctionsFunctions• NLOS acoustic/seismic detection, bearing estimation, localization
• Environmental data fusion for real- time data confidence formulation
• Low-cost research tool for proofing concepts and sensors
Input/OutputInput/Output• Acoustic and seismic waves, environmental data, commands via network is input
• Digital files of detection, confidences, position tracks, I/D, predictions of performance in current environment are outputs
Examples of Emerging Sensors
Every soldier a sensor; 4/29/2004Aviation Week
Use of Insect sensing organs (SPIE May 2003)
Plants as observers and sensors
NASA E-Nose (2004)
Signal Propagation
Both the natural physical environment and hostile attempts to degrade sensor effectiveness via countermeasures present complex problemsin the application of data fusion techniques.
Propagation
True SignalDegraded
Received Signal
AtmosphericNoise
Jamming
Multi-path
Attenuation
Jamming and Deception
Random signals may be added to the environment to decreasethe signal-to-noise ratio (SNR). This is called JAMMING.
Specific signals may be added to the environment to createfalse alarms at the receiver. This is called DECEPTION.
Normal Signal Noise
JAMMING
DECEPTION
Camouflage & Deception in nature
http://www.i-am-bored.com/bored_link.cfm?link_id=31398
Examples of on-line photos of animal camouflage and mimicry
http://www.oceanlight.com/lightbox.php?x=camoflage__fish_behavior__fish__animal
http://rainforests.mongabay.com/0306.htm
http://www.nytimes.com/2008/02/19/science/19camo.html?8br
Sensor Analysis Overview
KEY QUESTIONS:KEY QUESTIONS:• What can/should be sensed?• What sensors are available?• Physics of emissions, transmission,
reflection, scattering, detection• CM/CCM environment?• How does clutter noise, multi-path,
interference affect P(D), P(fa)?• Sensor Performance?• Mission constraints on sensing,
data communication?
PurposePurpose:: Select the best suite of sensors to detect, locate, characterize, and identify critical targets/entities
Key Factors:Key Factors:• Sensor selection/availability• Accuracy vs timeliness vs computer resources• Extent to which sensors are Smart• Scheduling, utilization of sensors• Contextual sensor performance
- Environmental effects- Countermeasures
• Sensor control Parameters• Stealth requirements
Analysis Tools and Techniques:Analysis Tools and Techniques:• Static threat vs observability analysis• Engagement timeline analysis• Observation modeling
- Observation prediction- Observation statistics
• Covariance error analysis• Monte Carlo simulation
• Emerging network centric operations– Ubiquitous connectivity– Soldier and civilian hosted sensors– Emerging “every soldier/civilian a sensor” concept
• New types of hybrid “hard/soft” sensors– Sensors carried/worn by soldiers/civilians– Sensors that monitor the condition of people (e.g., using the human body as a
sensor for complex chem/bio phenomena and monitoring the body’s responses)– Self-reporting via soldier as “continuous commentator” – Use of animals and plants as sensors– Emergent social phenomena – mining soldiers chat & blogs for emerging insights
and unconscious insights
• Need for research in– Development of a framework for fusing report-level data from human observers– Mapping and modeling the data flow for fusion involving humans as “soft
sensors” combined with conventional sensors
Mixed input environmentsTraditional and new “hard” and “soft” sensing
General Issues
• What is the general framework for human observing/reporting?
– What is the role of the human as both provider and consumer of information?– What is the process for human observations (e.g., do humans provide
information on a volunteer or ad hoc basis; can or should humans be tasked; is/should there be modulation/filtering/censuring of observations;
– Should human observers be characterized (e.g., self-report of expertise; system “grading” of human observers, etc) – consider Accuweather example
– What is the role of indirect reporting (e.g., blogging; news reports, chat, etc)
• How should knowledge and reports be solicited?– How should knowledge be solicited (e.g., via structured entry forms; via
prompted questions; etc.– What language/structure should be used (free-form text, specialized & restricted
language;– How should uncertainty be represented (e.g., via fuzzy terms, confidence factors,
subjective probabilities– How should 2nd order uncertainty be represented
Issues continued
• What is the framework for human reporting uncertainty representation/fusion with traditional sensors?
• How does “soft” data and web mined data flow into the traditional JDL model components? – Soft data provides contextual information (e.g., human judged
relationships) & input directly into higher levels of JDL processing
– Issues of legacy, out-of-sequence reports, significant differences in time scales require more sophisticated data management functions
Topic 4 Assignments• Preview the on-line topic 4 materials• Writing Assignment 4: Write a 2-page paper describing the
challenges and issues of searching for the Loch Ness Monster (see on-line questions) and web resources.
• Discussion 2: On-line discussion of the concept of camouflage in nature (and for human activities); what strategies are used to affect “observability”; can you find interesting examples on the web?
• Team Assignment (T-3): For your selected application; provide a summary table of the anticipated information sources and sensors (e.g., provide a list of sensors and/or sources; for each sensor or source identify the type of data (what is observed or reported), the data rate (e.g., reports, signals, images per second, minute or day), any information about the data format, and information about the characteristics of the sensor/source (e.g., reliability, capability of observing in various environments, etc.)
Data Fusion Tip of the Week
There’s no substitute for a good sensor that is appropriate to the situation or threat of interest: No amount of fusion of multiple poor sensors can substitute for an effective sensor!