automated intruder tracking using particle filtering and a network of binary motion sensors

Automated Intruder Tracking using Particle Filtering and a Network of

Binary Motion SensorsJeremy SchiffJeremy SchiffEECS DepartmentEECS DepartmentUniversity of California, BerkeleyUniversity of California, Berkeley

Ken GoldbergKen GoldbergIEOR and EECS DepartmentsIEOR and EECS DepartmentsUniversity of California, BerkeleyUniversity of California, Berkeley

http://www.cs.berkeley.edu/~jschiffSupported by NSF Grants: 0424422/0535218

http://www.cs.berkeley.edu/~jschiff

OutlineOutline IntroductionIntroduction Related WorkRelated Work Problem FormulationProblem Formulation

Setup and AssumptionsSetup and Assumptions Particle FilteringParticle Filtering

ResultsResults SimulationSimulation ExperimentalExperimental

Conclusion/Future WorkConclusion/Future Work

MotivationMotivation New class of technologies due New class of technologies due

to 9/11to 9/11 Automated SecurityAutomated Security Wireless Sensor NetworksWireless Sensor Networks

X10 PIR sensors - $25 X10 PIR sensors - $25 Robotic WebcamsRobotic Webcams

Pan, Tilt, ZoomPan, Tilt, Zoom 500 Mpixels/Steradian500 Mpixels/Steradian

Increased computer Increased computer processing speedsprocessing speeds Enables Realtime ApplicationsEnables Realtime Applications

Goal and ApproachGoal and Approach Wish to secure an Wish to secure an

environmentenvironment Low Cost Binary Sensors Low Cost Binary Sensors

X10 ~ $25X10 ~ $25 Optical BeamOptical Beam Floor PadFloor Pad Manufactured in ChinaManufactured in China

Noisy triggering patternNoisy triggering pattern RefractionRefraction

Use sensor triggering Use sensor triggering patterns to accurately patterns to accurately localize an intruderlocalize an intruder

IntuitionIntuition Utilize Sensor Overlap InformationUtilize Sensor Overlap Information



ExperimentsExperiments SimulationSimulation Real-worldReal-world


Related WorkRelated Work Pursuer/Evader GamesPursuer/Evader Games

Using line-of sight Using line-of sight optical sensorsoptical sensors

[Isler, Kannan, Khanna [Isler, Kannan, Khanna 2004]2004]

Tracking Multiple Tracking Multiple IntrudersIntruders

[Oh, Sastry 2005][Oh, Sastry 2005] Tracking Worn DevicesTracking Worn Devices

Track Infrared BeaconTrack Infrared Beacon [Shen et al. 2004][Shen et al. 2004]

Dynamic Shipment Dynamic Shipment Planning using RFIDsPlanning using RFIDs

[Kim et al. 2005][Kim et al. 2005]

Related Work IIRelated Work II Video Tracking SystemsVideo Tracking Systems

[Micilotta and Bowden 2004][Micilotta and Bowden 2004] Multiple Classes of SensorsMultiple Classes of Sensors

Multiple exclusive modes Multiple exclusive modes [Cochran, Sinno, Clausen 1999][Cochran, Sinno, Clausen 1999]

Fuse data of multiple sensor Fuse data of multiple sensor typestypes

[Jeffery et al. 2005][Jeffery et al. 2005] Automated Camera ControlAutomated Camera Control

[Song et al. 2005][Song et al. 2005]

Physical Devices

Virtual Devices

Related Work IIIRelated Work III Probabilistic Tracking Probabilistic Tracking

ApproachesApproaches Kalman Filtering Kalman Filtering

[Kalman 1960][Kalman 1960] Extended Kalman FilteringExtended Kalman Filtering

[Lefebvre, Bruyninckx, De [Lefebvre, Bruyninckx, De Schutter 2004]Schutter 2004]

Particle FilteringParticle Filtering Book: [Thrun, Burgard, Fox Book: [Thrun, Burgard, Fox

2005]2005] [Arulampalam et al. 2002][Arulampalam et al. 2002]

Related Work IVRelated Work IV Multiple humans controlling a cameraMultiple humans controlling a camera

[Song and Goldberg 2003][Song and Goldberg 2003] [Song, Goldberg and Pashkevich 2003][Song, Goldberg and Pashkevich 2003]

Panorama GenerationPanorama Generation [Song et al. 2005][Song et al. 2005]

Art Gallery ProblemArt Gallery Problem [Shermer 1990][Shermer 1990] [Urrutia 2000][Urrutia 2000]

Setup and AssumptionsSetup and Assumptions Room GeometryRoom Geometry

List of nodes and List of nodes and edgesedges

Discretize spaceDiscretize space Discretize timeDiscretize time

Setup and Assumptions Setup and Assumptions IIII

Intruder occupied Intruder occupied world-space cell j world-space cell j in iteration in iteration

Sensor i triggered Sensor i triggered during iterationduring iteration

Sensor i Sensor i experienced experienced refraction period refraction period in in iterationiteration

Setup and Assumptions Setup and Assumptions IIIIII

Three Conditional Three Conditional DistributionsDistributions Trigger while experiencing Trigger while experiencing

refractionrefraction

Trigger from intruderTrigger from intruder

Trigger from no intruderTrigger from no intruder

OutputOutput Estimated Estimated

intruder locationintruder location Objective:Objective:

Minimize error Minimize error between ground between ground truth and truth and estimation.estimation.

CharacterizationCharacterization Per sensor typePer sensor type Grid over sensor Grid over sensor

spacespace Determine Determine

Refraction period Refraction period False Negative RateFalse Negative Rate False Positive RateFalse Positive Rate

DeploymentDeployment Convert to Convert to

world-spaceworld-space Overlay Overlay

grid grid TransformeTransforme

d point to d point to CellsCells

Deployment IIDeployment II Determine potential non-zero Determine potential non-zero

characterization cells via convex characterization cells via convex hullhull

Inverse Distance Weighting Inverse Distance Weighting Interpolation according to distanceInterpolation according to distance Determines values for cells without Determines values for cells without

readings inside convex hullreadings inside convex hull

Particle filtersParticle filters Non-ParametricNon-Parametric

Sample Based Method (Particles)Sample Based Method (Particles) Particle Density ~ Likelihood Particle Density ~ Likelihood Tracking requires three distributionsTracking requires three distributions

Initialization Distribution Initialization Distribution

Transition Model (Intruder Model)Transition Model (Intruder Model)

Observation ModelObservation Model

Determines Determines

ExampleExample

Intruder ModelIntruder Model StateState

Position, Orientation, Speed, Position, Orientation, Speed, and Refracting Sensorsand Refracting Sensors

Euler Integration for positionEuler Integration for position Gaussian Random Walk for Gaussian Random Walk for

new speed and orientationnew speed and orientation Orientation change inversely Orientation change inversely

proportional to speedproportional to speed Deterministic refraction Deterministic refraction

periodsperiods Rejection Sampling to Rejection Sampling to

enforce room geometryenforce room geometry

Intruder Model IIIntruder Model II Time between iterations:Time between iterations: Empirically determined constants:Empirically determined constants:

Intruder Model - Intruder Model - ExampleExample

Example state at iteration 0


Accepted state for iteration 1


Rejected state for iteration 2

Sensor ModelSensor Model Evidence is vector of which sensors are Evidence is vector of which sensors are

triggeringtriggering Triggering of sensors independent Triggering of sensors independent

given intruder state impliesgiven intruder state implies

If sensor refractingIf sensor refracting

OtherwiseOtherwise

Simulation SetupSimulation Setup 22 Optical Beams22 Optical Beams

Perfect Perfect Optimal Optimal

PerformancePerformance

14 Triangular 14 Triangular Motion SensorMotion Sensor Perfect & ImperfectPerfect & Imperfect

Simulation ResultsSimulation Results Example PathExample Path Ground Truth Ground Truth

Red CirclesRed Circles Estimations Estimations

Grey CirclesGrey Circles

Simulation Results IISimulation Results II Baseline EstimateBaseline Estimate

Perfect Optical-Beam SensorsPerfect Optical-Beam Sensors

P(E)

Error E

Error E

P(E

)

Perfect Triangular Motion SensorsPerfect Triangular Motion Sensors

Imperfect Triangular Motion SensorsImperfect Triangular Motion Sensors

Simulation Results IIISimulation Results IIIP(

E)

Error E

Error E

P(E)

Error over Time – 4 Sec. Refraction, Error over Time – 4 Sec. Refraction, Imperfect SensorsImperfect Sensors

Density - 8 Sec. Refraction, Imperfect Density - 8 Sec. Refraction, Imperfect SensorsSensors

Simulation Results IVSimulation Results IVEr

ror

E

Time (Seconds)

Error E

P(E)

In-Lab ResultsIn-Lab Results 8 Passive Infrared Sensors8 Passive Infrared Sensors

X10X10 8 second refraction time8 second refraction time

Room 8x6 metersRoom 8x6 meters .3 m /Cell dimension.3 m /Cell dimension Sampled every 2 secondsSampled every 2 seconds 1000 Particles1000 Particles

In-Lab Results IIIn-Lab Results II



ResultsResults SimulationSimulation ExperimentalExperimental


ConclusionsConclusions Real-time Tracking SystemReal-time Tracking System Binary Sensors with Refraction PeriodBinary Sensors with Refraction Period Particle Filtering for Sensor FusionParticle Filtering for Sensor Fusion

Conditional Probability ModelsConditional Probability Models Models Models

Intruder Velocity Intruder Velocity Room GeometryRoom Geometry Sensor CharacterizationSensor Characterization

Future WorkFuture Work Effects of varying different componentsEffects of varying different components

Number ParticlesNumber Particles Types of sensorsTypes of sensors Spatial arrangements of sensorsSpatial arrangements of sensors

Multiple intrudersMultiple intruders DecentralizeDecentralize Vision ProcessingVision Processing Other applicationsOther applications

Warehouse TrackingWarehouse Tracking

Thank YouThank You Jeremy Schiff: Jeremy Schiff:

[email protected]@cs.berkeley.edu Ken Goldberg: Ken Goldberg:

[email protected]@ieor.berkeley.edu URL: URL: www.cs.berkeley.edu/~jschiffwww.cs.berkeley.edu/~jschiff

mailto:[email protected]

mailto:[email protected]

http://www.cs.berkeley.edu/~jschiff/c

automated intruder tracking using particle filtering and a network of binary motion sensors

Documents

patternrefractionuse

iteration sensor i

automated intruder tracking

likelihood tracking

tracking multiple intrudersoh

intruder trigger

refraction trigger

experienced refraction