www.kingston.ac.uk/dirc/ vigilant real-time storage and intelligent retrieval of visual surveillance...

www.kingston.ac.uk/dirc/

Vigilant Real-time storage and

intelligent retrieval of visual surveillance data

Dr Graeme A. Jones


Vigilant aims

• to design a database that provides real-time efficient storage of events occurring within a monitored scene.

• to enable untrained security operators to generate human-centric queries for video data i.e. queries based on content.


System constraints

• Handle video stream in real-time

• Compress the terabyte of digital video generated per camera per day onto an swappable one gigabyte disk

• Maximum of one high spec PC per camera

• Intuitive button-press query builder


System architecture

dB

Event detectionand tracking


OfflineAnnotation

ColourColour

TrajectoryTrajectory

ClassificationClassification

Fuzzy SearchFuzzy Search


System architecture

dB




Fast Robust Event Detection (and tracking) Algorithm

• Model-based approach exploiting expected projected object size, with shadow/reflection

iiB

iiW

AiiH

)(

)(min

)(

i

H

B

W


Sum

Gate

Sum

Z-1

Z-1

Gate

Temporal updating of reference images with soft gating

Blob detection from pixel comparison with mean and variance image

CurrentFrame

EventMask

DifferenceFrame

Threshold

SubtractReference

Map

VarianceMap

FREDA- Low-level


FREDA - High LevelRegion detection from

connected components

Unsupported regions generate new objects

EventMask

CC`

RegionList

`Objects`New

Objects

HypothesizeObjects

`Region

List

+

CurrentObjects

ValidateObjects

Z-1

Gate

Current objects validated from supporting regions


Example Results

Image Acquisition 11%Blob Detection 32%Updating Background 23%Connected Component 20%Object Validation 12%

100%Total Time (secs) 0.127Frame Rate 8 Hz


Information Balance Sheet

Input data

• [800x600] x (1+0.5+0.5) x 25 x (60x60x24)

2Tbytes/day

Output knowledge

• Periodic background + DCT updates

• Subimage sequences (pixels and contour) per temporal event


The Intelligent Camera

• Boundary between PC and intelligent camera depends on issues of frame rate, bandwidth and computational resources

Pix

el C

ompa

rsio

n

Con

nect

ed

Com

pone

nts

Hyp

othe

sis

Gen

erat

ion

Obj

ect V

alid

atio

n

Cha

ract

eris

atio

n


System architecture

dB



OfflineAnnotation

ColourColour




Object Classification• Object event may be classified into Person,

Vehicle, Large Vehicle classes based on history of depth-compensated dimensions and speed.


Object Classification


Colour Annotation

• Dominant colour(s) of object extracted as modes from colour histogram generated from pixels of temporal event.

Munsell Space. Semantic classification in HSV colour space

HUE

Val

ue


3D Trajectory

• Ground plane calibration (learnt) enables 3D speed to be computed, and hence velocity behaviours derived e.g. car maneouvring, person running, etc.

• Trajectory commentary derived from areas of interest previously assigned labels by operator e.g. gate, bikeshed, disabled parking.


Fasset Road

Fasset Lane

Zone 1

Zone 2

Zone 3

West Carpark Approach

REGIONS

GATE

CARPARK

GATE

08:34:56 Vehicle 3434 enters gate08:35:03 Vehicle 3434 enters F.Lane08:35:12 Vehicle 3434 enters carpark08:35:31 Vehicle 3434 enters zone308:36:05 Vehicle 3434 stops in zone3

15:55:23 Vehicle 3434 leaves zone3


Behaviour analysis

• Hidden Markov Models based on states derived from clusters of positions along training sets of trajectories used to classify object and its behaviour

•Car Entering•Car Leaving•Person Entering•Person Leaving

S1 S2 S3

0.8 0.7

0.05

0.15 0.25


System architecture

dB



OfflineAnnotation

ColourColour



Fuzzy SearchFuzzy Search


Conclusions

• Its easy to derive atomic units of useful user-oriented knowledge

• Requirement for human-oriented query specification tools employing fuzzy matching

• Plug ‘n play characteristics e.g. camera calibration

• Distribution of computer intelligence


Contact Details

Email: [email protected]

Web: www.kingston.ac.uk/dirc/

mailto:[email protected]

www.kingston.ac.uk/dirc/ vigilant real-time storage and intelligent retrieval of visual surveillance...

Documents

tracking slide

temporal event slide

level slide

tracking event detection

object classification

robust event detection

pixels of temporal event

h b w slide