smart video surveillance and privacy - crisp final conference
TRANSCRIPT
![Page 1: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/1.jpg)
© Fraunhofer
Smart Video Surveillance and Privacy
Dr.-Ing. Erik Krempel,
Fraunhofer IOSB, Karlsruhe, Germany
![Page 2: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/2.jpg)
© Fraunhofer IOSB 2
What can we do with video analytics…
SMART VIDEO SURVEILLANCE
![Page 3: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/3.jpg)
© Fraunhofer IOSB 3
From Visual to Symbolic Video Surveillance
• The classic approach of video surveillance is visual videomonitoring by security staff.
• Today, video analytics are more and more in use as assistance systems. Mainly to attract and sustain attention of the staff to possible points of interst.
• Current research is going towards useage of advanced video analytics for symbolic situation awareness and „privacy-aware management by exception“
Classic Approach Intelligent Video Analytics
Privacy-aware Situation
Awareness Tools
![Page 4: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/4.jpg)
© Fraunhofer IOSB 4
Video Analytics for Scene Understanding
Object Tracking
Activity RecognitionSemantic SceneUnderstanding
Scene Context Recognition
Object DetectionImage Enhancement
![Page 5: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/5.jpg)
© Fraunhofer IOSB 5
Current Research: Advanced Objekt Detection, Tracking and Re-Identification
• Multi-camera multi-object detection and tracking of persons and vehicles
• indoor / outdoor applications with complex illuminationconditions
• self-calibrating cameras (automated geometric and colorcalibration)
• New approaches on dynamic learning of objects‘ unique features
![Page 6: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/6.jpg)
© Fraunhofer IOSB 6
Towards Activity Recognition for (Global) Scene Understanding
• Classification of activities in videos.
![Page 7: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/7.jpg)
© Fraunhofer IOSB 7
Person Retrieval
• Component that is looking for similar persons
in archive in a specific time interval
• Features of the person of interest (color features and texture features) are compared to already computed features of other tracks
• System generates a listof tracks, sorted basedon their similarity tothe person of interest
• Operator can verify the results visually
![Page 8: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/8.jpg)
© Fraunhofer IOSB 8
Why do we need to certify smart video surveillance systems
CERTIFICATION
![Page 9: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/9.jpg)
© Fraunhofer IOSB 9
Security
What is the error rate of video processing algorithms?
False positive rates
False negative rates
Is the system robust against environment conditions?
Rain on the camera
Light conditions
Is the system safe against attacks?
The systems keeps secrets for itself (Confidentiality)
The systems stays operational in the presence of an attacker (Availability)
The systems detects the same results in the presence of an attacker (Integrity)
![Page 10: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/10.jpg)
© Fraunhofer IOSB 10
Trust / Transparency
![Page 11: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/11.jpg)
© Fraunhofer IOSB 11
Trust
How can you achieve real transparency in such complex systems?
Is there a human preventing the system from making wrong decisions?
Does the systems prevent discrimination?
© Axis Communications
![Page 12: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/12.jpg)
© Fraunhofer IOSB 12
Efficiency dimension
Is the system able to reduce crime / increase security?
Is the system easy to use?
Can you extend the system / use components of another manufacturer?
![Page 13: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/13.jpg)
© Fraunhofer IOSB 13
Freedom Infringement
Does the system prevent discrimination?
Are video processing algorithms restricted to legal applications?
Does the system prevent misuse by the operator?
Security of the data processing…
Secure against data theft…
Purpose limitation…
Errors by the systems…
![Page 14: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/14.jpg)
© Fraunhofer IOSB 14
RESEARCH WORKPrivacy in smart video surveillance
![Page 15: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/15.jpg)
© Fraunhofer IOSB 15
Situation-dependent video surveillance
Alexander Roßnagel, Monika Desoi, and Gerrit Hornung (2011)
![Page 16: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/16.jpg)
© Fraunhofer IOSB 16
Idea: Privacy-aware Surveillance Workflows
Default Mode: Optimized for privacy Assessment Mode:Event-specific, privacy preserving
Eventdetected
Eventconfirmed
Eventresolved
Investigation Mode:Event specific functions unlocked
![Page 17: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/17.jpg)
© Fraunhofer IOSB 17
Default Mode: Optimized for Privacy
Computer vision algorithms in background
Abstract representation of observed environment visualized
No access to video data
No access to archived data
Exposes as little information about observed persons as possible
![Page 18: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/18.jpg)
© Fraunhofer IOSB 18
Assessment Mode: Event-specific, Privacy Preserving
Eventdetected
Assessment view according to event type
Anonymized live video data released
Possibly anonymized access to a limited buffer of recorded (video) data
Only selected cameras
![Page 19: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/19.jpg)
© Fraunhofer IOSB 19
Investigation Mode: Event-specific Functions Unlocked
Eventconfirmed
Eventresolved
Allows additional privacy intrusions for investigation purposes
I.e., retrieving the person who dropped a piece of luggage
Restrict additional analyses to persons related to the event under investigation
![Page 20: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/20.jpg)
© Fraunhofer IOSB 20
IMPLEMENTATIONHow to build systems with this concept…
![Page 21: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/21.jpg)
© Fraunhofer IOSB 21
Security Use-Case
![Page 22: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/22.jpg)
© Fraunhofer IOSB 22
Safety Use-Case: Prototype NurseEye
![Page 23: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/23.jpg)
© Fraunhofer IOSB 23
Operator interaction
Default mode:
No Access
Assessment mode:
Access to anonymized video
Investigation mode:
Full access to video to help in emergency handling
![Page 24: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/24.jpg)
© Fraunhofer IOSB 24
Transparency
All cameras come with displays showing the current mode and data usage
Displays become monitor for chat in the investigation mode
Default mode Assessment mode Investigation mode
![Page 25: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/25.jpg)
© Fraunhofer IOSB 25
Video NurseEye
![Page 26: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/26.jpg)
© Fraunhofer IOSB 26
TOOLS
![Page 27: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/27.jpg)
© Fraunhofer IOSB 27
User Study on Anonymization Techniques: Introduction
Which obfuscation technique should be used?
Regarding privacy (identity leakage), utility and perceptual video quality
ROI Anonymized person
Blurring Silhouette Edge detection PixelizationBlurring(gray scale)
![Page 28: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/28.jpg)
© Fraunhofer IOSB 28
Results: Subjective assessments
1 = silhouette; 2 = pixelization; 3,5 = edge detection; 4 gray scale blurring; 6-8 color blurring
Error bars represent one standarddeviation of the data sample
Perceived privacy protection and perceived image quality
![Page 29: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/29.jpg)
© Fraunhofer IOSB 29
Video anonymisation
Edge detectionPixelization
Gaussian blurring Silhouette
![Page 30: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/30.jpg)
© Fraunhofer IOSB 30
Adaptive „Privacy-Masking“ for Pan/Tilt/Zoom Camera
• Advanced Privacy Masking for semi-stationary PTZ-cameras
• Given position of camera over ground (or an elevation model of the site) optical distance toobjects in the scene is estimated
• Given a parameter for „privacy-preserving resolution“ video is pixelized or blurredinhomoge-nously depending on distance to object.
• Highly senstive areas (e.g. buildings / windows) can be blacked out of the stream bydynamic adaptable polygons (depending on pan/tilt/zoom settings)
![Page 31: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/31.jpg)
© Fraunhofer IOSB 31
Information Flow Tracking in NEST
Video &Observation
archive
PEP
HMIGeoViewer
Observations
PEP
Requests
HMIStreamViewer
PEP
Imageexploitationalgorithm
Imageexploitationalgorithm
Imageexploitationalgorithm
PEPIMG
PolicyDecision
Point
PolicyInformation
Point
Policy
Observations
Control
PEP
Acce
ss Co
ntr
ol
OOWM
Association
PEP HLS-Alarms
PEP Classification
PEP Fusion
![Page 32: Smart Video Surveillance and Privacy - CRISP Final Conference](https://reader035.vdocuments.mx/reader035/viewer/2022062503/58e5fb2f1a28ab09478b5805/html5/thumbnails/32.jpg)
© Fraunhofer IOSB 32
Discussion
“Data is not an asset, it’s a liability”-Marko Karppinen-
Contact Information:Erik Krempel
Fraunhofer IOSBFraunhoferstr. 176131 Karlsruhe, Germany
[email protected]+49-721-6091-292