![Page 1: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/1.jpg)
{
Video Analytics for
Security and
Compliance
Vishal Kaushal
www.vishalkaushal.in
Advisor: Prof. Ganesh Ramakrishnan
![Page 2: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/2.jpg)
![Page 3: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/3.jpg)
The Flip Side
Time Consuming
Heavy on storage
![Page 4: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/4.jpg)
Video Summarization & Its Flavours
Watch
hours in
minutes
Query
Events of
Interest
Generate
Useful
Statistics
![Page 5: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/5.jpg)
Video Summarization & Its Flavours
Watch
hours in
minutes
Query
Events of
Interest
Generate
Useful
Statistics
![Page 6: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/6.jpg)
Video Summarization & Its Flavours
Watch
hours in
minutes
Query
Events of
Interest
Generate
Useful
Statistics
“Show me a digest of whatever
happened today”
“Show me all people wearing yellow
shirt”
“Alert me as soon as you see a man
with covered face”
“Number of people in the video”
“Motion graph”
“Heat map”
![Page 7: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/7.jpg)
A Multi-Faceted Video Summarization
System: Tradeoff Between Diversity,
Representation, Coverage and Importance
(WACV 2019)
• Extractive, Query-based, Entity or Concept based
• In a surveillance video, diversity is more important, while in
a movie, coverage and representation becomes more
important
![Page 8: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/8.jpg)
Learning From Less Data: A Unified Data
Subset Selection and Active Learning
Framework for Computer Vision (WACV
2019)
1. Supervised Data Selection for Quick Training/Inference
2. Supervised Data Selection for Quick Hyper-parameter
tuning
3. Unsupervised Data Selection for Labeling from Video
Data
4. Diversified Active Learning
![Page 9: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/9.jpg)
A Framework towards Domain Specific Video
Summarization (WACV 2019)
Surveillance Video Birthday Video Soccer Video
• Joint problem of learning domain specific importance of segments as well as
the desired summary characteristic for that domain
• Ratings more effective as opposed to binary inclusion/exclusion information
• A novel evaluation measure, more naturally suited in assessing the quality of
video summary for the task at hand than F1 like measures
• A gold standard dataset for furthering research in domain specific video
summarization
![Page 10: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/10.jpg)
Summarization via
Submodular Functions
Efficient greedy heuristic for optimization [G. Nemhauser, L. Wolsey, and M. Fisher. An
analysis of approximations for maximizing submodular set functions, 1978]
Often guaranteed to yield a near optimal solution
![Page 11: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/11.jpg)
Learning Framework
![Page 12: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/12.jpg)
Watch hours in minutes
![Page 13: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/13.jpg)
Smart Search
![Page 14: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/14.jpg)
Real-time Alerts
![Page 15: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/15.jpg)
Compliance
![Page 16: Video Analytics for Security and Compliancevkaushal/talk/risc2019/risc2019.pdf · Surveillance Video Birthday Video Soccer Video • Joint problem of learning domain specific importance](https://reader033.vdocuments.mx/reader033/viewer/2022060313/5f0b45017e708231d42faec0/html5/thumbnails/16.jpg)
Main challenge in video summarization community
– benchmarking dataset
Important practical event of interest for video
summarization – masked face
Real-time resource constrained masked face detection
Human interaction and gamification for making
video summarization a practical reality
Pre-training: For creating labelled data
During training: Hyper parameter tuning
Post-training: Model customization/localization by
human feddback to improve accuracy
What next?