![Page 1: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/1.jpg)
8/20/2014
1
Computer Vision
Mubarak [email protected]
Computer Vision The ability of computers to see.
Image Understanding Machine Vision Robot Vision Image Analysis Video Understanding
![Page 2: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/2.jpg)
8/20/2014
2
A picture is worth a thousand words.
A word is worth a thousand pictures.
A HUNT
![Page 3: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/3.jpg)
8/20/2014
3
Image 2-D array of numbers (intensity values,
gray levels) Gray levels 0 (black) to 255 (white) Color image is 3 2-D arrays of numbers
Red Green Blue
Resolution (number of rows and columns) 128X128 256X256 512X512 640X480
![Page 4: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/4.jpg)
8/20/2014
4
Image Formats
TIF PGM PBM GIF JPEG RAW
Video Sequence of frames 30 frames per second
Formats AVI MPEG Quick Time
![Page 5: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/5.jpg)
8/20/2014
5
Video Clip
Sequence of Images
![Page 6: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/6.jpg)
8/20/2014
6
Image Formation Light Source Camera (extrinsic and intrinsic
parameters) Scene (Surface reflectance, Surface
shape )
Perspective Projection (Pin Hole)
(X,Y,Z)
Lens
Image Plane
image Zy
fWorldpoint
![Page 7: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/7.jpg)
8/20/2014
7
Orthographic Projection(X,Y,Z)
Image Plane
image
y
Worldpoint
Shape from X Recover 3-D shape from 2-D image(s)
Stereo Motion Shading Texture Contours
![Page 8: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/8.jpg)
8/20/2014
8
Stereo
http://www.vision3d.com/stereo.html
![Page 9: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/9.jpg)
8/20/2014
9
Renault Stereo Pair
Depth Map
![Page 10: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/10.jpg)
8/20/2014
10
Shape from Shading
Lambertian Model
S=L, light source
I=S.N
![Page 11: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/11.jpg)
8/20/2014
11
Vase
(1, 0, 1) (-1, 1, 1) (-1,-1, 1)
Shape from Texture
![Page 12: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/12.jpg)
8/20/2014
12
Visual Motion
Shape from Motion: Moving Light Display
![Page 13: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/13.jpg)
8/20/2014
13
Shape from Motion
Photosynth
![Page 14: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/14.jpg)
8/20/2014
14
Sequence Raw Optical flow
27
Video Clip & Mosaic
![Page 15: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/15.jpg)
8/20/2014
15
Applications of Computer Vision Face Recognition Object Recognition Video Surveillance and Monitoring
Object detection, tracking and behavior analysis
Remote Sensing: UAVs Robotics Computer Graphics
Object Recognition
Finding People in imagesProblem 1: Given an image I Question: Does I contain an image of a
person?
![Page 16: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/16.jpg)
8/20/2014
16
“Yes” Instances
“No” Instances
![Page 17: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/17.jpg)
8/20/2014
17
Localize People (Human Detection)
Human Detection
![Page 18: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/18.jpg)
8/20/2014
18
Airplanes
Motor Cycles
![Page 19: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/19.jpg)
8/20/2014
19
Face Recognition
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
What are wild faces?
![Page 20: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/20.jpg)
8/20/2014
20
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
![Page 21: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/21.jpg)
8/20/2014
21
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
350 million photos uploaded daily350 million photos uploaded daily
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
100 hours of movies uploaded per hour100 hours of movies uploaded per hour
60,000 movies60,000 movies
![Page 22: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/22.jpg)
8/20/2014
22
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Why are wild faces important?
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Three Tasks of Face Recognition
Most Research Focus
![Page 23: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/23.jpg)
8/20/2014
23
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Three Tasks of Face Recognition
Most Research Focus
FERET200 people1400 faces
AR100 people1400 faces
Ext. Yale B38 people2400 faces
PubFig8383 people
>830 faces
YouTube Celebrities
47 people1910 faces
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Three Tasks of Face Recognition
Most Research Focus Pair-Matching Task
Same Not Same
PubFig200 people58,797 faces
LFW5,749 people13,233 faces
YouTube Faces
1,595 people3,425 faces
![Page 24: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/24.jpg)
8/20/2014
24
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Three Tasks of Face Recognition
Most Research Focus Pair-Matching TaskScenario
Most Realistic Scenario
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Open-Universe Face Identification
News Article: Label Important Figures
Social Network: Tag Facebook Friends
Barack Obama
Bob
Alice
![Page 25: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/25.jpg)
8/20/2014
25
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Open-Universe Face IdentificationFind Angelina Jolie and George Clooney
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Open-Universe Face IdentificationFind Angelina Jolie and George Clooney
![Page 26: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/26.jpg)
8/20/2014
26
Taming Wild Faces: Web‐Scale, Open‐Universe Face Recognition
CRCV | Center for Research in Computer VisionUn ive rs i t y o f Cent ra l F lo r ida
Open-Universe Face Identification
Gwenyth PaltrowRobert Downey Jr.
Face Recognition in Movie Trailers via Mean Sequence Sparse Representation‐based Classification
CVPR 2013
![Page 27: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/27.jpg)
8/20/2014
27
Recognition – Qualitative
Known:Paul RuddSteve Carell
Recognition – Qualitative
Known:Paul RuddSteve Carell
![Page 28: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/28.jpg)
8/20/2014
28
Recognition – Qualitative
Known:Owen WilsonReese WitherspoonPaul Rudd
Recognition – Qualitative
Known:Bruce WillisMorgan Freeman
![Page 29: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/29.jpg)
8/20/2014
29
Recognition – Qualitative
Known:Tom CruiseCameron Diaz
FACIAL EXPRESSIONS
RAISE EYE BROWS SMILE
![Page 30: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/30.jpg)
8/20/2014
30
Detecting Driver Alertness
Lipreading
![Page 31: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/31.jpg)
8/20/2014
31
Video Surveillance and Monitoring
Automated Surveillance System (Detection & Tracking)
Object detection Object tracking Object categorization and classification
Event or Activities Recognition
A. I Person Tracking
A. II Part Tracking
![Page 32: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/32.jpg)
8/20/2014
32
• Part of the WAS (wide area surveillance) project executed by the Homeland Security Advanced Research Project Agency (HSARPA)
• Current Sensor• 8 high-resolution cameras• provide a 100 mega-pixel, 360°field of view• frame rate: 5 frames per second
• Next Generation• 48 cameras with significantly higher resolution• smaller size
NONA: Project Overview
Current Sensor Next Generation
NONA Sysmtem—Airport Sequence 1
![Page 33: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/33.jpg)
8/20/2014
33
UAV: Unmanned Aerial Vehicle
UAVs: Unmanned Aerial Vehicles (Drones)
Global Hawk
Predator
Microdrone
![Page 34: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/34.jpg)
8/20/2014
34
KINGFISHER AEROSTAT BALLOON
8/20/2014 Computer Vision Lab, UCF 67
COCOA – System Flow
Ego Motion Compensation
Feature based + Gradient Based
Motion Detection
Accumulative Frame Differencing + Background
Modeling + Object Segmentation
Object Tracking
Kernel Tracking + Blob Tracking + Occlusion Handling
Telemetry*
Aerial V
ideo
Registered Images Motion Detection Tracks
CO
CO
A
Event Detection & Indexing
![Page 35: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/35.jpg)
8/20/2014
35
Registration Result ‐ I
Aerial Video - EO Mosaic
Alignment Mask
Registration Result ‐ II
Aerial Video - IR Mosaic
Alignment Mask
![Page 36: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/36.jpg)
8/20/2014
36
Detection Results
Tracking Results
![Page 37: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/37.jpg)
8/20/2014
37
Wide Area Surveillance
Wide Area Surveillance
![Page 38: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/38.jpg)
8/20/2014
38
Tracking Results
Robot Vision (Unmanned Ground Vehicle)
UGV
![Page 39: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/39.jpg)
8/20/2014
39
UGV
Human Action Recognition
![Page 40: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/40.jpg)
8/20/2014
40
Events, Actions, Activities,
• Action
• Event
• Movement
• Activity
• Interaction
• Verb
• ….
• 10 actions
• 9 actors per action
Bend Jack Wave2hands Wave1hand Jumpinplace
Sidestep Hop Walk Skip Run
Weizmann Action Dataset
![Page 41: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/41.jpg)
8/20/2014
41
KTH Data Set
• Six Categories, 25 actors, 4 instances, 600. clips
Boxing HandClapping HandWaving
Jogging Running Walking
UCF Sports Action Dataset
Bench Swing Dive Swing Run
Kick Lift Ride Golf Swing Skate
9actions,142videos.
![Page 42: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/42.jpg)
8/20/2014
42
IXMAS Multi‐view Data Set
• 13 action categories, 4 camera views, 10 actors, 3 instances.
View1 View2 View3 View4
CheckWatch
GetUp
UCF YouTube Action Dataset (UCF‐11)
Cycling Diving GolfSwinging Riding
Juggling BasketballShooting Swinging TennisSwinging
Volleyball Spiking TrampolineJumping WalkingDog
![Page 43: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/43.jpg)
8/20/2014
43
UCF-101
BaseballPitch BasketballShooting BikingBenchPress Billiards
Breaststroke CleanandJerk DrummingDiving Fencing
GolfSwing HighJump HorseRidingHorseRace HulaHoop
JavelinThrow JugglingBalls JumpRopeJumpingJack Kayaking
Lunges MilitaryParade NunchucksMixingBatter PizzaTossing
UCF50
![Page 44: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/44.jpg)
8/20/2014
44
PlayingGuitar PlayingPiano PlayingViolinPlayingTabla PoleVault
PommelHorse PullUps PushUpsPunch RockClimbingIndoor
RopeClimbing Rowing SkateBoardingSalsaSpins Skiing
Skijet SoccerJuggling TaiChiSwing TennisSwing
ThrowDiscus TrampolineJumping WalkingwithadogVolleyballSpiking Yo Yo
UCF50
Microsoft Kinect sensor
• Data Captured using Microsoft Kinect sensor
• Approximately 50,000 gesture samples
RGB Camera
IR Camera 1 IR Camera 2
![Page 45: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/45.jpg)
8/20/2014
45
Gesture Lexicons
Gestures from Depth camera
Gestures from RGB camera
Diving Signals Referee Signals Nurse Gesture Music Notes
Discovered Primitives
Representative Motion Primitives (out of 136) for different batches
Left arm moving down
Left arm moving up
Left arm moving to body
Left hand moving forward
Left arm waving up
Left arm moving away from body
Right arm moving laterally up
Right arm moving away from body
Left shoulder moving to right
Left arm moving to right
Left arm moving down
Left shoulder moving down
Right arm moving up
Left arm moving up
Left arm moving down
Right arm moving down
![Page 46: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/46.jpg)
8/20/2014
46
Test case 1: Torso motion adds noise (devel 01– 10 gestures)
Instances of Successful Recognition
Instances of Failed Recognition
Instances of Successful Recognition
Instances of Failed Recognition
Test Case 2: Improvisations (devel 06 – 9 gestures)
![Page 47: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/47.jpg)
8/20/2014
47
Test Case 3: Subtle differences (devel 09 – 10 gestures)
Instances of Successful Recognition
Instances of Failed Recognition
![Page 48: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/48.jpg)
8/20/2014
48
High Density Crowded Scenes
Political Rallies Religious Festivals Marathons High Density Moving Objects
Tracking in Crowds
Average chip size 14 x 22 pixels 492 Frames Selected 199 athletes for tracking Successfully tracked 143 athletes
![Page 49: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/49.jpg)
8/20/2014
49
Results
Experiment – 1
![Page 50: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/50.jpg)
8/20/2014
50
Experiment-3
Average chip size 14 x 17 pixels 453 Frames Selected 50 athletes for tracking
Experiment – 3
![Page 51: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/51.jpg)
8/20/2014
51
Behaviors in Crowded Scenes
Bottleneck Departure Lanes Arch/Rings Blockings
![Page 52: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/52.jpg)
8/20/2014
52
Proposed Method=640Ground truth=634
Proposed Method=1590Ground truth=1567
![Page 53: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/53.jpg)
8/20/2014
53
Proposed Method=1468Ground truth=1428
Proposed Method=673Ground truth=653
![Page 54: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/54.jpg)
8/20/2014
54
Ground truth=2322 Proposed Method=2203
Proposed Method=2496Ground truth=2319
![Page 55: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/55.jpg)
8/20/2014
55
Where Am I?
“Where Am I?” Problem:
Accurate Image Localization
Input Output
Mere Visual Information(Images) Location in Terms of λ (Lon.) and φ (Lat.)=40.4419, =-79.9986
![Page 56: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/56.jpg)
8/20/2014
56
Qualitative Image Localization Results:
68.3%: The best match found:
QueryMatch – Error: 6.4 m
QueryQuery
Match – Error: 7.5 m Match – Error: 62.7 m
QueryMatch – Error: 9.01 m
Qualitative Image Localization Results:
68.3%: The best match found:13.2%: A correct match found, but not the best one:
QueryMatch – Error: 244.7 m
Match – Error: 307.7 m
Match – Error: 157.3 m
Match – Error: 71.3 mQuery
Query
Query
![Page 57: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/57.jpg)
8/20/2014
57
GeospatialTrajectoryExtraction
Visual Business RecognitionACM Multimedia 2013
![Page 58: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/58.jpg)
8/20/2014
58
Computer Vision for Computer Graphics
Video Completion
![Page 59: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/59.jpg)
8/20/2014
59
Layer Based Video Composition
Results of Doll
![Page 60: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/60.jpg)
8/20/2014
60
Results of Mom-Daughter
Multimedia: Segmentation of Moving-Sounding Objects
120
Accepted in IEEE Transactions on Multimedia
![Page 61: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/61.jpg)
8/20/2014
61
Computer Vision Text Books
![Page 62: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/62.jpg)
8/20/2014
62
Computer Vision Researchers
![Page 63: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/63.jpg)
8/20/2014
63
Late Azriel Rosenfeld (Mayrland)
Berthold Horn (MIT)
![Page 64: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/64.jpg)
8/20/2014
64
Thomas Huang (UIUC)
Ruzena Bajcsy
![Page 65: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/65.jpg)
8/20/2014
65
Jake Aggarwal (UT Austin)
Chris Brown (Rochester)
![Page 66: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/66.jpg)
8/20/2014
66
Bob Haralick (CUNY)
Olivier Faugeras (INRIA, France)
![Page 67: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/67.jpg)
8/20/2014
67
Takeo Kanade (CMU)
Sandy Pentland (MIT)
![Page 68: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/68.jpg)
8/20/2014
68
Shree Nayar (Columbia)
John Canny (Berkeley)
![Page 69: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/69.jpg)
8/20/2014
69
Demetri Terzopoulos (UCLA)
Ramesh Jain (UCI)
![Page 70: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/70.jpg)
8/20/2014
70
Jitendra Malik (Berkeley)
Andrew Zisserman (Oxford)
![Page 71: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/71.jpg)
8/20/2014
71
Andrew Blake (Microsoft)
Rama Chellappa
![Page 72: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/72.jpg)
8/20/2014
72
Anil Jain
David Forsyth
![Page 73: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/73.jpg)
8/20/2014
73
Rick Szeliski (Microsof)
David Lowe
![Page 74: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/74.jpg)
8/20/2014
74
Cordelia Schmidt
Computer Vision Journals
![Page 75: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/75.jpg)
8/20/2014
75
![Page 76: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/76.jpg)
8/20/2014
76
![Page 77: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/77.jpg)
8/20/2014
77
Computer Vision Conferences
![Page 78: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/78.jpg)
8/20/2014
78
IEEE Conference on Computer Vision and Pattern Recognition
(CVPR)• June 24-27, 2014
• Greater Columbus Convention Center in Columbus, Ohio.
![Page 79: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/79.jpg)
8/20/2014
79
![Page 80: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/80.jpg)
8/20/2014
80
International Conference on Computer Vision (ICCV)
![Page 81: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/81.jpg)
8/20/2014
81
European Conference on Computer Vision (ECCV)
International Conference on Pattern Recognition (ICPR)
![Page 82: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/82.jpg)
8/20/2014
82
Asian Conference on Computer Vision (ACCV)
International Conference on Image Processing
![Page 83: Computer Visioncrcv.ucf.edu/courses/CAP5415/Fall2014/CVIntroductionAugust2014.pdf · Computer Vision Mubarak Shah shah@crcv.ucf.edu Computer Vision ... Playing Guitar Playing Piano](https://reader031.vdocuments.mx/reader031/viewer/2022022522/5b3209807f8b9a2c0b8c2b05/html5/thumbnails/83.jpg)
8/20/2014
83