stanford hci group / cs376 vision-based interaction€¦ · 640x480 sony firewire camera for video;...
TRANSCRIPT
![Page 1: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/1.jpg)
1
stanford hci group / cs376
http://cs376.stanford.eduScott Klemmer · 28 November 2006
Vision-Based Interaction
![Page 2: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/2.jpg)
2
2
cs547: Blake Ross and Asa Dotzler
Mozilla: Creating simple software in a geek-driven culture
![Page 3: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/3.jpg)
3
3
The first vision-based interface
Myron Krueger used computer vision to create Responsive Environments (1970s)“Reaction is the Medium”http://www.artmuseum.net/w2vr/timeline/videoplace_video.html
![Page 4: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/4.jpg)
4
4
How it works
Video and background are separated in analog using chroma key techniques(think broadcast news)The first and last points of each raster are stored in the computer, and represent the person’s outline
![Page 5: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/5.jpg)
5
5
Vision-based UIs: “Verbs”
Detecting and Tracking elements of a certain type in a sceneCapturing contents of detected objectsRecognizing individual members in an object class
![Page 6: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/6.jpg)
6
6
Vision-based UIs: “Verbs”
Detecting and Tracking elements of a certain type in a scene
![Page 7: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/7.jpg)
7
7
Vision-based UIs: “Verbs”
Capturing contents of detected objects
![Page 8: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/8.jpg)
8
8
Vision-based UIs: “Verbs”
Recognizing individual members in a class
![Page 9: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/9.jpg)
9
9
Vision-based UIs: “Nouns”
People (one or multiple)BodiesFacesHands
DocumentsObjects
![Page 10: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/10.jpg)
10
10
Vision-based UIs: “Nouns”
People (one or multiple)BodiesFacesHands
DocumentsObjects
![Page 11: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/11.jpg)
11
11
Vision-based UIs: “Nouns”
People (one or multiple)BodiesFaces Hands
DocumentsObjects
![Page 12: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/12.jpg)
12
12
Background SubtractionI N F R A S T R U C T U R E
Two Cameras
640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes
3 megapixel Epson still USB camera; Treat as ~1fps ‘video’ camera; Mount in front, use for ink capture
![Page 13: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/13.jpg)
13
13
Image Moments (of Inertia)
0th moment is mass(total number of pixels)
![Page 14: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/14.jpg)
14
14
Image Moments (of Inertia)
1st moment is center
![Page 15: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/15.jpg)
15
15
Image Moments (of Inertia)
2nd moment is orientation
![Page 16: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/16.jpg)
16
16
Tools for Vision appsIntel’s OpenCV
C API to highly optimized image processing functions (threshold, dilate, optical flow, …)http://www.intel.com/research/mrl/research/opencv
Fast to run! Slow to developGreat for vision folks; too low-level for app folks
Papier-MâchéJava API (and to some extent visual UI) for vision (and other physical input)http://guir.berkeley.edu/papier-macheFast to develop! Slow to runGreat for app folks; ~5 fps can sometimes be too slow
![Page 17: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/17.jpg)
17
17
Good Vision Books
Computer Vision: A Modern ApproachDavid Forsyth and Jean Ponce (2003)Fantastic book; but goal is more theoretical understanding than practical application
Robot VisionBerthold Horn (1987)More focused on apps and interactive algorithmsSomewhat out of date
![Page 18: stanford hci group / cs376 Vision-Based Interaction€¦ · 640x480 Sony Firewire camera for video; Mounted behind board; eliminates occlusion; Used for Tracking Notes 3 megapixel](https://reader033.vdocuments.mx/reader033/viewer/2022042208/5eab40a3e2289005ee773943/html5/thumbnails/18.jpg)
18
18
Next Time… Software Tools
Past, Present, and Future of User Interface Software Tools, Brad Myers, Scott E. Hudson, Randy Pausch
Natural Programming Languages and Environments, Brad A. Myers, John F. Pane, Andy Ko