components of a computer vision system
DESCRIPTION
Scene Interpretation. Components of a computer vision system. Camera. Lighting. Computer. Scene. Srinivasa Narasimhan’s slide. Computer vision vs Human Vision. What we see. What a computer sees. Srinivasa Narasimhan’s slide. A little story about Computer Vision. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/1.jpg)
Components of a computer vision system
Lighting
Scene
Camera
Computer
Scene Interpretation
Srinivasa Narasimhan’s slide
![Page 2: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/2.jpg)
Computer vision vs Human Vision
What we see What a computer sees
Srinivasa Narasimhan’s slide
![Page 3: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/3.jpg)
A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
![Page 4: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/4.jpg)
A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Founder, MIT AI project
![Page 5: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/5.jpg)
A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Founder, MIT AI project
Professor of Electrical Engineering, MIT
![Page 6: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/6.jpg)
A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Image Understanding
![Page 7: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/7.jpg)
A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Image Understanding
Image Sensing
![Page 8: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/8.jpg)
Continue on CAPTCHA
CAPTCHA stands for "Completely Automated Public Turing test to Tell Computers and Humans Apart".
Picture of a CAPTCHA in use at Yahoo.
http://www.cs.sfu.ca/~mori/research/gimpy/
![Page 9: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/9.jpg)
Breaking a Visual CAPTCHA
http://www.cs.sfu.ca/~mori/research/gimpy/
On EZ-Gimpy: a success rate of 176/191=92%!
Other exampleshttp://www.cs.sfu.ca/~mori/research/gimpy/ez/
![Page 10: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/10.jpg)
Breaking a Visual CAPTCHA
http://www.cs.sfu.ca/~mori/research/gimpy/
On more difficult Gimpy: a success rate of 33%!
Other exampleshttp://www.cs.sfu.ca/~mori/research/gimpy/hard/
![Page 11: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/11.jpg)
Breaking a Visual CAPTCHA
YAHOO’s current CAPTCHA format
http://en.wikipedia.org/wiki/CAPTCHA
![Page 12: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/12.jpg)
Face Detection and Recognition
Applications: Security, Law Enforcement, Surveillance
![Page 13: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/13.jpg)
Face Detection and Recognition
Smart cameras: auto focus, red eye removal, auto color correction
![Page 14: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/14.jpg)
Face Detection and Tracking
![Page 15: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/15.jpg)
Face Detection and Tracking
![Page 16: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/16.jpg)
Face Detection and Tracking
Lexus LS600 Driver Monitor System
![Page 17: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/17.jpg)
General Motion Tracking
Hidden Dragon Crouching Tiger
![Page 18: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/18.jpg)
General Motion Tracking
Application
Andy Serkis, Gollum, Lord of the Rings
![Page 19: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/19.jpg)
Segmentation
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/
![Page 20: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/20.jpg)
Segmentation using Graph Cuts
Application
Medical Image Processing
![Page 21: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/21.jpg)
Segmentation using Graph Cuts
Input Matting: Soft Segmentation
Composition
![Page 22: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/22.jpg)
Segmentation using Graph Cuts
State-of-the-art Tool (videosnapcut.mp4)http://juew.org/projects/SnapCut/snapcut.htm
![Page 23: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/23.jpg)
From 2D to 3D
http://www.eecs.harvard.edu/~zickler/helmholtz.html
![Page 24: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/24.jpg)
Projective Geometry
![Page 25: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/25.jpg)
Single View Metrology
• http://research.microsoft.com/vision/cambridge/3d/default.htm
![Page 26: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/26.jpg)
Single View Metrology
• http://research.microsoft.com/vision/cambridge/3d/default.htm
![Page 27: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/27.jpg)
Stereo
scene point
optical center
image plane
![Page 28: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/28.jpg)
Stereo
Basic Principle: Triangulation• Gives reconstruction as intersection of two rays• Requires
– Camera positions– point correspondence
![Page 29: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/29.jpg)
Using 3D structure to organize photos
http://phototour.cs.washington.edu/
![Page 30: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/30.jpg)
Using 3D structure to organize photos
http://photosynth.net/
![Page 31: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/31.jpg)
Reconstructing detailed 3D models
example input imagerendered model
![Page 32: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/32.jpg)
Reconstructing detailed 3D models
example input imagerendered model
![Page 33: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/33.jpg)
Reconstructing detailed 3D models
example input imagerendered model
http://grail.cs.washington.edu/projects/mvscpc/
![Page 34: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/34.jpg)
Reconstructing detailed 3D models
example input imagerendered model
![Page 35: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/35.jpg)
Reconstructing detailed 3D models
example input imagerendered model
![Page 36: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/36.jpg)
Application: View morphing
![Page 37: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/37.jpg)
Application: View morphing
![Page 38: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/38.jpg)
From Static Statues to Dynamic Targets
http://research.microsoft.com/~larryz/videoviewinterpolation.htm
…|
MSR Image based Reality Project
![Page 39: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/39.jpg)
Video Projectors
Color Cameras
Black & White Cameras
Spacetime Face Capture System
![Page 40: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/40.jpg)
System in Action
![Page 41: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/41.jpg)
Input Videos (640480, 60fps)
![Page 42: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/42.jpg)
Spacetime Stereo Reconstruction
![Page 43: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/43.jpg)
Applications
Entertainment: Games & Movies
Medical Practice:Prosthetics
![Page 44: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/44.jpg)
Computational Photography• High Dynamic Range
Conventional Image High Dynamic Range ImageNayar et al 2002
![Page 45: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/45.jpg)
Computational Photography• High Dynamic Range
High Dynamic Range ImageNayar et al 2002
Sensor Optics
Modulator
Assorted-pixel camera
![Page 46: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/46.jpg)
Computational Photography• High Dynamic Range
Digital Gain AdjustmentHandheld camera
![Page 47: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/47.jpg)
Computational Photography• High Dynamic Range
High Dynamic Range ImageZhang et al 2010
Handheld camera
![Page 48: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/48.jpg)
Summary• Recognize things• Reconstruct 3D structures• Enhance Photography
![Page 49: Components of a computer vision system](https://reader036.vdocuments.mx/reader036/viewer/2022062222/56816522550346895dd7a49d/html5/thumbnails/49.jpg)
If you are interested in,
Courses:CS766 Computer Vision CS638 Special Topics
Computational PhotographyCS638 Special Topics
Computational Methods in Medical Image Analysis
Faculty: Chuck Dyer, Vikas Singh, Li Zhang
Major Conferences: Computer Vision and Pattern Recognition (CVPR)International Conference on Computer Vision (ICCV)European Conference on Computer Vision (ECCV)ACM SIGGRAPH Conference (SIGGRAPH)