#1 lecture 1: computer vision - university of...
TRANSCRIPT
![Page 1: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/1.jpg)
ME5286 – Lecture 1 (Theory)
Lecture 1: Computer Vision
Introduction
Saad J Bedros, PhDOffice:105D Walter Library
Phone: (612) 624-4822, (612) 626-3421Email: [email protected]
#1
![Page 2: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/2.jpg)
ME5286 – Lecture 1 (Theory)
Book References
1. A.K. Jain: Fundamentals of Digital Image Processing, Prentice Hall,1989
2. E. Trucco and A. Verri: Introductory Techniques for 3-D Computer Vision
3. R.C. Gonzalez and R.E. Woods: Digital Image Processing, Prentice-Hall, 2002
4. David A. Forsyth and Jean Ponce: Computer vision: a modern approach. Prentice Hall, 2002.
5. Bernd Jähne: Digital image processing, Springer-Verlag Berlin Heidelberg 2001
#2
![Page 3: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/3.jpg)
ME5286 – Lecture 1 (Theory)
General Comments about the Class• Prerequisites—these are essential!
– Vector calculus– Statistics– Linear algebra
• Course does not assume prior imaging experience– computer vision, image processing, graphics, etc.
• Emphasis on programming projects!• Collaboration Policy
– Assignments may be discussed, but all written work and coding must be done individually. Mini project may not be discussed. Individuals found submitting duplicate or substantially similar materials due to inappropriate collaboration may get an “F” in this class and there may be other serious consequences.
• Matlab will be used for Homeworks and Projects
#3
![Page 4: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/4.jpg)
ME5286 – Lecture 1 (Theory)
Course Syllabus: Vision PartIntro, syllabus, robot overview.
20 Apr 3 F 1- Introduction to computer vision and image processing: a review. Matlab as a tool for image processing.
21 Apr 8 W 2- Image formation, Camera fundamentals.22 Apr 10 F 3- Spatial Domain Transformations:
Point, Area/Mask, Geometric and Frame processing Transformations. Image coordinate transformations
23 Apr 15 W 4- Image histograms, histogram equalization and histogram matching
24 Apr 17 F QUIZ – NO Lecture25 Apr 22 W 5- Edge detection methods26 Apr 24 F 6- Edge detection continued – Canny edge detection
27 Apr 29 W 7- Morphological binary image Operations:Erosion, dilation, opening, closing, hit or miss transforms.
28 May 1 F 8- Corner/line detection and Hough transforms29 May 6 W 9- Generalized Hough transform. Image Thresholding methods
30 May 8 F 10 - Course review. Take home final handed out.(Last day of class)
#4
![Page 5: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/5.jpg)
ME5286 – Lecture 1 (Theory)
About Computer Vision
• Deals with the formation, analysis and Interpretation Images
• Is an integral part of Artificial Intelligence (AI)• Is interdisciplinary subject area• Is Practical and useful• Is modern, challenging and continuously
evolving
#5
![Page 6: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/6.jpg)
ME5286 – Lecture 1 (Theory)
Computer Vision Difficulties• Images are ambiguous : Projective • Images are affected by many factors
– Sensor model– Illumination– Shape of object(s)– Color of object(s)– Texture of object(s)
• There is no “Universal Solution” to the Vision problem• There are many theories• We do not understand how the human Visual System works• Computer technology changes continuously, Hence Computer
Vision is evolving.
#6
![Page 7: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/7.jpg)
ME5286 – Lecture 1 (Theory)
Why is Computer Vision Difficult?
• It is a many-to-one mapping– A variety of surfaces with different material and
geometrical properties, possibly under different lighting conditions, could lead to identical images
– Inverse mapping is under-constrained – non-unique solution (a lot of information is lost in the transformation from the 3D world to the 2D image)
• It is computationally intensive• We still do not understand the recognition
problem
#7
![Page 8: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/8.jpg)
ME5286 – Lecture 1 (Theory)
Illusions#8
![Page 9: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/9.jpg)
ME5286 – Lecture 1 (Theory)
Illusions#9
![Page 10: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/10.jpg)
ME5286 – Lecture 1 (Theory)
What do humans see?
![Page 11: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/11.jpg)
ME5286 – Lecture 1 (Theory)
Nomenclature
• Somewhat interchangeable names, with somewhat different implications:– Computer Vision
• Most general term– Computational Vision
• Includes modeling of biological vision– Image Understanding or Scene Understanding
• Automated scene analysis (e.g., satellite images, robot navigation)– Machine Vision
• Industrial, factory-floor systems for inspection, measurements, part placement, etc.
#11
![Page 12: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/12.jpg)
ME5286 – Lecture 1 (Theory)
Related Fields• Largely built upon
– Image Processing– Statistical Pattern Recognition, Machine Learning– Artificial Intelligence
• Related areas– Robotics– Biological vision– Medical imaging– Computer graphics– Human-computer interaction
#12
![Page 13: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/13.jpg)
ME5286 – Lecture 1 (Theory)
Brief History of Computer Vision• B.C. (Before Computers)
– Philosophy– Optics– Psychophysics– Neurophysiology
• Psychophysics and neurophysiology describe the behavior of cells or subjects, but do not explain such behavior– Black box – how can we figure it out?
#13
![Page 14: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/14.jpg)
ME5286 – Lecture 1 (Theory)
• 1960s – First computer vision programs/systems• Minsky at MIT – Solve vision in a summer
– Not really…• Empirical approaches
– Ad hoc, “bag of tricks”– Image processing plus…
• Simplified worlds– “Blocks world” then generalize– AI approach
Brief History of Computer Vision
Enhance Threshold Removenoise
Connectedregions Analysis
#14
![Page 15: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/15.jpg)
ME5286 – Lecture 1 (Theory)
Brief History of Computer Vision• Computational
– Attempt at a rigorous study based on information processing model (David Marr)
• Task-oriented– Practical systems to accomplish specific tasks
• Currently– All of this (and more)– Limited progress
• However, we have a much deeper understanding of the issues (and of the difficulty)
– Still no “killer app”
#15
![Page 16: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/16.jpg)
ME5286 – Lecture 1 (Theory)
Progress in Computer Vision• First generation: Military/Early Research
– Few systems, each custom-built, cost $Ms– “Users” have PhDs– 1 hour per frame
• Second generation: Industrial/Medical– Numerous systems, 1000s of each, cost $10Ks– Users have college degree – Special hardware
• Third generation: Consumer– 100000(00) systems, cost $100s– Users have little or no training– Emphasis on software
#16
![Page 17: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/17.jpg)
ME5286 – Lecture 1 (Theory)
The Three Processing Levels• Low-level processing
– Standard procedures are applied to improve image quality– No “intelligent” capabilities
#17
![Page 18: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/18.jpg)
ME5286 – Lecture 1 (Theory)
The Three Processing Levels• Intermediate-level processing
– Extract and characterize components in the image– Some intelligent capabilities are required
#18
![Page 19: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/19.jpg)
ME5286 – Lecture 1 (Theory)
The Three Processing Levels• High-level processing
– Recognition and interpretation– Procedures require high intelligent capabilities
#19
![Page 20: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/20.jpg)
ME5286 – Lecture 1 (Theory)
Computer Vision• Some applications
– Robotics• Navigation, object manipulation, interaction with humans…
– Inspection, measurement– Medical imaging– Graphics and animation, special effects– Multimedia database indexing and retrieval– Human-computer interaction– Surveillance and security
#20
![Page 21: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/21.jpg)
ME5286 – Lecture 1 (Theory)
Computer Vision
Model
Output:
Real Scene
Cameras Images
#21
![Page 22: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/22.jpg)
ME5286 – Lecture 1 (Theory)
Computer Graphics
Image
Output:
ModelSyntheticCamera
Projection, shading, lighting models#22
![Page 23: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/23.jpg)
ME5286 – Lecture 1 (Theory)
Computer vision
Another View
Actually works
Interesting
Machinevision
Computationalvision
#23
![Page 24: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/24.jpg)
ME5286 – Lecture 1 (Theory)
Why study Computer Vision?
• Images and movies are everywhere– Mobile phone, cheaper cameras enabling the imaging capture
• Fast-growing collection of useful applications– Face finding, recognition, analysis– building representations of the 3D world from pictures– automated surveillance (who’s doing what)– movie post-processing
• Various deep and attractive scientific mysteries– how does object recognition work?
• Greater understanding of human vision
#24
![Page 25: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/25.jpg)
ME5286 – Lecture 1 (Theory)
Every picture tells a story
• Goal of computer vision is to write computer programs that can interpret images
#25
![Page 26: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/26.jpg)
ME5286 – Lecture 1 (Theory)
Examples of Commercial Vision Systems
High Precision Inspection System
NASA Rover
Par Systems: Vision Guided Robotics for manufacturing and assembly lines
There are many products that have an element of computer vision, and the number is increasing every day
#26
![Page 27: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/27.jpg)
ME5286 – Lecture 1 (Theory)
Google, Facebook, NEC•Face Detection•Face Labeling•Face Finding
Security Applications#27
![Page 28: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/28.jpg)
ME5286 – Lecture 1 (Theory)
Face Detection
Challenges: Pose and Illumination variations
#28
![Page 29: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/29.jpg)
ME5286 – Lecture 1 (Theory)
Face Recognition#29
![Page 30: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/30.jpg)
ME5286 – Lecture 1 (Theory)
Multi Biometrics#30
![Page 31: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/31.jpg)
ME5286 – Lecture 1 (Theory)
Vision-Based Interfaces
MIT Media Lab3D Face Analysis and Recognition
Intel and Microsoft New 3D Sensors
Gesture Recognition
#31
![Page 32: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/32.jpg)
ME5286 – Lecture 1 (Theory)
Visual Inspection#32
![Page 33: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/33.jpg)
ME5286 – Lecture 1 (Theory)
Indexing into Databases• Shape content
#33
![Page 34: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/34.jpg)
ME5286 – Lecture 1 (Theory)
Application: Optical Character Recognition
License Plate Recognition
#34
![Page 35: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/35.jpg)
ME5286 – Lecture 1 (Theory)
• Color, texture
Indexing into Databases
![Page 36: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/36.jpg)
ME5286 – Lecture 1 (Theory)
Target Recognition
• Department of Defense (Army, Airforce, Navy)
#36
![Page 37: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/37.jpg)
ME5286 – Lecture 1 (Theory)
Interpretation of Aerial Photography#37
![Page 38: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/38.jpg)
ME5286 – Lecture 1 (Theory)
Traffic Monitoring#38
![Page 39: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/39.jpg)
ME5286 – Lecture 1 (Theory)
Brain Imaging#39
![Page 40: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/40.jpg)
ME5286 – Lecture 1 (Theory)
Applications: Robotics#40
![Page 41: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/41.jpg)
ME5286 – Lecture 1 (Theory)
Autonomous Vehicles
• Land, Underwater, Space#41
![Page 42: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/42.jpg)
Stanford Racing Team
Grand Challenge 2005: 195 Teams
![Page 43: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/43.jpg)
ME5286 – Lecture 1 (Theory)
Darpa Challenge 2005
Following slides courtesy of Sebastian Thrun
![Page 44: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/44.jpg)
ME5286 – Lecture 1 (Theory)
Urban Darpa Challenge 2007
Autonomous Driving in Traffic
#44
![Page 45: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/45.jpg)
ME5286 – Lecture 1 (Theory)
Darpa Robotic Challenge 2014-2015
The DRC is a competition of robot systems and software teams vying to develop robots capable of assisting humans in responding to natural and man-made
#45
![Page 46: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/46.jpg)
ME5286 – Lecture 1 (Theory)
Types of images
• infra-red• ultra-violet• radio-waves (radio astronomy)• visible light• micro-waves (radar)• roentgen (tomography)• sound-waves (echoscopy, sonar)•electrons (microscopy)• positron emission (PET-scan)• magnetic resonance (NMR)
#46
![Page 47: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/47.jpg)
ME5286 – Lecture 1 (Theory)
Electromagnetic (EM) Spectrum
#47
![Page 48: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/48.jpg)
ME5286 – Lecture 1 (Theory)
Electromagnetic (EM) Spectrum#48
![Page 49: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/49.jpg)
ME5286 – Lecture 1 (Theory)
Imaging in Different Wavelengths#49
![Page 50: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/50.jpg)
ME5286 – Lecture 1 (Theory)
Face and Face Feature Localization#50
![Page 51: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/51.jpg)
ME5286 – Lecture 1 (Theory)
Thermal Imaging#51
![Page 52: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/52.jpg)
ME5286 – Lecture 1 (Theory)
Video Analytics Security Architecture#52
![Page 53: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/53.jpg)
ME5286 – Lecture 1 (Theory)
Camera Network Physical Design•color cameras
•medium resolution
•smaller Field of Views
60
camera symbol
Field of View symbol
#53
![Page 54: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/54.jpg)
ME5286 – Lecture 1 (Theory)
Statistical Modeling of Image Pixels#54
![Page 55: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/55.jpg)
ME5286 – Lecture 1 (Theory)
Continuous Learning of Foreground/Background pair
#55
![Page 56: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/56.jpg)
ME5286 – Lecture 1 (Theory)
Outdoor Motion Detection#56
![Page 57: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/57.jpg)
ME5286 – Lecture 1 (Theory)
Video Surveillance: Tracking#57
![Page 58: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/58.jpg)
ME5286 – Lecture 1 (Theory)
Video Surveillance#58
![Page 59: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/59.jpg)
ME5286 – Lecture 1 (Theory)
Video Surveillance: Tracking#59
![Page 60: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/60.jpg)
ME5286 – Lecture 1 (Theory)
Human Tracking and Pan/Tilt Control#60
![Page 61: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/61.jpg)
ME5286 – Lecture 1 (Theory)
Video Tracking: Multiple PTZCameras#61
![Page 62: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/62.jpg)
ME5286 – Lecture 1 (Theory)
People Detection and Tracking#62
![Page 63: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/63.jpg)
ME5286 – Lecture 1 (Theory)
Human Activity Recognition#63
![Page 64: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/64.jpg)
ME5286 – Lecture 1 (Theory)
Effective Command & Control
Coverage areas are enormous Too many images - it is difficult to interpret activities
Security Today:
#64
![Page 65: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/65.jpg)
ME5286 – Lecture 1 (Theory)
Command & Control Integrate hundreds of sensors in a
unified view Move seamlessly in space - god’s eye
view to up close Move seamlessly in time -
replay and rehearse Alert operators to key events ID individuals Track and overlay Record, compress and store Archive and data mine
#65
![Page 66: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/66.jpg)
ME5286 – Lecture 1 (Theory)
Next Generation Browser “Fly-Through” with Real-Time Video
Project video on a graphics model – with ability to render from any view
#66
![Page 67: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/67.jpg)
ME5286 – Lecture 1 (Theory)
3D Tracking and Registration#67
![Page 68: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/68.jpg)
ME5286 – Lecture 1 (Theory)
City Surveillance#68
![Page 69: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/69.jpg)
ME5286 – Lecture 1 (Theory)
Can computers match human perception?
• Not yet– computer vision is still no match for human perception– but catching up, particularly in certain areas
#69
![Page 70: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/70.jpg)
ME5286 – Lecture 1 (Theory)
Computer Vision Representations
Image(s)
Color
Shape
Shading
Texture
Motion
Stereo
Surfaces1
2"2 D Sketch"− Objects
Iconic Segmented Geometric Relational
#70
![Page 71: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/71.jpg)
ME5286 – Lecture 1 (Theory)
Part I: The Physics of Imaging• How images are formed
– Cameras• How a camera creates an image• How to tell where the camera was
– Light• How to measure light• What light does at surfaces• How the brightness values we see in cameras are
determined– Color
• The underlying mechanisms of color• How to describe it and measure it
#71
![Page 72: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/72.jpg)
ME5286 – Lecture 1 (Theory)
Part II: Early Vision in One Image• Representing small patches of image
– For three reasons• We wish to establish correspondence between (say)
points in different images, so we need to describe the neighborhood of the points
• Sharp changes are important in practice --- known as “edges”
• Representing texture by giving some statistics of the different kinds of small patch present in the texture.
– Tigers have lots of bars, few spots– Leopards are the other way
#72
![Page 73: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/73.jpg)
ME5286 – Lecture 1 (Theory)
Representing an image patch
• Filter outputs– essentially form a dot-product between a
pattern and an image, while shifting the pattern across the image
– strong response -> image locally looks like the pattern
– e.g. derivatives measured by filtering with a kernel that looks like a big derivative (bright bar next to dark bar)
#73
![Page 74: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/74.jpg)
ME5286 – Lecture 1 (Theory)
Texture• Many objects are distinguished by their texture
– Tigers, cheetahs, grass, trees• We represent texture with statistics of filter outputs
– For tigers, bar filters at a coarse scale respond strongly
– For cheetahs, spots at the same scale– For grass, long narrow bars– For the leaves of trees, extended spots
• Objects with different textures can be segmented• The variation in textures is a cue to shape
#74
![Page 75: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/75.jpg)
ME5286 – Lecture 1 (Theory)
Part III: Early Vision in Multiple Images
• The geometry of multiple views– Where could it appear in camera 2 (3, etc.)
given it was here in 1 (1 and 2, etc.)?• Stereopsis
– What we know about the world from having 2 eyes
• Structure from motion– What we know about the world from having
many eyes• or, more commonly, our eyes moving.
#75
![Page 76: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/76.jpg)
ME5286 – Lecture 1 (Theory)
Part IV: Mid-Level Vision
• Finding coherent structure so as to break the image or movie into big units– Segmentation:
• Breaking images and videos into useful pieces• E.g. finding image components that are coherent in
internal appearance– Tracking:
• Keeping track of a moving object through a long sequence of views
#76
![Page 77: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/77.jpg)
ME5286 – Lecture 1 (Theory)
Part V: High Level Vision (Geometry)• The relations between object geometry and
image geometry– Model based vision
• find the position and orientation of known objects– Smooth surfaces and outlines
• how the outline of a curved object is formed, and what it looks like
– Aspect graphs• how the outline of a curved object moves around as
you view it from different directions– Range data
#77
![Page 78: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/78.jpg)
ME5286 – Lecture 1 (Theory)
Part VI: High Level Vision (Probabilistic)• Using classifiers and probability to
recognize objects– Templates and classifiers
• how to find objects that look the same from view to view with a classifier
– Relations • break up objects into big, simple parts, find the parts
with a classifier, and then reason about the relationships between the parts to find the object.
– Geometric templates from spatial relations• extend this trick so that templates are formed from
relations between much smaller parts
#78
![Page 79: #1 Lecture 1: Computer Vision - University of Minnesotame.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture1.pdf · (Last day of class) #4: ME5286 – Lecture 1 ... • Most](https://reader034.vdocuments.mx/reader034/viewer/2022042502/5afaf94b7f8b9abd588ef85f/html5/thumbnails/79.jpg)
ME5286 – Lecture 1 (Theory)
Conclusion
• Computer Vision is an exciting and challenging field
• Applied in many real world solutions• Tremendous progress in the last decades• Still have long way to go …
#79