CS 423 (CS 423/CS 523)
Computer Vision
Lecture 1INTRODUCTION TO COMPUTER VISION
About the Course
2
http://vvgl.ozyegin.edu.tr
Objective
Introduction to the theory, tools, and algorithms of computer vision
Instructor
Assist. Prof. M. Furkan Kıraç
E-mail: [email protected]
Room: 219
Hours
Mondays, 9:40-12:30, Room: 246
Grading
Projects: 4x15%
Midterm Exam: 40%
Syllabus
3
Projects:Late submissions are not accepted. Copying answers from others’ work is not permitted.
Midterm Exam:At least 3 of the 4 Projects must be turned in by the due date in order to qualify for the Final Exam. No Composite Exam (Bütünleme Sınavı), as there is no final exam.
Grading
4
Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010.
Computer Vision: A Modern Approach, David A. Forsyth and Jean Ponce, Prentice-Hall, 2002.
Introductory Techniques for 3D Computer
Vision, Emanuele Trucco and Alessandro Verri, Prentice-Hall 1998.
Recommended Books
5
OpenCV Computer Vision Application Programming Cookbook Second Editon, Robert Laganiere, Packt Publishing, 2014.
Learning OpenCV, Gary Bradski and Adrian Kaehler, O'Reilly, 2008.
Mastering OpenCV with Practical Computer Vision Projects, Daniel Lelis Baggio, et al., Packt Publishing, 2012.
OpenCV Resources
6
Applications of Computer Vision
7
Image Stitching
Image Matching
Object Recognition
3D Reconstruction
Interior Modeling
12
3D Augmented Reality
13
3D Camera Tracking
14
15
Stereo Conversion for 3DTV
Depth Estimation and View Interpolation for 3DTV
16
Human Tracking
17
License Plate Recognition
18
Human Pose Estimation
19
Course Outline
20
Linear Filters, Frequency Domain Filtering, Edge and Boundary Detection Feature Detection Fitting, Alignment Histograms Covariance, Principle Component Analysis (PCA) Face Detection and PCA Optical Flow and Motion Tracking and Mean-Shift Randomized Decision Trees, Pose Estimation Bag of Features Context, Two-View Geometry Summary
Topics to be covered...
21
Relation to Other Fields
22
Computer Vision
23
Figure from "Computer Vision: Algorithms and Applications,” Richard Szeliski, Springer, 2010.
Lights and materials Shading Texture mapping Environment effects Animation 3D scene modeling 3D character modeling (OpenGL)
Computer Graphics
24
Computer Graphics
25
Resampling Enhancement Noise filtering Restoration Reconstruction Segmentation Image compression (MATLAB and OpenCV)
Image Processing Topics
26
Image Processing
27
Motion estimation Frame-rate conversion Multi-frame noise filtering Multi-frame restoration Super-resolution Video compression (MATLAB & OpenCV)
Video Processing Topics
28
Video acquisition-display chain
29
Capture Representation Coding
Transmission Decoding Rendering
Human vs. Computer
30
Optical illusions
Actual vs. Perceived Intensity (Mach band effect)
32
Brightness Adaptation of the Eye
33
Optical illusions
Optical illusions
Why is Computer Vision Difficult?
Human perception
Human perception
Human Visual System
40
Human Eye
Photoreceptors: Rods & Cones
Rods vs. Cones
RodsPerceive brightness onlyNight vision
ConesPerceive colorDay visionRed, green, and blue cones
Cone Distribution
64%
32%
2%
Blue is less-focused
Visual Threshold drop during Dark Adaptation
Spatial Resolution of the Human Eye Photopic (bright-light) vision:
Approximately 7 million cones Concentrated around fovea
Scotopic (dim-light) vision Approximately 75-150 million rods Distributed over retina
(HDTV: 1920x1080 = 2 million pixels)
49
Frequency Responses of Cones
Same amount of energy produces different sensations of brightness at different wavelengths
Green wavelength contributes most to the perceived brightness.
50
Trichromatic Color Mixing
Any color can be obtained by mixing three primary colors Red, Green, Blue (RGB) with the right proportion
valuessTristimulu :,3,2,1
kk
kk TCTC
Image Formation
53
Human Eye vs. Camera
Camera components Eye components
Lens Lens, cornea
Shutter Iris, pupil
Film Retina
Cable to transfer images Optic nerve to send the incident light information to the brain
Human Vision
Image formation
Pin-Hole Camera Model
Point Spread Effect
Out-of-Focus Blur
Shrinking the Aperture
Converging Lens
Correction with a Converging Lens
Perfectly In-Focus for a Certain Distance Only
“circle of confusion”
Depth-of-Field
Depth-of-Field
“Sharp Image” within Depth-of-Field due to Finite Sensor Size
NZFZ
Focal Length (F) and Depth (Z)
Z
YFy
F
y
Z
Y
Z
XFx
Aperture Size Affects Depth-Of-Field
f / 5.6
f / 32
Aperture
2dA
Camera f-number
d
Ff
2
f
FA
Exposure Time
Motion Blur Effect due to Finite Exposure Time
Decrease in aperture implies…
Increase in depth-of-field Decrease in motion blur Decrease in exposure
2D Image Representation
75
76
Image Capture
(Courtesy Gonzalez & Woods)
Digital Image Capture
Digital Image Capture
Light sensitive diodes convert photons to electrons
Color Image Capture: Single vs. Three CCD Arrays
RGB splitter(three separate imaging sensors, higher resolution)
Bayer filter(cheaper but introduces spatial resolution loss)
Digital Camera Issues
Noise caused by low light
Color color fringing (chromatic aberration) artifacts from Bayer patterns
Blooming charge overflowing into neighboring pixels
In-camera processing over-sharpening can produce halos
Compression creates blocking artefacts
Digitization: Sampling and Quantization
Sampling Rate Problem
Over Quantization
83
84
Images as Matrices of Integers
126 127 126
125 126 127
123 126 125
128 127 124
123 120 144
121 128 155
126 123 127
120 122 124
119 121 123
122 142 162
130 157 161
145 162 164
158
163
160
164
166
165
m
n
(0,0)
0 ≤ s(m,n) ≤ 255 } quantization
0 ≤ m ≤ M-1
0 ≤ n ≤ N-1
MxN 8-bit gray-scale (intensity, luminance) image
sampling
0 → black, 255 → white
Images as Functions
We can think of an image as a function, f, from R2 to R: f( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined
over a rectangle, with a finite range:• f: [a,b]x[c,d] [0,1]
A color image is just three functions pasted together. We can write this as a “vector-valued” function:
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
RGB Color Bands (Channels)
Red
Green Blue
YUV Bands
Also called Y Cb Cr Y : Luma
Cb : Chrominance_blueCr : Chrominance_red
Y
U (Cb)
V(Cr)
Color
YUV-RGB Conversion
Summary
89
Human visual system
Pin-hole camera model
Image representation
Summary
90