![Page 1: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/1.jpg)
CSC508
Homework Due
![Page 2: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/2.jpg)
CSC508
Homework
• Describe a task that you think might benefit from computer vision.– How is the task currently performed?– How could computer vision make it better?– Do you think it’s feasible to implement?
![Page 3: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/3.jpg)
CSC508
Image Formation
![Page 4: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/4.jpg)
CSC508
• First photograph due to Niepce – 1822
• Now: various films and digital (CCD, CMOS)• “Spectrum” of parameters
Image Formation
![Page 5: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/5.jpg)
CSC508
The Camera’s Job
• Basically, the job of the camera (no matter what the format) is mapping the 3D world onto a 2D plane – Yes, even a 3D camera does this…it just does it
twice
• The operation is called a “projection”
• There are two projections that we study/utilize– Perspective– Orthographic
![Page 6: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/6.jpg)
CSC508
Perspective Projection
• Size of the object on the image plane is dependent on the distance of the object from the image plane
![Page 7: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/7.jpg)
CSC508
Perspective Projection
• Parallel lines in the scene intersect at the horizon on the image plane
![Page 8: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/8.jpg)
CSC508
Vanishing Points
• Sets of parallel lines meet at a different points (for a given direction)– The vanishing point for this direction
• Sets of parallel lines on the same plane lead to collinear vanishing points. – The line is called the horizon for that plane
• An easy way to spot poorly faked images
![Page 9: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/9.jpg)
CSC508
Vanishing Points
![Page 10: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/10.jpg)
CSC508
Equation of Perspective Projection
Lens Center
Image PlaneReal Object
• (x, y, z) -> (f x/z, f y/z, -f) (by considering similar triangles – simple geometry)
• We ignore the 3rd coordinate since all image points are in the image plane
• Multiple real objects will map to the same image point• Why is that last point important?
![Page 11: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/11.jpg)
CSC508
Accuracy
• Most of computer vision is geared towards recognizing an object within a scene– For these applications, general knowledge of the
perspective projection is enough
• Some applications use computer vision to make measurements– For these applications accuracy is required– Therefore, we must calibrate the camera system
(lens, image plane)
![Page 12: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/12.jpg)
CSC508
Camera Parameters
• Optical axis relative to the image plane
• Angle between optical axis and image plane
• Focal length of the lens
• Size of the pixels in the image plane
• Position of camera in real world
• Orientation of camera in real world
![Page 13: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/13.jpg)
CSC508
Camera Calibration
• Through the use of appropriate target scenes and test set ups all these parameters can be derived
• Once derived, images can be compensated based on parameters thus creating a measurement device
![Page 14: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/14.jpg)
CSC508
Camera Calibration
• We know where the grid points are (relative to one another)• We know where the camera is (relative to the grid points)• Thus, we know where the [image of] the grid points should lie in the
image plane• We can create calibration factors based on where they should lie and
where they actually lie in the image plane• Search web for “Roger Tsai” and “Camera Calibration” for details
• It’s very math-intensive
![Page 15: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/15.jpg)
CSC508
Orthographic Projection
• If the camera is far from the objects relative to the depth (height?) of the objects– i.e. distance from scene objects to camera is
constant (flat scenes)– Provides an approximation of the perspective
projection– Sometimes useful for simplifying various
algorithms where depth is not a concern• Overhead aerial (air to ground) applications
![Page 16: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/16.jpg)
CSC508
Orthographic Projection
Lens Center
Image PlaneReal Object
![Page 17: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/17.jpg)
CSC508
Orthographic Projection
• Aerial image
![Page 18: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/18.jpg)
CSC508
Lens
• For our purposes the lens will– Provide a means of focus– Provide a means for more efficient light ray
capture
• We won’t go into the mathematics of lens design here – that’s better covered in a physics course
![Page 19: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/19.jpg)
CSC508
Image Representation
• Two dimensional array of values– Typically byte or integer (unsigned)– When performing operations you must do range checking
• Each array location is referred to as a pixel– Short for “picture element”
• Height and width of the array determine the image’s spatial resolution
• Bit depth of each pixel determines the intensity resolution– 8-bit image – each pixel is in the range of 0..255– If it’s greater than 8 you must scale it prior to display as most monitors
can only display 256 gray levels– 24-bit color images are merely three 8-bit color images “stacked”
together
![Page 20: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/20.jpg)
File Storage
• There are many image file formats in use today– jpg –JPEG– png – portable network graphics– tiff – tagged image file format– j2k – JPEG 2000– raw – no meta-data, just image data– gif – graphics interchange format– bmp – bitmap
CSC508
![Page 21: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/21.jpg)
File Storage
• Differences include– Compressed vs. uncompressed– Inclusion of image meta-data– Inclusion of device meta-data– Inclusion of processing meta-data– Allowable bit depth– etc.
CSC508
![Page 22: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/22.jpg)
CSC508
24-Bit BMP File Formatan example
• Bitmap file header– Data structure that tells the reader that it is a BMP
file
• Bitmap information header– Data structure that tells the reader the format of the
image
• Image data– The picture elements (pixels) that make up the
picture
![Page 23: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/23.jpg)
CSC508
Bitmap File Header Fields
• bfType – 2 bytes, always set to ‘B’ and ‘M’• bfSize – 4 bytes, size of the file in bytes (file header +
image header + pixels) • bfReserved1 – 2 bytes, reserved for future usage• bfReserved2 – 2 bytes, reserved for future usage • bfOffBits – 4 bytes, number of bytes in file before
image starts (file header + image header)
![Page 24: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/24.jpg)
CSC508
Bitmap Information Header Fields• biSize – 4 bytes, size in bytes of this structure (40)• biWidth – 4 bytes (signed), number of pixels per row• biHeight – 4 bytes (signed), number of rows per image• biPlanes – 2 bytes, number of image planes (1)• biBitCount – 2 bytes, number of bits per pixel (24)• biCompression – 4 bytes, type of compression (0)• biSizeImage – 4 bytes, number of pixels in the file
– Not always (biWidth * biHeight * 3) because image rows must be a multiple of 4 bytes (to support fast transfer on a 32-bit bus)
• biXPelsPerMeter – 4 bytes (signed), for printer usage (ignore)• biYPelsPerMeter – 4 bytes (signed), for printer usage (ignore)• biClrUsed – 4 bytes, number of colors actually used in images with less
than 24 bits per pixel (0)• biClrImportant – 4 bytes, number of “important” colors in images with less
than 24 bits per pixel (0)
![Page 25: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/25.jpg)
CSC508
Bitmap File Image Data
• Pixels are stored in Blue-Green-Red order
• Rows of pixels must be multiples of 4 bytes
• Bottom row of image is stored first, top row is stored last– i.e. image is stored upside down
![Page 26: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/26.jpg)
CSC508
BMP File Writer
• I can place code to read/write BMP and raw files on the class website
• Both Java and C++ versions can be there– If you want VB (or anything else) you’ll have to study the
code Java or C++ code and make your own
• They may not be the most efficient but they work• For those of you writing in Java, a better solution is
to use the ImageIO class or write ImageJ plugins– The popular image formats are supported for read and write
![Page 27: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/27.jpg)
CSC508
Preprocessing
![Page 28: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/28.jpg)
CSC508
Intensity Histogram
• Histogram– Distribution of pixel
intensities
– One dimensional array of integers
– Size of the array is directly related to the pixel resolution
![Page 29: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/29.jpg)
CSC508
Contrast Enhancement
• Histogram Stretch– Compute the image histogram
– Specify dark and bright cut-off points• This is usually done percentiles of the distribution – then
converted to actual intensity cutoffs
• Alternatively, may be specified as two fixed intensity cutoffs
– For each pixel I(i, j) compute:
)(
255,,'
cuttoffcutoffcutoff
darkbright
darkjiIjiI
![Page 30: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/30.jpg)
CSC508
Percentiles to Intensity Cutoffs
Count pixels in histogram bins until you reach the desired percentile values to get the two cutoff points
![Page 31: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/31.jpg)
CSC508
Contrast Enhancement
![Page 32: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/32.jpg)
CSC508
Resultant Histogram
Note: The scale on the vertical axis has changed due to the display programThe high frequency bumps are due to the multiply operation
![Page 33: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/33.jpg)
CSC508
Before/After Comparison
![Page 34: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/34.jpg)
CSC508
Contrast Enhancement
• Why do we want to do this?
• Because contrast enhancement will bring out some features and suppress others
• The problem is that it’s not easy to control
• Basically, it’s an operation that is more for human consumption than computer vision
• We study it as an introduction to the histogram
![Page 35: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/35.jpg)
CSC508
Contrast Enhancement
• Binarize/Threshold– Choose a threshold
– Perform the following [parallel] operation on every pixel
jiI
jiI
,
, 0,'
255,'
jiI
jiIif then
− The result is a 1-bit image (represented in 8-bits for display)
![Page 36: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/36.jpg)
CSC508
Binarize/Threshold
![Page 37: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/37.jpg)
CSC508
Noise Reduction
• Median Filter– Define a neighborhood around every pixel
• Should be odd dimensions (but not absolutely necessary)
– For every pixel I(i, j)• Numerically sort the pixel values in the neighborhood
• Replace I(i, j) with the median (middle) value of the sorted neighborhood
– The result is the removal of “salt and pepper” artifacts
– Note that this operation is parallel in that all pixels perform their operation simultaneously
• So, how do you do this on a sequential machine?
![Page 38: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/38.jpg)
CSC508
Median Filter
Little spots – trust me
![Page 39: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/39.jpg)
CSC508
Noise Reduction
• Outlier Filter– Define a neighborhood around every pixel
• Should be odd dimensions (but not absolutely necessary)
– For every pixel I(i, j)• Compute the average pixel value of the neighborhood, n(i, j)
• If |I(i, j)- n(i, j)| > threshold then replace I(i, j) with the neighborhood average
• Result will be sensitive to the selected threshold
– The result is the removal of “salt and pepper” artifacts
– Note that this operation is parallel in that all pixels perform their operation simultaneously
![Page 40: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/40.jpg)
CSC508
Gaussian Filter
• Width is the number of pixels covered by the filter
• Sigma is the standard deviation of the Gaussian curve in pixels
eGyx
yx 2
22
22
1, 2
2,2widthyxwidth
![Page 41: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/41.jpg)
CSC508
Gaussian Filter
• Sigma 1.0, Filter Width 7
1 2 3 4 5 6 7S1
S3
S5S7
0.00E+00
2.00E-02
4.00E-02
6.00E-02
8.00E-02
1.00E-01
1.20E-01
1.40E-01
1.60E-01
![Page 42: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/42.jpg)
CSC508
Gaussian Filter
• Sigma 1.6, Filter Width 7
1 2 3 4 5 6 7S1
S3
S50
0.01
0.02
0.03
0.04
0.05
0.06
0.07
![Page 43: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/43.jpg)
CSC508
Gaussian Filter
• Sigma 1.6, Filter Width 15
1 3 5 7 9
11
13
15
S1
S6
S11
0.00E+00
1.00E-02
2.00E-02
3.00E-02
4.00E-02
5.00E-02
6.00E-02
7.00E-02
![Page 44: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/44.jpg)
CSC508
Gaussian is Separable
• Two 1D Gaussians produces the same results a one 2D Gaussian– Filter horizontally first– Filter horizontal results vertically second– This speeds things up dramatically
![Page 45: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/45.jpg)
CSC508
Gaussian Filter – 1D
eGx
x 2
2
22
12
22widthxwidth
![Page 46: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/46.jpg)
CSC508
Gaussian Filter – 1D
• Sigma 1.6, Filter Width 15
0.00E+00
1.00E-02
2.00E-02
3.00E-02
4.00E-02
5.00E-02
6.00E-02
7.00E-02
1 3 5 7 9 11 13 15
![Page 47: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/47.jpg)
CSC508
Gaussian Filter
• How do we use the Gaussian Filter?– Convolution – which we’ll talk about next week
![Page 48: CSC508 Homework Due. CSC508 Homework Describe a task that you think might benefit from computer vision. –How is the task currently performed? –How could](https://reader036.vdocuments.mx/reader036/viewer/2022062408/56649e495503460f94b3c95b/html5/thumbnails/48.jpg)
CSC508
Things To Do
• Programming homework assignment– Histogram Stretch– Binarization– Median Filter– Outlier Filter
• You may write in any programming language you choose• Deliverables:
– Live demonstration of your code to me (if this is feasible)– Email the source code to [email protected] with the subject line
CSC508 PROGRAM 1• Due next week beginning of class
– (late assignments will be penalized 10%)– I will post test images
• Reading for Next Week – Chapter 8 – Edge Detection