templates, image pyramids, and filter banks computer vision james hays, brown 09/19/11 slides: hoiem...
TRANSCRIPT
![Page 1: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/1.jpg)
Templates, Image Pyramids, and Filter Banks
Computer VisionJames Hays, Brown
09/19/11
Slides: Hoiem and others
![Page 2: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/2.jpg)
Review
1. Match the spatial domain image to the Fourier magnitude image
1 54
A
32
C
B
DE
Slide: Hoiem
![Page 3: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/3.jpg)
Reminder• Project 1 due in one week
![Page 4: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/4.jpg)
Today’s class
• Template matching
• Image Pyramids
• Filter banks and texture
• Denoising, Compression
![Page 5: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/5.jpg)
Template matching• Goal: find in image
• Main challenge: What is a good similarity or distance measure between two patches?– Correlation– Zero-mean correlation– Sum Square Difference– Normalized Cross
Correlation
Slide: Hoiem
![Page 6: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/6.jpg)
Matching with filters• Goal: find in image• Method 0: filter the image with eye patch
Input Filtered Image
],[],[],[,
lnkmflkgnmhlk
What went wrong?
f = imageg = filter
Slide: Hoiem
![Page 7: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/7.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image• Method 1: filter the image with zero-mean eye
Input Filtered Image (scaled) Thresholded Image
)],[()],[(],[,
lnkmgflkfnmhlk
True detections
False detections
mean of f
![Page 8: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/8.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image• Method 2: SSD
Input 1- sqrt(SSD) Thresholded Image
2
,
)],[],[(],[ lnkmflkgnmhlk
True detections
![Page 9: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/9.jpg)
Matching with filters• Goal: find in image• Method 2: SSD
Input 1- sqrt(SSD)
2
,
)],[],[(],[ lnkmflkgnmhlk
What’s the potential downside of SSD?
Slide: Hoiem
![Page 10: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/10.jpg)
Matching with filters• Goal: find in image• Method 3: Normalized cross-correlation
5.0
,
2,
,
2
,,
)],[()],[(
)],[)(],[(
],[
lknm
lk
nmlk
flnkmfglkg
flnkmfglkg
nmh
Matlab: normxcorr2(template, im)
mean image patchmean template
Slide: Hoiem
![Page 11: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/11.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
![Page 12: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/12.jpg)
Matching with filters• Goal: find in image• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
Slide: Hoiem
![Page 13: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/13.jpg)
Q: What is the best method to use?
A: Depends• SSD: faster, sensitive to overall intensity• Normalized cross-correlation: slower,
invariant to local average intensity and contrast
![Page 14: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/14.jpg)
Q: What if we want to find larger or smaller eyes?
A: Image Pyramid
![Page 15: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/15.jpg)
Review of Sampling
Low-Pass Filtered ImageImage
GaussianFilter Sample
Low-Res Image
Slide: Hoiem
![Page 16: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/16.jpg)
Gaussian pyramid
Source: Forsyth
![Page 17: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/17.jpg)
Template Matching with Image Pyramids
Input: Image, Template1. Match template at current scale
2. Downsample image
3. Repeat 1-2 until image is very small
4. Take responses above some threshold, perhaps with non-maxima suppression
Slide: Hoiem
![Page 18: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/18.jpg)
Coarse-to-fine Image Registration1. Compute Gaussian pyramid2. Align with coarse pyramid3. Successively align with finer
pyramids– Search smaller range
Why is this faster?
Are we guaranteed to get the same result?
Slide: Hoiem
![Page 19: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/19.jpg)
Laplacian filter
Gaussianunit impulse
Laplacian of Gaussian
Source: Lazebnik
![Page 20: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/20.jpg)
2D edge detection filters
is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
![Page 21: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/21.jpg)
Laplacian pyramid
Source: Forsyth
![Page 22: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/22.jpg)
Computing Gaussian/Laplacian Pyramid
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original from the laplacian pyramid?
![Page 23: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/23.jpg)
Hybrid Image
Slide: Hoiem
![Page 24: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/24.jpg)
Hybrid Image in Laplacian Pyramid
High frequency Low frequency
Slide: Hoiem
![Page 25: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/25.jpg)
Image representation
• Pixels: great for spatial resolution, poor access to frequency
• Fourier transform: great for frequency, not for spatial info
• Pyramids/filter banks: balance between spatial and frequency information
Slide: Hoiem
![Page 26: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/26.jpg)
Major uses of image pyramids
• Compression
• Object detection– Scale search– Features
• Detecting stable interest points
• Registration– Course-to-fine
Slide: Hoiem
![Page 27: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/27.jpg)
Denoising
Additive Gaussian Noise
Gaussian Filter
Slide: Hoiem
![Page 28: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/28.jpg)
Smoothing with larger standard deviations suppresses noise, but also blurs the image
Reducing Gaussian noise
Source: S. Lazebnik
![Page 29: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/29.jpg)
Reducing salt-and-pepper noise by Gaussian smoothing
3x3 5x5 7x7
![Page 30: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/30.jpg)
Alternative idea: Median filtering• A median filter operates over a window by
selecting the median intensity in the window
• Is median filtering linear?Source: K. Grauman
![Page 31: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/31.jpg)
Median filter• What advantage does median filtering have
over Gaussian filtering?– Robustness to outliers
Source: K. Grauman
![Page 32: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/32.jpg)
Median filterSalt-and-pepper noise Median filtered
Source: M. Hebert
• MATLAB: medfilt2(image, [h w])
![Page 33: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/33.jpg)
Median vs. Gaussian filtering3x3 5x5 7x7
Gaussian
Median
![Page 34: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/34.jpg)
Other non-linear filters• Weighted median (pixels further from center count less)
• Clipped mean (average, ignoring few brightest and darkest pixels)
• Bilateral filtering (weight by spatial distance and intensity difference)
http://vision.ai.uiuc.edu/?p=1455Image:
Bilateral filtering
![Page 35: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/35.jpg)
Review of last three days
![Page 36: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/36.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Credit: S. Seitz
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Review: Image filtering111
111
111
],[g
![Page 37: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/37.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering111
111
111
],[g
Credit: S. Seitz
![Page 38: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/38.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering111
111
111
],[g
Credit: S. Seitz
![Page 39: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/39.jpg)
Filtering in spatial domain-101
-202
-101
* =
![Page 40: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/40.jpg)
Filtering in frequency domain
FFT
FFT
Inverse FFT
=
Slide: Hoiem
![Page 41: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/41.jpg)
Review of Last 3 Days• Linear filters for basic processing
– Edge filter (high-pass)– Gaussian filter (low-pass)
FFT of Gaussian
[-1 1]
FFT of Gradient Filter
Gaussian
Slide: Hoiem
![Page 42: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/42.jpg)
Review of Last 3 Days• Derivative of Gaussian
Slide: Hoiem
![Page 43: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/43.jpg)
Review of Last 3 Days• Applications of filters
– Template matching (SSD or Normxcorr2)• SSD can be done with linear filters, is sensitive to
overall intensity– Gaussian pyramid
• Coarse-to-fine search, multi-scale detection– Laplacian pyramid
• More compact image representation• Can be used for compositing in graphics
– Downsampling• Need to sufficiently low-pass before downsampling
![Page 44: Templates, Image Pyramids, and Filter Banks Computer Vision James Hays, Brown 09/19/11 Slides: Hoiem and others](https://reader035.vdocuments.mx/reader035/viewer/2022062713/56649f535503460f94c7713d/html5/thumbnails/44.jpg)
Next Lectures• Machine Learning Crash Course