image processing - pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · gabor filters gabor...
TRANSCRIPT
![Page 1: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/1.jpg)
Image Processing
David Kauchak cs160
Fall 2009
Empirical Evaluation of Dissimilarity Measures for Color and Texture
Jan Puzicha, Joachim M. Buhmann, Yossi Rubner & Carlo Tomasi
![Page 2: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/2.jpg)
Administrative
11/4 class project discussion – project proposal draft due
11/5 4:15pm Rose Hill Theatre CS Lunch today
![Page 3: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/3.jpg)
Image processing
Image processing – http://webcast.berkeley.edu/course_details.php?seriesid=1906978282
Computer vision – http://cseweb.ucsd.edu/classes/sp09/cse252b/
CVPR – http://www.cvpr2009.org/full-program
![Page 4: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/4.jpg)
Text retrieval
What was the key problem we needed to solve for text retrieval?
sim( ) = ? , query document
![Page 5: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/5.jpg)
The Problem: Image Similarity
sim( ) = ? ,
![Page 6: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/6.jpg)
Where does this problem arise in computer vision?
Image Classification Image Retrieval Image Segmentation
![Page 7: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/7.jpg)
Classification
? ?
![Page 8: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/8.jpg)
Retrieval
Jeremy S. De Bonet, Paul Viola (1997). Structure Driven Image Database Retrieval. Neural Information Processing 10 (1997).
![Page 9: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/9.jpg)
Segmentation
http://vizlab.rutgers.edu/~comanici/segm_images.html
![Page 10: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/10.jpg)
How is an image represented?
![Page 11: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/11.jpg)
How is an image represented?
• images are made up of pixels • for a color image, each pixel corresponds to an RGB value (i.e. three numbers)
![Page 12: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/12.jpg)
Image file formats
BitMaP JPEG TIFF Gif Png …
![Page 13: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/13.jpg)
Bitmap
R, G, B
![Page 14: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/14.jpg)
JPEG Compression Process
![Page 15: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/15.jpg)
JPEG Compression Process
![Page 16: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/16.jpg)
JPEG Compression Process
![Page 17: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/17.jpg)
JPEG Compression Process
Quantizer: Weights the various spectral coefficients according to their importance, with respect to the human visual system.
![Page 18: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/18.jpg)
JPEG Compression
![Page 19: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/19.jpg)
Image features
![Page 20: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/20.jpg)
Color
Which is more similar?
L*a*b* was designed to be uniform in that perceptual “closeness” corresponds to Euclidean distance in the space.
![Page 21: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/21.jpg)
L*a*b*
L – lightness (white to black)
a – red-greeness
b – yellowness-blueness
![Page 22: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/22.jpg)
L*a*b*
![Page 23: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/23.jpg)
Texture
How is texture different than color?
![Page 24: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/24.jpg)
Texture
Texture is not pointwise like color
Texture involves a local neighborhood
How can we capture texture? How did we capture audio texture?
![Page 25: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/25.jpg)
Gabor Filters
Gabor filters are Gaussians modulated by sinusoids
They can be tuned in both the scale (size) and the orientation
A filter is applied to a region and is characterized by some feature of the energy distribution (often mean and standard deviation)
Similar idea to wavelets (Gabor wavelet)!
![Page 26: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/26.jpg)
Examples of Gabor Filters
Scale: 4 at 108° Scale: 5 at 144° Scale: 3 at 72°
![Page 27: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/27.jpg)
Gabor filters
What would the response look like to a vertical filter?
![Page 28: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/28.jpg)
Gabor filters
![Page 29: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/29.jpg)
Features
any problem?
For each pixel: • set of color features • set of texture features (i.e. responses to different filters) • …
![Page 30: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/30.jpg)
Features
For each pixel: • set of color features • set of texture features (i.e. responses to different filters) • …
• Lots of features! • Extremely sparse • Features are position dependent
Ideas?
![Page 31: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/31.jpg)
One approach: histograms
Examine the distribution of features, rather than the features themselves
General purpose (i.e. any distribution of features)
Resilient to variations (shadowing, changes in illumination, shading, etc.)
Can use previous work in statistics, etc.
![Page 32: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/32.jpg)
Histogram Example
![Page 33: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/33.jpg)
Cumulative Histogram
Normal Histogram
Cumulative Histogram
![Page 34: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/34.jpg)
Similarity Measures Using the Histograms
Histogram 1 Histogram 2
Need to quantify how similar two histograms are
![Page 35: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/35.jpg)
Heuristic Histogram Distances
Minkowski-form distance Lp
Special cases: – L1: absolute, cityblock, or
Manhattan distance – L2: Euclidian distance – L∞: Maximum value distance
€
D(I,J) = Ii − Jip
i∑
1/ p
![Page 36: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/36.jpg)
More heuristic distances
Weighted-Mean-Variance (WMV)
– Only includes minimal information about distribution
![Page 37: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/37.jpg)
Cumulative Difference Example
Histogram 1 Histogram 2 Difference
- =
K-S = CvM =
![Page 38: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/38.jpg)
How would you test the perfomance of these algorithms?
Three tasks – classification – retrieval – segmentation
![Page 39: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/39.jpg)
Data Set: Color
Randomly chose 94 images from set of 2000 – 94 images represent separate classes
Randomly select disjoint set of pixels from the images – Set size of 4, 8, 16, 32, 64 pixels – 16 disjoint samples per set per image
![Page 40: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/40.jpg)
Data Set: Texture
Brodatz album – Collection of wide range of texture (e.g. cork, lawn,
straw, pebbles, sand, etc.) Each image is considered a class (as in color) Extract sets of 16 non-overlapping blocks
– sizes 8x8, 16x16,…, 256x256
![Page 41: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/41.jpg)
Setup: Classification
How can we use similarity for classification? k-Nearest Neighbor classifier is used
– Nearest Neighbor classification: given a collection of labeled points S and a query point q, what point belonging to S is closest to q?
– k nearest is a majority vote of the k closest points
![Page 42: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/42.jpg)
Results: Classification, color data set
sample size
![Page 43: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/43.jpg)
Results: Classification, texture data set
sample size
![Page 44: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/44.jpg)
Results: Image Retrieval
![Page 45: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/45.jpg)
Setup: Segmentation
100 images Each image consists of 5 different textures
![Page 46: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/46.jpg)
Setup: Segmentation
How can we solve this problem using our similarity measures?
![Page 47: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/47.jpg)
Setup: Segmentation (cont.)
Image is divided into 16384 sites (128 x 128 grid)
A histogram is calculate for each site Each site histogram is then compared with 80
randomly selected sites Image sites with high average similarity are
then grouped
![Page 48: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/48.jpg)
Results: Segmentation
![Page 49: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/49.jpg)
Something fun…
http://www.popsci.com/gear-amp-gadgets/article/2009-09/building-virtual-cities-automatically-150000-flickr-photos
![Page 50: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/50.jpg)
Cyberchondria
![Page 51: Image Processing - Pomonadkauchak/classes/f09/cs160-f09/... · 2009. 11. 2. · Gabor Filters Gabor filters are Gaussians modulated by sinusoids They can be tuned in both the scale](https://reader035.vdocuments.mx/reader035/viewer/2022070213/610bc1722040405e091cb4b0/html5/thumbnails/51.jpg)
Homework for next time…
Spend 15 minutes playing with three different image retrieval systems – http://en.wikipedia.org/wiki/Image_retrieval has a
number – What works well? – What doesn’t work well? – Anything interesting you noticed?
You won’t hand anything in, but we’ll start class on Monday with a discussion of the systems