non-metric affinity propagation for unsupervised image categorization
DESCRIPTION
Non-metric affinity propagation for unsupervised image categorization. Delbert Dueck and Brendan J. Frey ICCV 2007. Outline. Introduction Comparison of the NIPS (2006) and Science (2007) Algorithms Unsupervised Categorization of Olivetti Face Images - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/1.jpg)
Non-metric affinity propagation for unsupervised image categorization
Delbert Dueck and Brendan J. FreyICCV 2007
![Page 2: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/2.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 3: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/3.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 4: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/4.jpg)
Introduction
• Many vision tasks– produce as output a categorization of input features– require unsupervised categorization of input features
as a preprocessing step• A powerful approach to representing image
categories is to identify exemplars1. high-order statistics2. represented efficiently as pointers into the training
data
![Page 5: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/5.jpg)
Introduction
• N training cases 1, …, N• Denoting the index of the exemplar
representing training case by • similarity between training case and by • the fitness function is• An example of metric similarity• If training case is an exemplar
is not computed in the same way as
i ici k ( , )s i k
1( ) ( , )N
iiS c s i c
2( , ) || ||i ks i k x x
i( , )s i i ( , )s i k
![Page 6: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/6.jpg)
Introduction
• Maximizing w.r.t , subject to the constraints that for all
• This problem is NP-hard [12]
• In [10], the input similarities need not be metric (i.e., need not be symmetric or satisfy the triangle inequality)
( )S c ci
[12] M. Charikar, S. Guha, A. Tardos, D. B. Shmoys 2002. J. Computer and System Science 65, 129.[10] K. Toyama, A. Blake 2002. Probabilistic tracking with exemplars in a metric space. Int. J. of Computer Vision 48:1, 9-19.
![Page 7: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/7.jpg)
Introduction
• Affinity propagation algorithm [15]
– clustering face images using Euclidean distance– finding genes using microarray data– airline routing
• Using non-metric measures of similarity?• Compare [15]&[18]
[15] B. J. Frey, D. Dueck 2007. Clustering by passing messages between data points. Science 315, 972-976.[18] B. J. Frey, D. Dueck 2006. Mixture modelling by affinity propagation. In Advances in Neural Information Processing Systems 18, MIT Press.
![Page 8: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/8.jpg)
Introduction
• Affinity propagation– “responsibility” r(i,k), sent from data point i to
candidate exemplar point k, reflects the accumulated evidence for how well-suited point k is to serve as the exemplar for point i
– “availability” a(i,k), sent from candidate exemplar point k to point i, reflects the accumulated evidence for how appropriate it would be for point i to choose point k as its exemplar
![Page 9: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/9.jpg)
Introduction
![Page 10: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/10.jpg)
Introduction
![Page 11: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/11.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 12: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/12.jpg)
Comparison of the NIPS (2006) and Science (2007) Algorithms
• NIPS (2006) disallows singleton clusters• To compare the two algorithms– clustering patches taken from an image [18]
– a tiling of 24 × 24 non-overlapping patches– translation-invariant similarities were computed
by comparing smaller 16 × 16 windows– similarity measure: the lowest squared error
between windows (over all possible translations)
![Page 13: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/13.jpg)
Comparison of the NIPS (2006) and Science (2007) Algorithms
• 100,000 k-centers clustering for each K
![Page 14: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/14.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 15: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/15.jpg)
Unsupervised Categorization of Olivetti Face Images
• Olivetti face database– ten 64 × 64 grey-scale images of each of 40 individuals– extracted a centered 50 × 50 region– normalized the pixel intensities
• To examine the effect of a wider range in image variation for each individual– extracting the images of 10 individuals, applying 3 in-
plane rotations and 3 scalings, producing a data set of 900 images
• equal to the same common value( , )s i i
![Page 16: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/16.jpg)
Unsupervised Categorization of Olivetti Face Images
• Performance on squared error• Performance on unsupervised image
classification• Performance using non-metric similarities
![Page 17: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/17.jpg)
Performance on squared error
• Using the 900 images including rotations and scales
• similarity between image and image to the negative of the sum of squared pixel differences
• 10,000 runs of k-centers clustering• defined the baseline error to be the 1st
percentile of error found by the 10,000 runs of k-centers clustering
i k
![Page 18: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/18.jpg)
Performance on squared error
• Compare with1. the best of one million runs of k-centers clustering2. k-centers clustering initialized by placing centers
uniformly along the first principal component of the data
3. the best quantized output of 10 runs of the EM algorithm applied to isotropic mixtures of Gaussians
4. hierarchical agglomerative clustering using the similarities to pick the best new exemplar at each agglomeration step
![Page 19: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/19.jpg)
Performance on squared error
![Page 20: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/20.jpg)
Performance on unsupervised image classification
• Two approaches to measuring the unsupervised classification error1. Each learned category is associated with the true
category that accounts for the largest number of training cases
2. ‘rate of true association’, pairs of images from the same true category‘rate of false association’, pairs of images from different true categories
![Page 21: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/21.jpg)
Performance on unsupervised image classification
![Page 22: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/22.jpg)
Performance using non-metric similarities
• When comparing two face images– Euclidean distance ignores the fact that certain
facial features may appear in different positions• Making the similarity non-metric can achieve
higher classification rates
![Page 23: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/23.jpg)
Performance using non-metric similarities
• Previous similarity:• non-metric similarity :
: window out of the center of the image : window of a fixed size
2( , ) || ||i ks i k x x
20( , ) min || ||i kT
s i k T x Tx
T0T
0T T
![Page 24: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/24.jpg)
Performance using non-metric similarities
![Page 25: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/25.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 26: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/26.jpg)
Unsupervised Categorization of Caltech101 Images Using SIFT Features
• Caltech101 image dataset• SIFT– For each local feature from the first image, the
nearest and second nearest features are found in the second image (Euclidean distance)
– If the distance ratio between the nearest and second-nearest neighbors is greater than 0.8, the match is considered significant
![Page 27: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/27.jpg)
Unsupervised Categorization of Caltech101 Images Using SIFT Features
• : the number of significant feature matches found comparing image with image
• • selected 20 of the 101 classes– faces, motorbikes, binocular, brain, camera,
garfield, pagoda, snoopy, stapler, stop sign, …– 1230 images
( , )m i ki k
1 1
1 1( , ) ( , ) ( , ) ( , )n n
j js i k m i k m i j m j k
n n
![Page 28: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/28.jpg)
An example of a category learned by affinity propagation
![Page 29: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/29.jpg)
Unsupervised Categorization of Caltech101 Images Using SIFT Features
![Page 30: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/30.jpg)
Outline
1. Introduction2. Comparison of the NIPS (2006) and Science
(2007) Algorithms3. Unsupervised Categorization of Olivetti Face
Images4. Unsupervised Categorization of Caltech101
Images Using SIFT Features5. Conclusions
![Page 31: Non-metric affinity propagation for unsupervised image categorization](https://reader035.vdocuments.mx/reader035/viewer/2022081505/568166a8550346895dda9bc5/html5/thumbnails/31.jpg)
Conclusions
• Affinity propagation can be used to achieve high classification rates
• Using non-metric similarity functions increases classification rates