similarity measures spring 2009 ben-gurion university of the negev

Similarity Measures

Spring 2009

Ben-Gurion University of the Negev

Sensor Fusion Spring 2009

Instructor

• Dr. H. B Mitchell email: [email protected]


Similarity Measures

Fundamental importance in all fusion Conceptually consists of two parts: Transformation T. This extracts characteristics from input image and

represents it as a feature vector Distance D. This quantifies similarity of feature vectors. Symbolically:


Similarity Measures. Example

Content Based Information Retrieval (CBIR) Input: Database containing many images Requirement: Retrieve images which are similar to test image Transformation T: Represent each image in database as multi-

dimensional feature vector extracted using low-level descriptors: co-occurrence matrix, color histogram, wavelets, moments etc

Distance D: Euclidean, Manhattan, Minkowski, Hausdorff distances


Similarity Measure. Shape Context

In matching shapes introduced the shape context for each point on the contour C:

which is a two-dimensional histogram We measure the similarity between two shapes with shapes contexts and using the test.


Similarity Measures Metric

A similarity measure is a metric if it obeys following:

Experiments show that valid similarity measures generally obey the first three requirements but not the triangle inequality.

Similarity measures therefore not metric


Similarity Measures: Global vs Local

Can classify similarity measures as global or local: Global measures act on the entire images and return a single scalar

value Local measures act on small patches. They return a local similarity

map. We can convert global measures into local measures and vice

versa.



Example. Global similarity measure can be used as a local measure:

(m,n)

Local Window W(m,n)



Example. Local similarity measure can be used as a global measure:

Then we aggregate the local similarity map to obtain a global measure e.g.

(m,n)

Local Window W(m,n)


Global Measures Without Spatial Alignment

These similarity measures are very robust but tend to have low discriminative power.

Examples are the probabilistic measures: Chernoff Bhattacharyya Jeffrey’s-Matusita Kullback-Leibler Symmetric Kullback-Leibler

The measures may be applied locally. In this case typical window size is 20x20. This is needed to ensure we have sufficient pixels to calculate the probability densities p(x) and q(x).


Distance.

If we represent the distributions p(a) and q(b) as discrete histograms we may use the test. This test assumes common bins.

If two images A and B have histograms

Then test is

Problems. (1) Optimal value for K. Rule of thumb says no bin should contain less than 5 samples. (2) No cross-bin correlations allowed


Distance. Example

The test is simple and widely used similarity measure. In shape matching we have two closed contours c and C. Each point on c is characterized by a 2D histogram

and each point on C is characterized by If is associated with the point , then the match between the

two histograms is

We may define the overall similarity as

We may also impose an order constraint on : If then we require


Optimal Equal-Sized Histogram Bins.

A recent suggestion is the following empirical formula for the optimum number of equal-sized bins in interval [0,1]:

where H(l) is number of samples which fall in kth bin N is the total number of samples k is the number of equal spaced bins


Earth Mover’s Distance.

Earth Movers Distance allows cross-bin associations Given two histograms and Treat P as “supplies” where = amount of kth supply Treat Q as “demands” where = amount of lth demand EMD is defined as minimum normalized work required to transform

P into Q: where is the amount transferred from to . is work involved in transferring one unit from to

Project: Circular Earth Mover’s Distance Project: Pele-Werman EMD variant (A linear time histogram materic

for improved SIFT matching. Pele and Werman ECCV 2008)


Earth Mover’s Distance

If the histograms P and Q are normalized then EMD is Mallows distance. If the histograms each have N bins then

where the min is taken over all permutations of In one dimension this reduces to

Example.

Not normalized. Each box is a unit. EMD=0

Normalized. Each box is probability=0.25. For p=1, EMD=0.5


Earth Mover’s Distance

Recent development in EMD is A linear time histogram metric for improved SIFT matching by Pele and Werman ECCV 2008 Project


Color-based Image Retrieval

Jeffrey divergence

Quadratic form distance

Earth Mover Distance

χ2 statistics

L1 distance


Global Measure With Spatial Alignment

Tend to have higher discriminative power but require accurate registration.

The measures tend to give lower similarity values as mis-registration increases. They are therefore used in spatial alignment algorithms.

Simple measures are mse mae Correlation coefficient

Mutual Information


Ordinal Measures

The mse, mae, correlation coefficient are sensitive to illumination changes. Mutual information and other ordinal measures are designed to overcome this sensitivity by using ordered gray-levels.

Ordinal measures are insensitive to illumination changes if order of the gray-levels is maintained.

Two classical measures are Spearman Kendall

Experiments on stereo matching showed Kendall measure gave best results from all ordinal measures.


Ordinal Measures

Kemeny-Snell distance comapres the realtive ranking of each ordered pair of locations in one image with its relative ranking in the second image. Smaller values of d indicate more agreement.

Luo et al. suggests the measure is very powerful. However computational complexity is very high.


Local Ordinal Measures. Bhat-Nayar

We may apply the Spearman, Kendall and Kemeny-Snell distances locally. However recommendation is the local windows measure at least 20x20.

Bhat-Nayar is a powerful specially designed local ordinal operator. The window size in Bhat-Nayar is from 3x3 to 13x13.

Bhat-Nayar uses a local window containing K gray-levels in A and B with rank vectors

Bhat-Nayar creates a composite rank vector by ranking wrt where and


Local Ordinal Measures. Bhat-Nayar Bhat-Nayar creates a composite rank vector by ranking wrt where and Example. A: [10 20 30 50 40 70 60 90 80] B: [90 60 70 50 40 80 10 30 20] rA: [1 2 3 5 4 7 6 9 8] rB: [9 6 7 5 4 8 1 3 2] s : [9 6 7 4 5 1 8 2 3] Consider k=6. . Therefore Thus . Therefore


Local Ordinal Measures. Bhat-Nayar

Bhat-Nayar assumes all ranks are unique. How can we handle ties? Project: Use fuzzy ranks in the Bhat-Nayar scheme.


Ordinal Measures. Mittal-Ramesh

The ordinal similarity measures (spearman, kendall, Kemeny-Snell and Bhat-Nayar) are robust to changes in illumination but not to Gaussian noise.

Small amounts of Gaussian noise can completely change the rankings between pixels that are not far from each other in gray-level.

Mittal-Ramesh measure is an ordinal measure which also takes into accunt the pixel gray-levels. In general this should give best results. However the Mittal-Ramesh operator has very high computational complexity.


Example

Comparative results from four algorithms: Normalized Cross-Correlation

Census Algorithm Bhat-Nayar Algorithm

Mittal-Ramesh Algorithm


Binary Image Measures

Special measures are used for binary images Following is a local binary measure. Given two binary images A and B the corresponding distance transforms are:

where

Local distance map is

0 0 1 0 0 1 1 0 0

Distance transform


Binary Image Measures

May convert the local similarity map L(m,n) into a global measure. This is the Hausdorff distance:

Hausdorff distance is very sensitive to noise. Robust alternatives are:

similarity measures spring 2009 ben-gurion university of the negev

Documents