On Fast Non-Metric Similarity Search by Metric Access Methods

Tomáš Skopal (tomas@skopal.net)

Charles University in PragueFaculty of Mathematics and Physics

Department of Software Engineering Prague, Czech Republic

EDBT 2006, Munich 2

Presentation Outline

introduction motivation of non-metric similarity search metric access methods, intrinsic dimensionality

our objective: fast non-metric search turning non-metric into metric the TriGen algorithm experimental results conclusions and future work

EDBT 2006, Munich 3

Similarity Search in Multimedia Databases non-structured data instances

multimedia objects, texts, sequences, time series, etc. distance function d: U U R

d(O1,O2) interpreted as a dissimilarity score of two objects metric properties (Oi, Oj, Ok U)

reflexivity d(Oi, Oj) = 0 Oi = Oj

positivity d(Oi, Oj) > 0 Oi Oj

symmetry d(Oi, Oj) = d(Oj, Oi) triangular inequality d(Oi, Oj) + d(Oj, Ok) d(Oi, Ok)

triangular triplet (a,b,c) = a + b c & a + c b & b + c a when triangular inequality satisfied by d, then for every O1,O2,O3

U

(d(O1,O2), d(O2,O3), d(O1,O3)) is a triangular triplet

semi-metricmetric

EDBT 2006, Munich 4

Metric Access Methods

given a metric d and a dataset S U, metric access methods

(MAMs) can be used to organize objects of S Reason: fast query processing (range & k-nearest neighbor queries) Principle of MAMs: structured decomposition of objects into equivalence

classes, such that only some “candidate” classes have to be searched

when querying the filtering of non-relevant classes is possible due to the metric properties

(esp. triangular inequality) Examples: M-tree, PM-tree, D-index,

gh-tree, vp-tree, LAESA, etc.

EDBT 2006, Munich 5

Metric Access Methods, cont.

intrinsic dimensionality definition (as proposed in [4]) : (S,d) = 2 / 22

( is mean and 2 is variance of distance distribution

in S) indicates how effeciently (quickly) could be a dataset S queried

using a metric d low (e.g. below 10) means the dataset is well-structured

– i.e. there exist tight clusters of objects high means the dataset is poorly structured – i.e. objects are almost

equaly distant in consequence, intrinsically high-dimensional datasets are hard to

organize, so that querying becomes inefficient (sequential scan) example: an M-tree hierarchy built

on a high-dimensional dataset

low intrinsic dimensionality high intrinsic dimensionality

EDBT 2006, Munich 6

Metric vs. non-metric measures

non-metric measures are often robust(resistant to outliers, errors in objects, etc.) the symmetry and mainly the triangular inequality

are often violated

cannot be directly used with MAMs

a

b

a > b + c

c

ab

a ≠ b

EDBT 2006, Munich 7

Examples of Non-metric measures

various k-median distances measure distance between the two (k-th) most

similar portions in objects

COSIMIR back-propagation network with single output

neuron serving as a distance, allows training

Dynamic Time Warping distance sequence alignment technique minimizes the sum of distances between

sequence elements

fractional Lp distances generalization of Minkowski distances (p<1) more robust to extreme differences in coordinates

EDBT 2006, Munich 8

Turning Non-metric into Metric

the reflexivity & positivity by setting a minimum distance lowerbound d- < 0,

i.e. O1≠ O2 drp(O1, O2) = d(O1, O2) + |d-| + some small value, otherwise drp(O1, O2) = 0

the symmetry e.g. ds(O1, O2) = min(d(O1, O2), d(O2, O1)) query is processed using ds,

and the query result is re-filtered using d

how to satisfy the triangular inequality ? we apply a modifying function f on d, making

semi-metric a metric

EDBT 2006, Munich 9

SP-modifiersLet f be a function f: R R, such that f(0) = 0 and f is increasing

(i.e. f(x) > f(y) x > y).

For similarity search purposes f(d(,)) – further denoted as df – can be safely used instead of just d. (In case of range query (Q, rQ) the query

radius rQ is modified to f(rQ).)

Proof: All similarity orderings are preserved.1) Consider the set of all pairs of objects from U.2) Create ordering of the pairs with respect

to distances of the two objects in the pair.3) The ordering does not change after

the application of any f on the distances, because f is increasing.

We call such function f as similarity-preserving modifier (simply SP-modifier.)

f1()f2()

f3()

f4()

EDBT 2006, Munich 10

We want to find such SP-modifier, that forces d to satisfy the triangular inequality: any concave SP-modifier f is metric-preserving (proof in [3])

when applied on any metric d(,), df is metric as well when applied on a triangular triplet (a,b,c), (f(a),f(b),f(c)) is triangular triplet as well

any concave SP-modifier is triangle-generating (TG-modifier) when applied on all possible triplets,

some of them become triangular

(theory of concave functions) the more concave f, the more triplets

become triangular once a triplet becomes triangular,

after application of any other TG-modifier

it remains triangular

Theorem: Every semi-metric can be turned by a single TG-modifier into a metric.

TG-modifiers

EDBT 2006, Munich 11

Proof: Incremental Triplet Stretching

c

a

b

a > b + c

f1(a)f1(b)

f1(c)

f1(a) > f1(b) + f1(c)(low-concave f1)

f2(a) f2(b)

f2(c)

f2(a) ≤ f2(b) + f2(c)(high-concave f2)

f3(a) f3(b) f3(c)

(extremely high-concave f3)

f3(a) f3(b)

f3(c)

We repeatedly apply TG-modifiers on all triangular triplets (generated by d(,) on S), starting with a less concave TG-modifier, proceeding with more concave ones.

We continue with applying more and more concave TG-modifiers (e.g. by nesting them) until we turn all the triplets into triangular ones.

EDBT 2006, Munich 12

Optimal TG-modifier

There exist infinitely many TG-modifiers that turn a given semi-metric into a metric. However, not all are suitable for fast similarity search.

The optimal TG-modifier should: turn every non-triangular triplet generated by d (considering the

objects from S) into a triangular one (i.e. enforce the triangular inequality)

keep the intrinsic dimensionality of S with respect to df as low as possible

Example:This TG-modifier turns every semi-metric into metric, but is useless for searching by MAMs.

All classes maintained by a MAM overlap every query, so the search deteriorates to sequential scan.

EDBT 2006, Munich 13

Scaling the concavity

How to find an optimal TG-modifier for a given d (and S)?

We make use of some predefined TG-bases: TG-base is an extended TG-modifier such that it uses

a concavity weight w 0 as second parameter, i.e. f: R R R for w = 0, the TG-base turns into identity, i.e. f(x,0) = x with increasing w, the TG-modifier f(x,w) becomes more concave

the greater w (thus more concave f), the more triplets become triangular the higher the intrinsic dimensionality is

we can relax the strict condition of needing all triplets to become triangular by introducing a TG-error tolerance (a ratio of triangular triplets to non-triangular triplets) to be satisfied a choice of exact or approximate search ( = 0 or > 0)

EDBT 2006, Munich 14

Proposed TG-bases general-purpose TG-bases

Fractional Power TG-base (FP-base) Rational Bezier Quadric TG-bases (RBQ-bases)

each such TG-base is additionally provided by the second Bezier point (a,b) choosing different (a,b) allows to predefine the “place of maximum concavity” in the TG-base

we need to find an optimal w for a TG-base f, such that df becomes metric, but w is as low as possible – the TriGen algorithm

EDBT 2006, Munich 15

The TriGen algorithmThe algorithm finds a TG-modifier (formed by a TG-base and the appropriate concavity weight w), which turns a given semi-metric d into an (approximated) metric, while the intrinsic dimensionality is kept as low as possible.

The algorithm makes use of halving the concavity interval, when searching for the optimal concavity weight.

EDBT 2006, Munich 16

Experimental Results

The testbed: two dataset (1 real – images (histograms), 1 synthetic - polygons) 10 non-metric measures (6 for images, 4 for polygons)

TriGen was used to create the modification of a semi-metric into metric 2 MAMs – M-tree and PM-tree

Testing of:1) intrinsic dimensionalities of the datasets (with respect to df, where f is the

TG-modifier found by TriGen)

2) performance k-NN queries – performance, retrieval error (when the TG-error tolerance > 0)

EDBT 2006, Munich 17

Experiments – intrinsic dimensionalities

EDBT 2006, Munich 18

Experiments – k-NN queries

EDBT 2006, Munich 19

Experiments – k-NN queries

EDBT 2006, Munich 20

Conclusions and Future Work

We have presented: a way of fast searching in non-metric datasets

by metric access methods in particular, the Trigen algorithm for turning any

semi-metric into a metric future work:

a generalized framework for fast exact and approximate similarity search (either metric or non-metric) – a combination with previous work [2]

EDBT 2006, Munich 21

References

[1] T. Skopal, J. Pokorný, V. Snášel Nearest Neighbours Search using the PM-tree. In DASFAA 2005, Beijing, China, pages 803–815. LNCS 3453, Springer.

[2] T. Skopal, P. Moravec, Jaroslav Pokorný, V. SnášelMetric Indexing for the Vector Model in Text Retrieval, In SPIRE 2004, Padova, Italy, pages 183-195, LNCS 3246, Springer.

[3] P. CorazzaIntroduction to metric-preserving functions, American Mathematical Monthly 104(4), 1999.

[4] E. Chávez, G. NavarroA Probabilistic Spell for the Curse of Dimensionality, In ALENEX 2001, LNCS 2153, Springer.