Download - אשכול בעזרת אלגורתמים בתורת הגרפים
אשכול בעזרת אלגורתמים בתורת
הגרפיםרועי יצחק
Elements of Graph TheoryA graph G = (V,E) consists of a vertex set V and
an edge set E. If G is a directed graph, each edge is an ordered
pair of verticesA bipartite graph is one in which the vertices can
be divided into two groups, so that all edges join vertices in different groups.
העברת הנק' לגרףקבע את הנק' במישורקבע מרחק בין כל זוג נקודות
0 1 1.5 2 5 6 7 9
1 0 2 1 6.5 6 8 8
1.5 2 0 1 4 4 6 5.5
.
.
.
n-D data pointsgraph
representationdistance matrix
Minimal Cut Create a graph G(V,E) from the data Compute all the distances in the graph The weight of each edge is the distance Remove edges from the graph according to
some threshold
0.1
0.2
0.8
0.70.6
0.8
0.8
0.8
E={Wij} Set of weighted edges indicating pair-wise similarity between points
Similarity Graph Distance decrease similarty increase Represent dataset as a weighted graph G(V,E)
1 2 6{ , ,..., }v v v
1
2
3
4
5
6
V={xi} Set of n vertices representing data points
Similarity Graph Wij represent similarity between vertex If Wij=0 where isn’t similarity Wii=0
Graph Partitioning Clustering can be viewed as
partitioning a similarity graph
Bi-partitioning task: Divide vertices into two disjoint
groups (A,B) 1
23
4
5
6
A B
V=A U B
Clustering Objectives Traditional definition of a “good” clustering:
1. Points assigned to same cluster should be highly similar.2. Points assigned to different clusters should be highly dissimilar.
Minimize weight of between-group connections
0.1
0.2
0.8
0.70.6
0.8
0.8
0.81
2
3
4
5
6
Apply these objectives to our graph representation
Graph Cuts Express partitioning objectives as a function of the
“edge cut” of the partition. Cut: Set of edges with only one vertex in a group.we
wants to find the minimal cut beetween groups. The groups that has the minimal cut would be the partition
BjAi
ijwBAcut,
),(0.1
0.2
0.8
0.7
0.6
0.8
0.8
1
2
3
4
5
6
0.8
A B
cut(A,B) = 0.3
Graph Cut Criteria Criterion: Minimum-cut
Minimise weight of connections between groupsmin cut(A,B)
Optimal cutMinimum cut
Problem: Only considers external cluster connections Does not consider internal cluster density
Degenerate case:
Graph Cut Criteria (continued)
Criterion: Normalised-cut (Shi & Malik,’97) Consider the connectivity between groups
relative to the density of each group.
)(),(
)(),(),(min
BvolBAcut
AvolBAcutBANcut
Normalise the association between groups by volume. Vol(A): The total weight of the edges originating from
group A. Why use this criterion?
Minimising the normalised cut is equivalent to maximising normalised association. Produces more balanced partitions.
(MST-Prim)עץ פורש מינימאלי קבע קודקוד מקור והכנס אותו לסטA)עץ( מצא את הקשת הקלה ביותר אשר המחברת בין הסטB שאר הקודקודים(
(A)בגרף( לעץ חזור על התהליך עד שלא ישארו קודקודים בסטB
clustringמציאת קבע את כיוון ההתקדמות בעץ )כל הוספה של צומת( בפונקציה של משקל
הקשת שהוספה בגרף מייצג "עמק"כל cluster
Example
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
4
8
4
8
8
Example
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
4
8
8
7
4
2
Example
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
4 7
8 4
8
2
7 64 7
7 6 4
8
2
2
10
Example
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
4 7
10
7 2 4
8
2
14 7
10
1 2 4
8
2
Example
a
b c d
e
h g f
i
4
8 7
8
11
1 2
7
2
4 14
9
106
4 7
10
1 2 4
8
2 9
Similarity Graph Wij represent similarity between vertex If Wij=0 where isn’t similarity Wii=0
Example – 2 Spirals
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
Dataset exhibits complex Dataset exhibits complex cluster shapescluster shapes K-means performs very K-means performs very poorly in this space due poorly in this space due bias toward dense bias toward dense spherical clusters.spherical clusters.
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
-0.709 -0.7085 -0.708 -0.7075 -0.707 -0.7065 -0.706In the embedded space In the embedded space given by two leading given by two leading eigenvectors, clusters eigenvectors, clusters are trivial to separate.are trivial to separate.
Spectral Graph Theory Possible approach
Represent a similarity graph as a matrix Apply knowledge from Linear Algebra…
Spectral Graph Theory Analyse the “spectrum” of matrix representing a graph. Spectrum : The eigenvectors of a graph, ordered by the
magnitude(strength) of their corresponding eigenvalues.
},...,,{ 21 n
The eigenvalues and eigenvectors of a matrix provide global information about its structure.
11 1 1 1
1
n
n nn n n
w w x xλ
w w x x
Matrix Representations Adjacency matrix (A)
n x n matrix : edge weight between vertex xi and xj
x1 x2 x3 x4 x5 x6
x1 0 0.8 0.6 0 0.1 0
x2 0.8 0 0.8 0 0 0
x3 0.6 0.8 0 0.2 0 0
x4 0 0 0.2 0 0.8 0.7
x5 0.1 0 0 0.8 0 0.8
x6 0 0 0 0.7 0.8 0
0.1
0.2
0.8
0.7
0.60.8
0.8
1
2
3
4
5
6
0.8
Important properties: Symmetric matrix Eigenvalues are real Eigenvector could span orthogonal base
][ ijwA
Matrix Representations (continued)
Important application: Normalise adjacency matrix
Degree matrix (D) n x n diagonal matrix : total weight of edges incident to vertex xi
x1 x2 x3 x4 x5 x6
x1 1.5 0 0 0 0 0
x2 0 1.6 0 0 0 0
x3 0 0 1.6 0 0 0
x4 0 0 0 1.7 0 0
x5 0 0 0 0 1.7 0
x6 0 0 0 0 0 1.5
0.1
0.2
0.8
0.7
0.60.8
0.8
1
2
3
4
5
6
0.8
j
ijwiiD ),(
Matrix Representations (continued)
Laplacian matrix (L) n x n symmetric matrix
Important properties: Eigenvalues are non-negative real numbers Eigenvectors are real and orthogonal Eigenvalues and eigenvectors provide an insight into the
connectivity of the graph…
0.1
0.2
0.8
0.7
0.60.8
0.8
1
2
3
4
5
6
0.8
L = D - Ax1 x2 x3 x4 x5 x6
x1 1.5 -0.8 -0.6 0 -0.1 0
x2 -0.8 1.6 -0.8 0 0 0
x3 -0.6 -0.8 1.6 -0.2 0 0
x4 0 0 -0.2 1.7 -0.8 -0.7
x5 -0.1 0 0 0.8- 1.7 -0.8
x6 0 0 0 -0.7 -0.8 1.5
Another option – normalized laplasian Laplacian matrix (L)
n x n symmetric matrix1.00 -0.52 -0.39 0.00 -0.06 0.00
-0.52 1.00 -0.50 0.00 0.00 0.00
-0.39 -0.50 1.00 -0.12 0.00 0.00
0.00 0.00 -0.12 1.00 -0.47 -0.44
-0.06 0.00 0.00 0.47- 1.00 -0.50
0.00 0.00 0.00 -0.44 -0.50 1.00 Important properties: Eigenvectors are real and normalize Each Aij which i,j is not equal =
0.1
0.2
0.8
0.7
0.60.8
0.8
1
2
3
4
5
6
0.8
0.5 0.5( )D D A D
ijADii
Spectral Clustering Algorithms Three basic stages:
1. Pre-processing Construct a matrix representation of the dataset.
2. Decomposition Compute eigenvalues and eigenvectors of the matrix. Map each point to a lower-dimensional representation
based on one or more eigenvectors.
3. Grouping Assign points to two or more clusters, based on the
new representation.
Spectral Bi-partitioning Algorithm
1. Pre-processing Build Laplacian
matrix L of the graph
0.90.80.5-0.2-0.70.4
-0.2-0.6-0.8-0.4-0.70.4
-0.6-0.40.20.9-0.40.4
0.6-0.20.0-0.20.20.4
0.30.4-0.0.10.20.4
-0.9-0.20.40.10.20.4
3.0
2.5
2.3
2.2
0.4
0.0
ΛΛ = = XX = =
2. Decomposition Find eigenvalues X
and eigenvectors Λ of the matrix L
-0.7x6
-0.7x5
-0.4x4
0.2x3
0.2x2
0.2x1 Map vertices to corresponding components of λ2
x1 x2 x3 x4 x5 x6x1 1.5 -0.8 -0.6 0 -0.1 0
x2 -0.8 1.6 -0.8 0 0 0
x3 -0.6 -0.8 1.6 -0.2 0 0
x4 0 0 -0.2 1.7 -0.8 -0.7
x5 -0.1 0 0 -0.8 1.7 -0.8
x6 0 0 0 -0.7 -0.8 1.5
Spectral Bi-partitioning Algorithm
123456
0.41 -0.41 -0.65 -0.31 -0.38 0.11
0.41 -0.44 0.01 0.30 0.71 0.22
0.41 -0.37 0.64 0.04 -0.39 -0.37
0.41 0.37 0.34 -0.45 0.00 0.61
0.41 0.41 -0.17 -0.30 0.35 -0.65
0.41 0.45 -0.18 0.72 -0.29 0.09
The matrix which represents the eigenvector of the laplacian the eigenvector matched to the corresponded eigenvalues with increasing order
Spectral Bi-partitioning (continued) Grouping
Sort components of reduced 1-dimensional vector. Identify clusters by splitting the sorted vector in two.
How to choose a splitting point? Naïve approaches:
Split at 0, mean or median value More expensive approaches
Attempt to minimise normalised cut criterion in 1-dimension
-0.7x6
-0.7x5
-0.4x4
0.2x3
0.2x2
0.2x1 Split at 0Split at 0Cluster Cluster AA: Positive points: Positive points
Cluster Cluster BB: Negative : Negative pointspoints
0.2x3
0.2x2
0.2x1
-0.7x6
-0.7x5
-0.4x4
A B