15-853page1 separatosr in the real world practice slides combined from demmel/cs267_spr07...
TRANSCRIPT
15-853 Page1
Separatosr in the Real World Practice
Slides combined fromwww.cs.berkeley.edu/~demmel/cs267_Spr07www.cs.cmu.edu/~guyb/realworld/slidesF03/src/separators1.ppt
15-853 Page2
Edge Separators
7
8
3
4
01
2
5
6
An edge separator : a set of edges E’ ½ E which partitions V into V1 and V2
Criteria:|V1|, |V2| balanced
|E’| is small
V1
V2
E’
15-853 Page3
Vertex Separators
7
8
3
4
01
2
5
6
An vertex separator : a set of vertices V’ ½ V which partitions V into V1 and V2
Criteria:|V1|, |V2| balanced
|V’| is small
V1
V2
15-853 Page4
Other names
Sometimes referred to as – graph partitioning (probably more
common than “graph separators”)– graph bisectors– graph bifurcators– balanced graph cuts
Application
15-853 Page5
Telephone network design– Original application, algorithm due to Kernighan
Load Balancing while Minimizing CommunicationSparse Matrix times Vector Multiplication
– Solving PDEs– N = {1,…,n}, (j,k) in E if A(j,k) nonzero, – WN(j) = #nonzeros in row j, WE(j,k) = 1
VLSI Layout– N = {units on chip}, E = {wires}, WE(j,k) = wire
lengthSparse Gaussian Elimination
– Used to reorder rows and columns to increase parallelism, and to decrease “fill-in”
Data mining and clusteringPhysical Mapping of DNA
15-853 Page6
Recursive Separation
78
3
4
01
2
56
8
12
56
7
3
4
0
2
68
1
574
3
0
3 0 7 4
85
1 8 5 2 6
15-853 Page7
What graphs have small separators
Planar graphs: O(n1/2) vertex separators2d meshes, constant genus, excluded minors
Almost planar graphs:the internet, power networks, road networks
Circuitsneed to be laid out without too many crossings
Social network graphs:phone-call graphs, link structure of the web, citation graphs, “friends graphs”
3d-grids and meshes: O(n1/2)
15-853 Page8
What graphs don’t have small separatos
Expanders: Can someone give a quick proof that they don’t have small separators
Hypercubes: O(n) edge separators O(n/(log n)1/2) vertex separators
Butterfly networks:O(n/log n) separators ?
It is exactly the fact that they don’t have small separators that make them useful.
15-853 Page9
Applications of Separators
Circuit Layout (dates back to the 60s)VLSI layoutSolving linear systems (nested dissection)
n3/2 time for planar graphsPartitioning for parallel algorithmsApproximations to certain NP hard problems
TSP, maximum-independent-setClustering and machine learningMachine vision
15-853 Page10
More Applications of Separators
Out of core algorithmsRegister allocationShortest PathsGraph compressionGraph embeddings
15-853 Page11
Available Software
METIS: U. MinnessotaPARTY: University of PaderbornCHACO: Sandia national labsJOSTLE: U. GeenwichSCOTCH: U. BordeauxGNU: Popinet
Benchmarks:• Graph Partitioning Archive
15-853 Page12
Different Balance Criteria
Bisectors: 50/50Constant fraction cuts: e.g. 1/3, 2/3Trading off cut size for balance:
min cut criteria:
min quotient separator:
All versions are NP-hard
21'
'min
VV
V
VV
),min(
'min
21' VV
V
VV
15-853 Page13
Other Variants of Separators
k-Partitioning:Might be done with recursive partitioning, but direct solution can give better answers.
Weighted:Weights on edges (cut size), vertices (balance)
Hypergraphs:Each edge can have more than 2 end pointscommon in VLSI circuits
Multiconstraint:Trying to balance different values at the same time.
15-853 Page14
Edge vs. Vertex separators
If a class of graphs satisfies an f(n) edge-separator theorem then it satisfies an f(n) vertex-separator.
The other way is not true (unless degree is bounded)
|C| = n/2
15-853 Page15
Separator Trees
78
3
4
01
2
56
8
12
56
7
3
4
0
2
68
1
574
3
0
3 0 7 4
85
1 8 5 2 6
15-853 Page16
Asymptotics
If S is a class of graphs closed under the subgraph relation, then
Definition: S satisfies a f(n) vertex-separator theorem if there are constants < 1 and > 0 so that for every G S there exists a cut set C V, with1. |C| < f(|G|) cut size2. |A| < |G|, |B| < |G| balance
Similar definition for edge separators.
15-853 Page17
Separator Trees
Theorem: For S satisfying an (,) n1- edge separator theorem, we can generate a perfectly balanced separator tree with separator size |C| < k f(|G|).
Proof: by picture |C| = n1-(1 + + 2 + …) = n1-(1/1-)
15-853 Page18
Algorithms
All are either heuristics or approximations– Kernighan-Lin (heuristic)– Planar graph separators
(finds O(n1/2) separators)– Geometric separators
(finds O(n(d-1)/d) separators)– Spectral (finds O(n(d-1)/d) separators)– Flow techniques (give log(n)
approximations)– Multilevel recursive bisection
(heuristic, currently most practical)
04/13/2007 CS267 Guest Lecture 19
Partitioning via Breadth First Search
BFS identifies 3 kinds of edges– Tree Edges - part of T– Horizontal Edges - connect nodes at same level– Interlevel Edges - connect nodes at adjacent levels
No edges connect nodes in levels differing by more than 1 (why?)BFS partioning heuristic
– N = N1 U N2, where • N1 = {nodes at level <= L},
• N2 = {nodes at level > L}
– Choose L so |N1| close to |N2|BFS partition of a 2D Mesh using center as root: N1 = levels 0, 1, 2, 3 N2 = levels 4, 5, 6
root
15-853 Page20
Kernighan-Lin Heuristic
Local heuristic for edge-separators based on “hill climbing”. Will most likely end in a local-minima.
Two versions:Original: takes n2 times per stepFiduccia-Mattheyses: takes n times per step
15-853 Page21
High-level description for both
Start with an initial cut that partitions the vertices into two equal size sets V1 and V2
Want to swap two equal sized sets X A and Y B to reduce the cut size.
Note that finding the optimal subsets X and Y solves the optimal separator problem, so it is NP hard.
We want some heuristic that might help.
A B
YX
C
15-853 Page22
Some Terminology
C(A,B) : the weighted cut between A and B
I(v) : the number of edges incident on v that stay within the partition
E(v) : the number of edges incident on v that go to the other partition
D(v) : E(v) - I(v) D(u,v) : D(u) + D(v) - 2 w(u,v)
the gain for swapping u and v
A
B
Cv
u
15-853 Page23
Kernighan-Lin improvement step
KL(G,A0,B0)
(u A0, v B0 ) put (u,v) in a PQ based on
D(u,v) for k = 1 to |V|/2
(u,v) = max(PQ)(Ak,Bk) = (Ak-1,Bk-1) swap (u,v)delete u and v entries from PQ update D on neighbors (and PQ)
select Ak,Bk with best Ck
Note that can take backward steps (D(u,v) can be negative).
A
B
Cv
u
15-853 Page24
Fiduccia-Mattheyses improvement step
FM(G,A0,B0)
u A0 put u in PQA based on D(u)
v B0 put v in PQB based on D(v)
for k = 1 to |V|/2 u = max(PQA)
put u on B side and update Dv = max(PQb)
put v on A side and update D select Ak,Bk with best Ck
A
B
Cv
u
15-853 Page25
A Bad Example for KL or FMConsider following graph with initial cut given in red.
KL (or FM) will start on one side of grid and flip pairs over moving across the grid until the whole thing is flipped.
The cut will never be better than the original bad cut.
2 2 2
1 1 1
15-853 Page26
Performance in Practice
In general the algorithms do very well at smoothing a cut that is approximately correct.
Used by most separator packages either1. to smooth final results2. to smooth partial results during the
algorithm3. K/L tested on very small graphs (|N|<=360)
and got convergence after 2-4 sweeps
4. For random graphs (of theoretical interest) the probability of convergence in one step appears to drop like 2-|N|/30
27
Introduction to Multilevel Partitioning
If we want to partition G(N,E), but it is too big to do efficiently, what can we do?
– 1) Replace G(N,E) by a coarse approximation Gc(Nc,Ec), and partition Gc instead
– 2) Use partition of Gc to get a rough partitioning of G, and then iteratively improve it
What if Gc still too big?
– Apply same idea recursively
28
Multilevel Partitioning - High Level Algorithm
(N+,N- ) = Multilevel_Partition( N, E ) … recursive partitioning routine returns N+ and N- where N = N+ U N- if |N| is small(1) Partition G = (N,E) directly to get N = N+ U N- Return (N+, N- ) else
(2) Coarsen G to get an approximation Gc = (Nc, Ec)
(3) (Nc+ , Nc- ) = Multilevel_Partition( Nc, Ec )
(4) Expand (Nc+ , Nc- ) to a partition (N+ , N- ) of N(5) Improve the partition ( N+ , N- ) Return ( N+ , N- ) endif
(2,3)
(2,3)
(2,3)
(1)
(4)
(4)
(4)
(5)
(5)
(5)
How do we Coarsen? Expand? Improve?
“V - cycle:”
29
Multilevel Kernighan-LinCoarsen graph and expand partition using
maximal matchingsImprove partition using Kernighan-Lin
30
Maximal Matching: Example
31
Example of Coarsening
32
Coarsening using a maximal matching
1) Construct a maximal matching Em of G(N,E)for all edges e=(j,k) in Em 2) collapse matched nodes into a single one Put node n(e) in Nc
W(n(e)) = W(j) + W(k) … gray statements update node/edge weightsfor all nodes n in N not incident on an edge in Em 3) add unmatched nodes Put n in Nc … do not change W(n)… Now each node r in N is “inside” a unique node n(r) in Nc
… 4) Connect two nodes in Nc if nodes inside them are connected in Efor all edges e=(j,k) in Em
for each other edge e’=(j,r) or (k,r) in E Put edge ee = (n(e),n(r)) in Ec
W(ee) = W(e’) If there are multiple edges connecting two nodes in Nc, collapse them, adding edge weights
33
Expanding a partition of Gc to a partition of G
34
Multilevel Spectral Bisection
Coarsen graph and expand partition using maximal independent sets
Improve partition using Rayleigh Quotient Iteration
35
Example of Coarsening
- encloses domain Dk = node of Nc
36
Coarsening using Maximal Independent Sets… Build “domains” D(k) around each node k in Ni to get nodes in Nc
… Add an edge to Ec whenever it would connect two such domains
Ec = empty set
for all nodes k in Ni
D(k) = ( {k}, empty set ) … first set contains nodes in D(k), second set contains edges in D(k)unmark all edges in Erepeat choose an unmarked edge e = (k,j) from E if exactly one of k and j (say k) is in some D(m) mark e add j and e to D(m) else if k and j are in two different D(m)’s (say D(mi) and D(mj)) mark e add edge (mi, mj) to Ec
else if both k and j are in the same D(m) mark e add e to D(m) else leave e unmarked endifuntil no unmarked edges
37
Expanding a partition of Gc to a partition of G
Need to convert an eigenvector vc of L(Gc) to an approximate eigenvector v of L(G)
Use interpolation:
For each node j in N
if j is also a node in Nc, then
v(j) = vc(j) … use same eigenvector component else
v(j) = average of vc(k) for all neighbors k of j in Nc end ifendif
38
Example: 1D mesh of 9 nodes
39
Improve eigenvector: Rayleigh Quotient Iteration
j = 0
pick starting vector v(0) … from expanding vcrepeat j=j+1
r(j) = vT(j-1) * L(G) * v(j-1) … r(j) = Rayleigh Quotient of v(j-1) … = good approximate eigenvalue
v(j) = (L(G) - r(j)*I)-1 * v(j-1) … expensive to do exactly, so solve approximately … using an iteration called SYMMLQ, … which uses matrix-vector multiply (no surprise) v(j) = v(j) / || v(j) || … normalize v(j) until v(j) converges… Convergence is very fast: cubic
40
Example of convergence for 1D mesh
41
Available Implementations
Multilevel Kernighan/Lin– METIS (www.cs.umn.edu/~metis)– ParMETIS - parallel version
Multilevel Spectral Bisection– S. Barnard and H. Simon, “A fast multilevel
implementation of recursive spectral bisection …”, Proc. 6th SIAM Conf. On Parallel Processing, 1993
– Chaco (www.cs.sandia.gov/CRF/papers_chaco.html)
Hybrids possible – Ex: Using Kernighan/Lin to improve a partition
from spectral bisection
42
Comparison of methods
Compare only methods that use edges, not nodal coordinates – CS267 webpage and KK95a (see below) have other
comparisons
Metrics– Speed of partitioning– Number of edge cuts– Other application dependent metrics
Summary– No one method best– Multi-level Kernighan/Lin fastest by far, comparable to Spectral
in the number of edge cuts– Spectral give much better cuts for some applications
• Ex: image segmentation• See “Normalized Cuts and Image Segmentation” by J. Malik, J. Shi
43
Number of edges cut for a 64-way partition
Graph
1444ELTADD32AUTOBBMATFINAN512LHR10MAP1MEMPLUSSHYY161TORSO
# of Nodes
144649 15606 4960 448695 38744 74752 10672 267241 17758 76480 201142
# of Edges
1074393 45878 94623314611 993481 261120 209093 334931 54196 1520021479989
Description
3D FE Mesh2D FE Mesh32 bit adder3D FE Mesh2D Stiffness M.Lin. Prog.Chem. Eng.Highway Net.Memory circuitNavier-Stokes3D FE Mesh
# Edges cut
88806 2965 675 194436 55753 11388 58784 1388 17894 4365 117997
Expected# cuts
6427 2111 1190 11320 3326 4620 1746 8736 2252 4674 7579
3D mesh
31805 7208 3357 67647 13215 20481 5595 47887 7856 20796 39623
Expected # cuts for 64-way partition of 2D mesh of n nodes
n1/2 + 2*(n/2)1/2 + 4*(n/4)1/2 + … + 32*(n/32)1/2 ~ 17 * n1/2
Expected # cuts for 64-way partition of 3D mesh of n nodes =
n2/3 + 2*(n/2)2/3 + 4*(n/4)2/3 + … + 32*(n/32)2/3 ~ 11.5 * n2/3
For Multilevel Kernighan/Lin, as implemented in METIS (see KK95a)
Thank you!
44