combinatorial and algebraic tools for multigrid
DESCRIPTION
Combinatorial and algebraic tools for multigrid. Yiannis Koutis Computer Science Department Carnegie Mellon University. multilevel methods. www.mgnet.org 3500 citations 25 free software packages 10 special conferences since 1983 Algorithms not always working - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/1.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Combinatorial and algebraic tools for
multigridYiannis Koutis
Computer Science DepartmentCarnegie Mellon University
![Page 2: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/2.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multilevel methods
www.mgnet.org• 3500 citations• 25 free software packages• 10 special conferences since 1983
Algorithms not always workingLimited theoretical understanding
![Page 3: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/3.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multilevel methods: our goals
• provide theoretical understanding• solve multilevel design problems• small changes in current software
• study structure of eigenspaces of Laplacians
• extensions for multilevel eigensolvers
![Page 4: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/4.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 5: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/5.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
quick definitions
• Given a graph G, with weights wij
• Laplacian: A(i,j) = -wij, row sums =0
• Normalized Laplacian:
• (A,B) is a measure of how well B approximates A (and vice-versa)
![Page 6: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/6.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
linear systems : preconditioning
• Goal: Solve Ax = b via an iterative method
• A is a Laplacian of size n with m edges. Complexity depends on (A,I) and m
• Solution: Solve B-1Ax = B-1b• Bz=y must be easily solvable• (A,B) is small• B is the preconditioner
![Page 7: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/7.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 8: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/8.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• B is a sparse subgraph of A, possibly with additional edges
Solving Bz=y is performed as follows:1. Gaussian elimination on degree ·2 nodes
of B2. A new system must be solved 3. Recursively call the same algorithm on
to get an approximate solution.
![Page 9: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/9.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• Graph Sparsification [Spielman, Teng]• Low stretch trees [Elkin, Emek,
Spielman, Teng]• Near optimal O(m poly( log n))
complexity
![Page 10: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/10.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditionersthe Vaidya thread
• Graph Sparsification [Spielman, Teng]• Low stretch trees [Elkin, Emek, Spielman,
Teng]• Near optimal O(m poly( log n)) complexity
• Focus on constructing a good B• (A,B) is well understood – B is sparser than
A• B can look complicated even for simple
graphs A
![Page 11: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/11.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 12: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/12.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• the support graph S is bigger than A
1 12 3 1 2 2 1 1
![Page 13: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/13.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• the support graph S is bigger than A
2
12 3 1 2 2 1 1
1 12 3 1 2 2 1 1
3 2 1
1 3 3 4 4 3 4 3 2 1
Quotient
1
![Page 14: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/14.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
combinatorial preconditioners
the Gremban - Miller thread• The preconditioner S is often a
natural graph • S inherits the sparsity properties of A• S is equivalent to a dense graph B of
size equal to that of A : (A,S) = (A,B)• Analysis of (A,S) made easy by work of
[Maggs, Miller, Ravi, Woo, Parekh]
• Existence of good S by work of [Racke]
![Page 15: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/15.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions• Other results
![Page 16: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/16.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Suppose we are given m clusters in A
• R(i,j) = 1 if the jth cluster contains node i
• R is n x m • Quotient
• R is the clustering matrix
algebraic expressions
![Page 17: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/17.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• The inverse preconditioner
• The normalized version
• RT D1/2 is the weighted clustering matrix
algebraic expressions
![Page 18: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/18.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions• Other results
![Page 19: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/19.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
good partitions and low frequency invariant
subspaces• Suppose the graph A has a good
clustering defined by the clustering matrix R
• Let• Let y be any vector such that
![Page 20: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/20.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Suppose the graph A has a good clustering defined by the clustering matrix R
• Let• Let y be any vector such that
Theorem: The inequality is tight up to a constant for
certain graphs
good partitions and low frequency invariant
subspaces
quality
test?
![Page 21: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/21.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
good partitions and low frequency invariant
subspaces• Let y be any vector such that • Let x be mostly a linear combination of
eigenvectors corresponding to eigenvalues close to
Theorem: • Prove ?• We can find random vector x and check
the distance to the closest y
![Page 22: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/22.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 23: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/23.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
multigrid – short introduction
• General class of algorithms
• Richardson iteration:
• High frequency components are reduced:
![Page 24: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/24.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
initial and smoothed error
initial error smoothed error
![Page 25: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/25.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
• Define a smaller graph Q• Define a projection operator
Rproject• Define a lift operator Rlift
the basic multigrid algorithm
1. Apply t rounds of smoothing 2. Take the residual r = b-Axold
3. Solve Qz = Rprojectr4. Form new iterate xnew = xold + Rlift z
5. Apply t rounds of smoothing
how many? which
iteration ?
recursion
is this needed ?
![Page 26: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/26.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)
Goals: The range of Rproject must approximate the unreduced error very well. The error not reduced by smoothing must be reduced by the smaller grid.
![Page 27: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/27.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)Goals: The range of Rproject must approximate
the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid.
• Jacobi iteration: • or ‘scaled’ Richardson:• Find a clustering • Rproject = (Rlift)T
• Q = RprojectT A Rproject
![Page 28: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/28.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
algebraic multigrid (AMG)Goals: The range of Rproject must approximate
the unreduced error very well. The error not reduced by smoothing must be reduced in the smaller grid.
• Jacobi iteration: • or ‘scaled’ Richardson• Find a clustering [heuristic]• Rproject = (Rlift)T [heuristic]
• Q = RprojectT A Rproject
![Page 29: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/29.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
two level analysis
• Analyze the maximum eigenvalue of
• where
• The matrix T1 eliminates the error in
• A low frequency eigenvector has a significant component in
![Page 30: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/30.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
two level analysis
• Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject. Assume, <X,Y>2 · /
• Two level convergence : error reduced by
• Proving the hypothesis ? Limited cases
![Page 31: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/31.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
current state
‘there is no systematic AMG approach that has proven effective in any kind of general context’
[BCFHJMMR, SIAM Journal on Scientific Computing, 2003]
![Page 32: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/32.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 33: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/33.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions – two level
• There exists a good clustering given by R. The quality is measured by the condition number (A,S)
• Q = RT A R• Richardson’s with
• Projection matrix
![Page 34: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/34.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions - two level analysis
• Starting hypothesis: Let X be the subspace corresponding to eigenvalues smaller than . Let Y be the null space of Rproject = RTD1/2 Assume, <X,Y>2 · /
• Two level convergence : error reduced by • Proving the hypothesis ? Yes! Using (A,S)• Result holds for t=1 smoothing• Additional smoothings do not help
![Page 35: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/35.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
our contributions - recursion
• There is a matrix M which characterizes the error reduction after one full multigrid cycle
• We need to upper bound its maximum eigenvalue as a function of the two-level eigenvalues
the maximum eigenvalue of M is upper bounded by the sum of the maximum
eigenvalues over all two-levels
![Page 36: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/36.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
towards full convergence
• Goal: The error not reduced by smoothing must be reduced by the smaller grid
A different point of viewThe small grid does not reduce part
of the error. It rather changes its spectral profile.
![Page 37: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/37.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
full convergence for regular d-dimensional toroidal
meshes• A simple change in the
implementation of the algorithm:
• where
• T2 has eigenvalues 1 and -1
• T2 xlow = xhigh
![Page 38: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/38.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
full convergence for regular d-dimensional toroidal
meshes• With t=O(log log n) smoothings
• Using recursive analysis: max(M) · 1/2
• Both pre-smoothings and post-smoothings are needed
• Holds for perturbations of toroidal meshes
![Page 39: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/39.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Overview
• Quick definitions• Subgraph preconditioners• Support graph preconditioners• Algebraic expressions• Low frequency eigenvectors and good
partitionings• Multigrid introduction and current state • Multigrid – Our contributions
![Page 40: Combinatorial and algebraic tools for multigrid](https://reader034.vdocuments.mx/reader034/viewer/2022050909/56814e24550346895dbb89df/html5/thumbnails/40.jpg)
05/11/2005
Carnegie Mellon School of Computer Science
Aladdin Lamps 05
Thanks!