fast linear solvers for laplacian systems - ucsd research...

SolvingLaplacianSystems

OliviaSimpson

Introduction

BackgroundInformation

Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Fast Linear Solvers for Laplacian SystemsUCSD Research Exam

Olivia Simpson

University of California, San Diego

[email protected]

November 8, 2013


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Overview

1 Introduction

2 Background InformationLaplacian MatricesLinear Systems in the Graph Laplacian

3 Fast Linear SolversPrevious MethodsST-SolverA Better Sparsifier

4 Solving Linear Systems with Boundary Conditions UsingRandom Walks

Boundary ConditionsComputing Heat Kernel PagerankA Variation of the ProblemSolving the System

5 Future Directions


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Definitions

Let G = (V ,E ) be an undirected edge-weighted graph on nvertices and m edges. We can generalize an unweighted graphto the case where all edge weights are equal to 1.The degree of a vertex v ∈ V is the sum of the weights of theedges adjacent to it,

dv =∑u∼v

w(u, v).

The degree matrix is the diagonal matrix whose entries are thedegrees of the vertices,

(D)vv = dv .


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Definitions

The adjacency matrix is the n × n matrix which is the weightw(e) in the entries corresponding to an edge,

(A)uv =

{w(u, v) if {u, v} ∈ E ,

0 otherwise.

The Laplacian is the matrix

L = D − A.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Example

For the followingsimple graph,

L =

2 0 0 0 00 2 0 0 00 0 4 0 00 0 0 1 00 0 0 0 3

−

0 0 1 0 10 0 1 0 11 1 0 1 10 0 1 0 01 1 1 0 0

=

2 0 −1 0 −10 2 −1 0 −1

−1 −1 4 −1 −10 0 −1 1 0

−1 −1 −1 0 3


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Properties of the Laplacian

• Symmetric

• Diagonally dominant

• All zero row-sums

• Non-positive off-diagonal entries

• Clean quadratic form:

xTLx =∑u∼v

(x(u)− x(v))2


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Consensus

A network of n decision-making agents

• edges are communicationchannels

• xi is the decision value ofeach agent

• values can be influenced bycommunicating neighbors

• two agents vi , vj are said toagree when xi = xj

Goal: all agents reach a common decision value, called the con-sensus


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Consensus

A network of n decision-making agents

The Laplacian potential is a mea-sure of disagreement in the system,

ΨG (x) =1

2xTLx

=1

2

∑vi∼vj

(xi − xj)2.

all agents agree ⇔ ΨG (x) = 0.

Goal: find the vector x which minimizes the Laplacian poten-tial [OM03]


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Why Laplacian Systems?

Laplacian Systems arise in a number of natural contexts,

• Characterizing the motion of coupled oscillators [HS08]

• Approximating Fiedler eigenvectors [ST04]

• ...

• Computing effective resistance in an electricalnetwork [Kir1847]


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

• Edges are wires in the network

• Edge weights correspond to the conductance of each wire

• Goal: set voltages x1, x2, x3 on the vertices to create aflow of electrical current


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

Ohm’s law: take conductance(e) = 1/resistance(e), then

current(e) = conductance(e)× |voltage(u)− voltage(v)|

Consider the flow of current from V 1. By Ohm’s law:

current(V 1,V 2) = 1(voltage(V 1)− voltage(V 2))

current(V 1,V 3) = 2(voltage(V 1)− voltage(V 3))


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

Kirchhoff’s law [Kir1847]: For every point in the network,

netflow(v) := flowin(v)− flowout(v) = 0,

except at the injection point, netflow = 1, and the extractionpoint, netflow = −1.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

So, at vertex V 1 the voltage x1 must satisfy:

netflow(V 1) = current(V 1,V 2) + current(V 1,V 3)

1 = 1(x1 − x2) + 2(x1 − x3)

1 = 3x1 − x2 − 2x3


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

Applying the same rules at V 2 and V 3 yields the following sys-tem of equations:

3x1 − x2 − 2x3 = 1

−x1 + 2x2 − x3 = 0

−2x1 − x2 + 3x3 = −1


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3

x1

x2

x3

=

10−1


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

Or, in matrix form, 3 −1 −2−1 2 −1−2 −1 3

x1

x2

x3

=

10−1

A system in the Laplacian of the network.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Electrical Networks

When current is injected at V 1 and extracted at V 3, the solutionvector x can be used to compute the effective resistance of theedge (V 1,V 3),

R(V 1,V 3) = |x1 − x3|


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Consensus Again

[OM03] A connected networkof decision-making agents

• edges are communicationchannels

• xi is the decision value ofeach agent

• values can be influenced bycommunicating neighbors

• two agents vi , vj are said toagree when xi = xj

Goal: all agents reach a common decision value, called theconsensus


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Consensus Again

[OM03] A connected networkof decision-making agents

Suppose each agent evolves theirdecision value according to thedistributed linear protocol,

xi (t) =∑vj∼vi

xj(t)− xi (t).

Then the solution to the system

x = −Lx , x(0) ∈ Rn

is the vector of decision values as afunction of t.

Goal: all agents reach a common decision value, called theconsensus


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Overview

1 Introduction





5 Future Directions


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Solving Directly

Given a system of linear equations, Ax = b, we can solve thesystem directly using Gaussian elimination.

Gaussian elimination takes O(n3) time in general.

- [CW90] show the exponent can be as small as 2.376

- [Wil12] improves this to 2.3727


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Iterative Methods

Iterative methods for solving systems Ax = b are based on asequence of increasingly better approximations.

The idea is to construct a sequencex (0), x (1), . . . , x (i), . . . , x (N), . . . that converges to the truesolution, x ,

limk→∞

x (k) = x .

In practice, the iterative process is stopped after N iterationswhen

||x (N) − x || < ε

for any vector norm and a prescribed error parameter ε.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Iterative Methods

Richardson’s Method([Young53],[GV61])

An iterative method that improves approximations using theresidual error, b − Ax (i), at each step:

x (i+1) = x (i) + (b − Ax (i))

= b + (I − A)x (i)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Preconditioned Iterative Methods

To accelerate the iterative process, instead use anapproximation of the matrix, called a preconditioner.This method will produce a solution to the preconditionedsystem:

B−1Ax = B−1b.

What makes a matrix B a good preconditioner for A?

1 It is a very good approximation of the matrix

2 It reduces the number of iterations

3 B can be computed quickly

4 Systems in B can be solved quickly


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Preconditioned Richardson’s Method([Young53],[GV61])

x (i+1) = b + (I − A)x (i)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



x (i+1) = B−1b + (I − B−1A)x (i)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



x (i+1) = B−1b + (I − B−1A)x (i)

The term B−1b involves solving a system in B.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Convergence of Iterative Methods

Since the solution x is not known, different criteria must beused to determine the value N for which

||x (N) − x || < ε.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



||x (N) − x || < ε.

- One criterion uses the residual vector, r (i) = b − Ax (i)

- The minimum iterations N should satisfy

||x − x (N)||||x ||

≤ κ(A)||r (N)||||b||

≤ εκ(A),

where κ(A) = ||A−1|| · ||A|| is the condition number of A for anymatrix norm || · || [QRS07].


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



||x (N) − x || < ε.

In the case of preconditioned methods, this becomes

||B−1r (N)||||B−1r (0)||

≤ ε.

In particular, this means the rate of convergence will depend onhow quickly systems in B can be solved.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Preconditioned Chebyshev method will find solutions withabsolute error ε in time

O(mS(B) log(κ(A)/ε)√κ(A,B)), [GO88]

where S(B) is the time required so solve a system in B and

κ(A,B) =

(max

x :Ax 6=0

xTAx

xTBx

)(max

x :Ax 6=0

xTBx

xTAx

).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Factors:

• minimum number of iterations N, depends on κ(A)

• time to solve the system in B

In general, worst case time bounds are O(mn).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Overview

1 Introduction





5 Future Directions


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

A First Nearly-Linear Time Solver

Spielman and Teng [ST04] presented the first nearly-lineartime algorithm for solving systems of equations in symmetric,diagonally-dominant (SDD) systems.

The success of the ST-solver can be ascribed to twoinnovations:

1 enhancing a one-level iterative solver using recursion

2 using a preconditioning matrix based on a subgraph


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

The Recursive Solver

The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:

solve(A,b):

if dimension-check(A) and sparse(A):

return PrecChebyshev(A,b)

B = precondition(A)

A’ = reduce(B)

solve(A’,b)

• Compute B, the preconditioner for A

• Reduce the system in B to a system in A1, a refinedversion of A

• Recursively solve the system in A1

• At the base, use the Preconditioned Chebyshev method


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


The ST-solver addresses the problem of solving systems in thepreconditioning matrix B with recursion:

A→ B → A1 → B1 → · · · → Ak

• Compute B, the preconditioner for A

• Reduce the system in B to a system in A1, a refinedversion of A

• Recursively solve the system in A1

• At the base, use the Preconditioned Chebyshev method


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Two methods are used at each level of the recursion.

reduce(B): Bi → Ai+1

- Reduce the preconditioned matrix by greedily removing rowsand columns with at most two non-zero entries- Done with a partial Cholesky decomposition

precondition(A): Ai → Bi

- Sparsify the graph associated to A to obtain a subgraph H- Set B to be a matrix of H


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Graph Preconditioners

[GMZ95]: There is a linear time transformation:

Ax = b SDD system

⇓Lx ′ = b′ Laplacian system

For the recursive sparsifying procedures, Spielman and Tenguse the underlying graphs of the matrices to produce a chain ofprogressively sparser graphs.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



Ax = b SDD system



A→ B → A1 → B1 → · · · → Ak


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



Ax = b SDD system



LA → LB → LA1 → LB1 → · · · → LAk


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



Ax = b SDD system



G → H → G1 → H1 → · · · → Gk


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Spanning Trees

[Vaidya91]: subgraphs serve as good preconditioners

• maximum weight spanning tree as a preconditioning base

• solves SDD systems with non-positive off-diagonal entriesof degree d in time

O(dn)1.75 log(κ(A)/ε)

[ST04]: subgraphs serve as good preconditioners

• a better base tree uses an edge measure called stretch


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

The Stretch of an Edge

The detour forced by traversing T instead of G


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



e = {V 1,V 4},w(e) = 1


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



A spanning tree, T


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



e = {V 1,V 4}, Tree path: {V 1,V 3}, {V 3,V 5}, {V 5,V 4}


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Let T be a spanning tree of a weighted graph G , and let theweight of an edge e = {u, v} be denoted by w(e). Definew ′(e) = 1/w(e), which is the resistance of the edge, re .

Let e1, e2, . . . , ek be the unique path in T from u to v . Thenthe stretch of the edge by T is defined

stretchT (e) =

∑ki=1 w ′(e1)

w ′(e)= 1/re

k∑i=1

rei

The total stretch of a graph G by the tree T is the sum of thestretch of all the off-tree edges.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Low Stretch Spanning Trees

Low stretch spanning tree (LSST) for the preconditioning base↓

A “spine” that keeps resistance low

LSSTs for preconditioners had been implemented before([AKPW95, BH01, BH03]), but combining the high qualitypreconditioners with a recursive solver amounted to analgorithm for solving SDD systems in nearly-linear time.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

ST-Solver

[ST04]: Given a system of equations Ax = b in a symmetric,diagonally dominant matrix, the ST-solver computes a vector xsatisfying

||Ax − b|| < ε and ||x − x || ≤ ε

in time O(m logO(1) m), where the exponent is a large constant.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

A Better Sparsifier

Sparsifier of the ST-solver

1 Compute an LSST T of the graph G

2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′

3 Replace T in G by T ′

4 H ← T ′

5 Add off-tree edges to H by sampling with probabilitiesrelated to vertex degree


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

A Better Sparsifier

Sparsifier of Koutis et al. [KMP10, KMP11]

1 Compute an LSST T of the graph G

2 Reweight the edges of T by a constant factor k , call thereweighted tree T ′

3 Replace T in G by T ′

4 H ← T ′

5 Add off-tree edges to H by sampling with probabilitiesrelated to effective resistance


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

A Better Sparsifier

pro: Probabilities related to effective resistance yield sparsifierswith few edges [SS11]con: Computing effective resistances involves solving a systemof linear equations

This is problem is avoided by instead using upper bounds foredge probabilities,

pe ≥ weRe ,

where Re is the effective resistance of the edge.

The sparsifier of [KMP11] improved the expected time boundto O(m log2 n log(1/ε)) for an approximate solution withabsolute error bounded by ε.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Summary of Methods Discussed

Gaussian elimination O(n3)[CW90] O(n2.376)[Wil12] O(n2.3727)

Prec Chebyshev [GO88] O(mS(B) log(κ(A)/ε)√κ(A,B))

[Vaidya91] O(dn)1.75 log(κ(A)/ε)

[ST04] O(m logO(1) m)[KMP11] O(m log2 n log(1/ε))


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Overview

1 Introduction





5 Future Directions


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

The Boundary of a Subset

Let S be a subset of vertices ina graph.

S = {S1,S2,S3}


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Let S be a subset of vertices ina graph.

The vertex boundary of S is theset of vertices not in S whichborder S

δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Let b be a vector over the ver-tices of a graph.

Then a vector x over the satis-fies the boundary condition of bfor a subset S when

x(v) = b(v) ∀v ∈ δ(S).

δ(S) = {v /∈ S : {v , u} ∈ E for some u ∈ S}.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Leader-Following Formation

• a network with a leader, anda group of agents

• values xi correspond toposition

• leader moves independently ofagents

Goal: design a distributed protocol for agents to follow the leader


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


The leader is the boundary of thesubset of agents.

So our solution should:- Respect the leader’s position (fixthe solution on the boundary)- Use local information among theagents (compute the solution onthe subset)

Goal: design a distributed protocol for agents to follow the leader


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


[NC10]: Consider a multi-agent system of n agents and oneleader.

1 The dynamics of the leader is

x0 = Ax0,

which is independent.

2 The dynamics of each agent is

xi =∑vj∼vi

xj − xi ,

and the vector x of agentpositions is given by

x = −Lx .


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Satisfying the Boundary Condition

Consider a linear system Lx = b. Suppose there exists a subsetS such that

• the induced subgraph on S is connected

• the boundary δ(S) is non-empty

• support(b) ⊆ δ(S)

• x satisfies the boundary condition b:

x(v) = b(v), ∀v ∈ δ(S).

Goal: find the solution x restricted to S .That is, the solution will satisfy

x(v) =

{1dv

∑u∼v x(u) if v ∈ S

b(v) if v ∈ δ(S).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Solving Linear Systems withBoundary Conditions

Main tools and techniques:

• Fast computation of a heat kernel pagerank vector

• Expressing a Laplacian linear system with boundaryconditions in terms of the heat kernel of the graph

• Approximate the solution by a sum of heat kernelpagerank vectors


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Walks on a Graph

Consider P = D−1A as a random walk matrix.

P =

0 0 1/2 0 1/20 0 1/2 0 1/2

1/4 1/4 0 1/4 1/40 0 1 0 0

1/3 1/3 1/3 0 0


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Walks on a Graph

Consider P = D−1A as a random walk matrix.

P =

0 0 1/2 0 1/20 0 1/2 0 1/2

1/4 1/4 0 1/4 1/40 0 1 0 0

1/3 1/3 1/3 0 0

When f is a probability distribution vector, f TPk is thedistribution after k random walk steps.

Define ∆ to be the Laplace operator, defined ∆ = I − P.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Heat Kernel PagerankThe heat kernel pagerank vector is determined by parameterst ∈ R+ and f ∈ Rn,

ρt,f = f T e−t∆ = e−t∞∑k=0

tk

k!f TPk .

• If f is a starting distribution over the vertices, then f TPk

will be the distribution after k random walk steps

• The sum of the coefficients satisfies

∞∑k=0

e−ttk

k!= e−t · et = 1

• Then if k steps of a P random walk are taken withprobability e−t t

k

k!

• ⇒ ρt,f is the expected distribution of the process


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Computing Heat Kernel Pagerank

Q: A good way to compute the heat kernel pagerank?

A: Draw enough samples of the random process to get theexpected value.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections




? Avoid an exponential sum by taking samples.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections




? Avoid an exponential sum by taking samples.◦ Control error by drawing enough samples r(ε)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections





? Avoid long walk processes by sampling truncated randomwalks.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections





? Avoid long walk processes by sampling truncated randomwalks.◦ Control contribution lost in later step by stopping after enoughsteps K (ε)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections




Let pk be the probability of taking k random walk steps,

pk = e−ttk

k!.

Let X be the distribution over values of k.• Then obtain a good approximation by taking the average of rrandom walks of length at most K .• The variables r ,K are both in terms of a prescribed errorparameter, ε.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Let the output of the algorithm be ρt,f . Then, to achieve

(1− ε)ρt,f [v ]− ε ≤ ρt,f [v ] ≤ (1 + ε)ρt,f [v ],

we choose

r ← 16

ε3log s,

K ← log(ε−1)

log log(ε−1).

Then, assuming that (1) taking a random walk step and (2)sampling from a distribution take constant time, the runtime ofthe heat kernel pagerank computation is

r · K =16

ε3log s · log(ε−1)

log log(ε−1)= O

( log s log(ε−1)

ε3 log log ε−1

).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections







OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Linear Systems with BoundaryConditions

Recall what it means for a solution vector x to satisfy theboundary condition in the system Lx = b.

S = {v ∈ V : b[v ] = 0},

x(v) =

{1dv

∑u∼v x(u) if v ∈ S

b(v) if v ∈ δ(S).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Given a graph G and a vector b of boundary conditions,• Let S be the subset of s = |S | vertices

S = {v ∈ V : b[v ] = 0}.

• Let AδS be the s × |δ(S)| matrix A with columns restricted tothe vertices of δ(S) and rows to vertices of S .• Let LS , AS and DS be L, A and D with rows and columnsrestricted to vertices of S .• Let xS be the solution vector over the vertices of S , and letbδS be b over δ(S).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Then we would like to find a vector x ∈ Rs satisfying:

DSxS = ASxS + AδSbδS

or, equivalently:

xS = (DS − AS)−1(AδSbδS)

= L−1S (AδSbδS),

The inverse L−1S exists when the induced subgraph on S is

connected and the boundary of S is non-empty.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

The Normalized Laplacian

The normalized Laplacian is defined

(L)uv =

1 if u = v ,−1√dudv

if u ∼ v ,

0 otherwise.

• Degree-nomalized version of L,

L = D−1/2LD−1/2

• Symmetric version of ∆ = I − P,

L = D1/2∆D−1/2


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


When G has no isolated vertex and the matrix D is invertible,

Lx = b and Lx ′ = b′

the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.

Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy

x ′S = L−1S (AδSb′δS)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


When G has no isolated vertex and the matrix D is invertible,

Lx = b and Lx ′ = b′

the system in L can be deduced from the system in L bysetting x ′ = D1/2x and b′ = D−1/2b.

Then the solution x ∈ Rs satisfying the boundary conditionshould satisfy

x ′S = L−1S (AδSb′δS)

x ′S = L−1S b1.

And computing b1 takes time proportional to the sum of vertexdegrees in δ(S) := vol(δ(S)).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Dirichlet Heat Kernel

The Dirichlet heat kernel is defined

HS,t = e−tLS = D1/2S e−t∆S D

−1/2S .

Lemma ([CS13])


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections




−1/2S .

Lemma ([CS13])

Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,

L−1S =

∫ ∞0HS,t dt.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections




−1/2S .

Lemma ([CS13])

Let S be a strict subset of vertices of a graph G and let theinduced subgraph on S be connected. Then,

L−1S =

∫ ∞0HS,t dt.

x ′S = L−1S b1 =

∫ ∞0HS,tb1 dt.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections







OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


• Approximate the solution

1 express the solution as an improper integral of heat kernelpagerank

2 approximate with a definite integral by limiting the range3 approximate by taking a finite sum of heat kernel pagerank

vectors4 approximate each term by sampling truncated random

walks


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Approximating with Heat Kernel

ClaimFor Lx ′ = b′,

x ′S =

(∫ ∞0

ρt,f dt

)D−1/2S , f = bT

1 D1/2S .


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


ClaimFor Lx ′ = b′: x ′S =

( ∫∞0 ρt,f dt

)D−1/2S , f = bT

1 D1/2S .

Proof.

x ′S =

∫ ∞0HS ,tb1 dt from Lemma

x ′S =

∫ ∞0

bT1 HS,t dt HS ,t symmetric

x ′S =

∫ ∞0

bT1 (D

1/2S e−t∆S D

−1/2S ) dt definition of HS,t

x ′S =

∫ ∞0

bT1 D

1/2S e−t∆S dt D

−1/2S


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


ClaimFor Lx ′ = b′: x ′S =

( ∫∞0 ρt,f dt

)D−1/2S , f = bT

1 D1/2S .

Proof.

x ′S =

∫ ∞0HS ,tb1 dt from Lemma

x ′S =

∫ ∞0

bT1 HS,t dt HS ,t symmetric

x ′S =

∫ ∞0

bT1 (D

1/2S e−t∆S D

−1/2S ) dt definition of HS,t

x ′S =

∫ ∞0

bT1 D

1/2S e−t∆S︸︷︷︸ dt D

−1/2S

ρt,f


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Finding the Solution

Let x1 = x ′S D1/2S . We can compute

x1 =

∫ ∞0

ρt,f dt, f = bT1 D

1/2S .

by a number of approximations,

1. Ignore the tail, take the integral to a finite T ,

x1 ≈∫ T

0ρt,f dt


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



x1 =

∫ ∞0

ρt,f dt, f = bT1 D

1/2S .


2. Discretize with a finite Riemann sum for small intervals T/N,

x1 ≈N∑j=1

ρjT/N,f · T/N


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections



x1 =

∫ ∞0

ρt,f dt, f = bT1 D

1/2S .


3. Obtain a good approximation by finitely many samples ofρjT/N,f over values of t.

for r times:

draw j from the interval [1,N]

compute ApproxHKPR(G, jT/N, f)


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Solving the System with BoundaryConditions

[CS13] show that to achieve an approximation vector xsatisfying

||x ′ − x || ≤ O(ε(1 + ||b||)),

the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.

Then using that the time to compute a heat kernel pagerankvector is

O( log s log(ε−1)

ε3 log log ε−1

),

and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


[CS13] show that to achieve an approximation vector xsatisfying

||x ′ − x || ≤ O(ε(1 + ||b||)),

the number of samples needed is r = ε−2(log s + log(ε−1)),where s = |S |.Then using that the time to compute a heat kernel pagerankvector is

O( log s log(ε−1)

ε3 log log ε−1

),

and assuming additional preprocessing time proportional tovol(δS), the main result of [CS13] is the following.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Theorem (Boundary Solver [CS13])

For a graph G and a linear system Lx = b, assume:

• x is required to satisfy the boundary condition b

• S = V \ support(b) and s = |S |• the boundary of S is nonempty

• the induced subgraph on S is connected.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections


Theorem (Boundary Solver [CS13])

Then the approximate solution x output by the boundary solversatisfies the following with probability ≥ 1− ε.

1 ||x − x || ≤ O(ε(1 + ||b||))

2 The running time is

O((log s)2(log(ε−1))2

ε5 log log(ε−1)

)with additional preprocessing time O(vol(δ(S))).


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Overview

1 Introduction





5 Future Directions


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Future Directions

Capitalize on speed.

• Use fast linear solvers to improve existing graph algorithms

• Interior point algorithms [SD08]• Learning problems [ZHS05]• Graph partitioning [ST04],[ACL06]• Sparsification [ST04],[SS11]

• Find more applications for these linear solvers

• Find more applications for heat kernel pagerank• Expected distribution of random walks• Vertex similarity → graph distance


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

ReferencesNoga Alon, Richard M. Karp, David Peleg, and Douglas West (1995)

A Graph-Theoretic Game and its Applications to the k-SeverProblem.

SIAM J. Computing, 21(1), 78–100.

Reid Andersen, Fan Chung, and Kevin Lang (2006)

Local Graph Partitioning Using PageRank Vectors.

FOCS’06, 475–486.

Erik Boman and Bruce Hendrickson (2001)

On Spanning Tree Preconditioners.

Sandia National Laboratories Manuscript.

Erik Boman and Bruce Hendrickson (2003)

Support Theory for Preconditioning.

SIAM J. Matrix Analysis and Applications, 25(3), 694–717.

Sergey Brin and Lawrence Page (1998)

The Anatomy of a Large-Scale Hypertextual Web Search Engine.

Computer Networks and ISDN Systems, 30(1), 107–117.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

References

Fan Chung and Olivia Simpson (2013)

Solving Linear Systems with Boundary Conditions using Heat KernelPagerank.

WAW’13.

Don Coppersmith and Shmuel Winograd (1990)

Matrix Multiplication via Arithmetic Progressions.

J. Symbolic Computation, 9(3), 251–280.

Samuel I. Daitch and Daniel A. Spielman (2008)

Faster Approximate Lossy Generalized Flow Via Interior PointAlgorithms.

STOC’08, 451–460.

Gene H. Golub and Michael L. Overton (1988)

The Convergence of Inexact Chebyshev and Richardson IterativeMethods for Solving Linear Systems.

Numerische Mathematik, 53(5), 571–593.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

References

Gene H. Golub and Richard S. Varga (1961)

Chebyshev Semi-Iterative Methods, Successive OverrelaxationIterative Methods, and Second Order Richardson Iterative Methods.

Numerical Math. 3, 147–168.

Keith D. Gremban, Gary L. Miller, and Marco Zagha (1995)

Performance Evaluation of a New Parallel Preconditioner.

International Parallel Processing Symposium ’95, 65–69.

Aric Hagberg and Daniel A. Schult (2008)

Rewiring Networks for Synchronization.

Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(3).

Gustav Kirchhoff (1847)

Uber die Auflosung der Gleichungen, auf welche man bei derUntersuchung der linearen Vertheilung galvanischer Strome gefuhrtwird.

Annalen der Physik, 148(12), 497–508.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

References

Ioannis Koutis, Gary L. Miller, and Richard Peng (2010)

Approaching Optimality for Solving SDD Linear Systems.

FOCS’10, 235–244.


A Nearly-m log n Time Solver for SDD Linear Systems.

FOCS’11, 590–598.


A Fast Solver for a Class of Linear Systems.

Communications of the ACM, 55(10), 99–107.

Wei Ni and Daizhan Cheng (2010)

Leader-Following Consensus of Multi-Agent Systems Under Fixed andSwitching Topologies.

Systems & Control Letters, 59(3), 209–217.

Reza Olfati-Saber and Richard Murray (2003)

Consensus Protocols for Networks of Dynamic Agents.

American Control Conference ’03, 951–956.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

References

Daniel A. Spielman and Nikhil Srivastava (2011)

Graph Sparsification by Effective Resistances.

SIAM J. Computing, 40(6), 1913–1926.

Daniel A. Spielman and Shang-Hua Teng (2004)

Nearly-Linear Time Algorithms for Graph Partitioning, GraphSparsification, and Solving Linear Systems.

STOC’04, 81–90.

Pravin M. Vaidya (1991)

Solving Linear Equations with Symmetric Diagonally DominantMatrices by Constructing Good Preconditioners.

UIUC manuscript based on a talk.

Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri (2007)

Numerical Mathematics, vol 37.

Virginia Vassilevska Williams (2012)

Multiplying Matrices Faster than Coppersmith-Winograd.

STOC’12, 887–898.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

References

David Young (1950)

On Richardson’s Method for Solving Linear Systems with PositiveDefinite Matrices.

J. Math. and Physics 32, 243–255.

Denyong Zhou, Jiayuan Huang, and Bernhard Sch’olkopf (2005)

Learning From Labeled and Unlabeled Data on a Directed Graph

International Conference on Machine Learning 1036–1043.


OliviaSimpson

Introduction


Laplacians

LaplacianSystems

Fast LinearSolvers

PreviousMethods

ST-Solver

A BetterSparsifier

RandomWalks

BoundaryConditions

Heat KernelPagerank

Variation

Boundary Solver

FutureDirections

Thank You

fast linear solvers for laplacian systems - ucsd research...

Documents