matroid bases and matrix concentration nick harvey university of british columbia joint work with...
TRANSCRIPT
Matroid Bases andMatrix Concentration
Nick Harvey University of British Columbia
Joint work with Neil Olver (Vrije Universiteit)
Scalar concentration inequalities
Theorem: [Chernoff / Hoeffding Bound]Let Y1,…,Ym be independent, non-negative scalar random variables.Let Y=i Yi and ¹=E [ Y ]. Suppose Yi · 1 a.s. Then
Scalar concentration inequalities
Theorem: [Panconesi-Srinivasan ‘92, Dubhashi-Ranjan ‘96, etc.]Let Y1,…,Ym be negatively dependent, non-negative scalar rvs.Let Y=i Yi and ¹=E [ Y ]. Suppose Yi · 1 a.s. Then
Negative cylinder dependence: Yi 2 {0,1},
Stronger notions: negative association, determinantal distributions,strongly Rayleigh measures, etc.
Matrix concentration inequalities
Theorem: [Tropp ‘12, etc.]Let Y1,…,Ym be independent, PSD matrices of size nxn.Let Y=i Yi and M=E [ Y ]. Suppose ¹¢Yi ¹ M a.s. Then
Extensions of Chernoff Bounds
Independent Negative Dependent
Scalars Chernoff-Hoeffding
Panconesi-Srinivasan, etc.
Matrices Tropp, etc. ?
This talk: a special case of the missing common generalization, where the negatively dependent distribution is a certainrandom walk in a matroid base polytope.
Negative Dependence
Arises in many natural scenarios.
• Random spanning trees: Let Ye indicate if edge e is in tree.
Knowing that e2T decreases probability that f2T
e f
Negative Dependence
Arises in many natural scenarios.
• Random spanning trees: Let Ye indicate if edge e is in tree.
• Balls and bins: Let Yi be number of balls in bin i.
• Sampling without replacement, random permutations,random cluster models, etc.
Thin trees
A spanning tree T is ®-thin if |±T(S)| · ®¢|±G(S)| 8S
Global connectivity: K = min {|±G(S)| : ;(S(V }
Conjecture [Goddyn ’80s]: Every n-vertex graph has an®-thin tree with ®=O(1/K).
Would have deep consequences in graph theory.
S S
Cut ±(S) = { edge st : s2S, tS }
Thin trees
A spanning tree T is ®-thin if |±T(S)| · ®¢|±G(S)| 8S
Global connectivity: K = min {|±G(S)| : ;(S(V }
Theorem [Asadpour et al ‘10]: Every n-vertex graph has an®-thin spanning tree with ®= .
Uses negative dependence and Chernoff bounds.
S S
Cut ±(S) = { edge st : s2S, tS }
Asymmetric Traveling Salesman Problem[Julia Robinson, 1949]
• Let D=(V,E,w) be a weighted, directed graph.
• Goal: Find a tour sequence v1,v2,…,vk=v1 of vertices thatvisits every vertex in V at least once,has vivi+12E for every i,and minimizes total weight §1·i·k
w(vivi+1).
• Let D=(V,E,w) be a weighted, directed graph.
• Goal: Find a tour sequence v1,v2,…,vk=v1 of vertices thatvisits every vertex in V at least once,has vivi+12E for every i,and minimizes total weight §1·i·k w(vivi+1).
• Reduction [Oveis Gharan, Saberi ‘11]: If you can efficiently find an ®/K-thin spanning tree in any n-vertex graph, then you can find a tour whose weight is within O(®) of optimal.
Asymmetric Traveling Salesman Problem[Julia Robinson, 1949]
Graph Laplacians
Lbc =
0 0 0 0
0 1 -1 0
0 -1 1 0
0 0 0 0
a
b
c
d
a b c d
a
b
dc
Laplacian of edge bc
Graph Laplacians
LG = §e2E Le =
2 -1 -1
-1 2 -1
-1 -1 3 -1
-1 1
a
b
c
d
a b c d
degree of node
-1 for every edge
a
b
dc
Laplacian of graph G
Spectrally-thin trees
A spanning tree T is ®-spectrally-thin if LT ¹ ®¢LG
Effective Resistance from s to t: Rst = voltage difference when a 1-amp current source placed between s and t
Theorem [Harvey-Olver '14]: Every n-vertex graph has an®-spectrally-thin spanning tree with ®= .
Uses matrix concentration bounds. Algorithmic.
5 -1 -1 -1 -1 -14 -1 -1 -1 -1
-1 -1 6 -1 -1 -1 -1-1 5 -1 -1 -1 -1
-1 -1 -1 7 -1 -1 -1 -1-1 -1 -1 5 -1 -1-1 -1 -1 5 -1 -1
-1 -1 -1 -1 6 -1 -1-1 -1 -1 -1 -1 5
-1 -1 -1 -1 -1 -1 6
6 -1 -55 -1 -3 -1
-1 2 -18 -8
-1 2 -11 -1
-3 -1 5 -12 -1 -1
-5 -1 -1 -1 8-1 -8 -1 10
Spectrally-thin trees
A spanning tree T is ®-spectrally-thin if LT ¹ ®¢LG
Effective Resistance from s to t: Rst = voltage difference when a 1-amp current source placed between s and t
Theorem: Every n-vertex graph has an®-spectrally-thin spanning tree with ®= .
Follows from Kadison-Singer solution of MSS'13. Not algorithmic.
5 -1 -1 -1 -1 -14 -1 -1 -1 -1
-1 -1 6 -1 -1 -1 -1-1 5 -1 -1 -1 -1
-1 -1 -1 7 -1 -1 -1 -1-1 -1 -1 5 -1 -1-1 -1 -1 5 -1 -1
-1 -1 -1 -1 6 -1 -1-1 -1 -1 -1 -1 5
-1 -1 -1 -1 -1 -1 6
6 -1 -55 -1 -3 -1
-1 2 -18 -8
-1 2 -11 -1
-3 -1 5 -12 -1 -1
-5 -1 -1 -1 8-1 -8 -1 10
Asymmetric Traveling Salesman Problem
• Recent breakthrough: [Ansari, Oveis-Gharan Dec 2014]Show how to build on the O(1)-spectrally-thin tree resultto approximate optimal weight of an ATSP solution to within poly(log log n) of optimal.
• But, no algorithm to find the actual sequence of vertices!
Our Main ResultLet P½[0,1]m be a matroid base
polytope (e.g., convex hull of characteristic
vectors of spanning trees)Let A1,…, Am be PSD matrices of size
nxn.Define and Q;.There is an extreme point Â(S) of P with
Our Main ResultLet P½[0,1]m be a matroid base polytope.Let A1,…, Am be PSD matrices of size nxn.
Define and Q;.There is an extreme point Â(S) of P with
What is dependence on ®?• Easy: ® ¸ 1.5, even with n=2.• Standard random matrix theory: ® = O(log
n).• Our result: • Ideally: ®<2. This would solve Kadison-
Singer problem.• MSS ‘13: Solved Kadison-Singer, achieve ®
= O(1).
Our Main ResultLet P½[0,1]m be a matroid base polytope.Let A1,…, Am be PSD matrices of size nxn.
Define and Q;.There is an extreme point Â(S) of P with ,
Furthermore,• there is a random process that starts at any
x02Q and terminates after m steps at such a point Â(S), whp.
• each step of this process can be performed algorithmically.
• the entire process can be derandomized.
Pipage rounding[Ageev-Svirideno ‘04, Srinivasan ‘01, Calinescu et al. ‘07, Chekuri et al. ‘09]
Let P be any matroid polytope.
Given fractional xFind coordinates a and b s.t. linez x + z ( ea – eb ) stays in current face
Find two points where line leaves P
Randomly choose one of thosepoints s.t. expectation is x
Repeat until x = ÂT is integral
x is a martingale: expectation of final ÂT is original fractional x.
ÂT1
ÂT2
ÂT3
ÂT4
ÂT5
ÂT6
x
Definition: “Pessimistic Estimator”Let E µ {0,1}m be an event. Let D(x) be the product distribution on {0,1}m with expectation x.Then g : [0,1]m ! R is a pessimistic estimator for E if
Example: If E is the event { x : wT x>t }
then Chernoff bounds give the pessimistic estimator
Pessimistic estimators
Definition:A functionf : Rm ! R is concave under swaps if z ! f( x + z(ea-eb) ) is concave 8x2P, 8a, b2[m].
Example: is concave under swaps.
Pipage Rounding:Let X0 be initial point and ÂT be final point visited by pipage rounding.
Claim: If f concave under swaps then E[f(ÂT)] · f(X0). [by Jensen]
Pessimistic Estimators:
Let E be an event and g a pessimistic estimator for E.Claim: Suppose g is concave under swaps. Then Pr[ ÂT 2
E ] · g(X0).
Concavity under swaps
Matrix Pessimistic Estimators
Main Technical Result: gt,µ is concave under swaps.
Special case of Tropp ‘12: Let A1,…,Am be nxn PSD
matrices.
Let D(x) be the product distribution on {0,1}m with expectation x.Let Suppose ¹¢Ai ¹ M.
Let
Then
and .
) Tropp’s bound for independent sampling also achieved by pipage rounding
Pessimistic estimator
Our Variant of Lieb’s Theorem:
PD
Questions
• Does Tropp’s matrix concentration bound hold in a negatively dependent scenario?
• Does our variant of Lieb’s theorem have other uses?
• O(maxe Re)-spectrally thin trees exist by MSS’13.Can they be constructed algorithmically?