matroid bases and matrix concentration nick harvey university of british columbia joint work with...

25
Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit)

Upload: ruth-raven

Post on 14-Dec-2015

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Matroid Bases andMatrix Concentration

Nick Harvey University of British Columbia

Joint work with Neil Olver (Vrije Universiteit)

Page 2: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Scalar concentration inequalities

Theorem: [Chernoff / Hoeffding Bound]Let Y1,…,Ym be independent, non-negative scalar random variables.Let Y=i Yi and ¹=E [ Y ]. Suppose Yi · 1 a.s. Then

Page 3: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Scalar concentration inequalities

Theorem: [Panconesi-Srinivasan ‘92, Dubhashi-Ranjan ‘96, etc.]Let Y1,…,Ym be negatively dependent, non-negative scalar rvs.Let Y=i Yi and ¹=E [ Y ]. Suppose Yi · 1 a.s. Then

Negative cylinder dependence: Yi 2 {0,1},

Stronger notions: negative association, determinantal distributions,strongly Rayleigh measures, etc.

Page 4: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Matrix concentration inequalities

Theorem: [Tropp ‘12, etc.]Let Y1,…,Ym be independent, PSD matrices of size nxn.Let Y=i Yi and M=E [ Y ]. Suppose ¹¢Yi ¹ M a.s. Then

Page 5: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Extensions of Chernoff Bounds

Independent Negative Dependent

Scalars Chernoff-Hoeffding

Panconesi-Srinivasan, etc.

Matrices Tropp, etc. ?

This talk: a special case of the missing common generalization, where the negatively dependent distribution is a certainrandom walk in a matroid base polytope.

Page 6: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Negative Dependence

Arises in many natural scenarios.

• Random spanning trees: Let Ye indicate if edge e is in tree.

Knowing that e2T decreases probability that f2T

e f

Page 7: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Negative Dependence

Arises in many natural scenarios.

• Random spanning trees: Let Ye indicate if edge e is in tree.

• Balls and bins: Let Yi be number of balls in bin i.

• Sampling without replacement, random permutations,random cluster models, etc.

Page 8: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Thin trees

A spanning tree T is ®-thin if |±T(S)| · ®¢|±G(S)| 8S

Global connectivity: K = min {|±G(S)| : ;(S(V }

Conjecture [Goddyn ’80s]: Every n-vertex graph has an®-thin tree with ®=O(1/K).

Would have deep consequences in graph theory.

S S

Cut ±(S) = { edge st : s2S, tS }

Page 9: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Thin trees

A spanning tree T is ®-thin if |±T(S)| · ®¢|±G(S)| 8S

Global connectivity: K = min {|±G(S)| : ;(S(V }

Theorem [Asadpour et al ‘10]: Every n-vertex graph has an®-thin spanning tree with ®= .

Uses negative dependence and Chernoff bounds.

S S

Cut ±(S) = { edge st : s2S, tS }

Page 10: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Asymmetric Traveling Salesman Problem[Julia Robinson, 1949]

• Let D=(V,E,w) be a weighted, directed graph.

• Goal: Find a tour sequence v1,v2,…,vk=v1 of vertices thatvisits every vertex in V at least once,has vivi+12E for every i,and minimizes total weight §1·i·k

w(vivi+1).

Page 11: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

• Let D=(V,E,w) be a weighted, directed graph.

• Goal: Find a tour sequence v1,v2,…,vk=v1 of vertices thatvisits every vertex in V at least once,has vivi+12E for every i,and minimizes total weight §1·i·k w(vivi+1).

• Reduction [Oveis Gharan, Saberi ‘11]: If you can efficiently find an ®/K-thin spanning tree in any n-vertex graph, then you can find a tour whose weight is within O(®) of optimal.

Asymmetric Traveling Salesman Problem[Julia Robinson, 1949]

Page 12: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Graph Laplacians

Lbc =

0 0 0 0

0 1 -1 0

0 -1 1 0

0 0 0 0

a

b

c

d

a b c d

a

b

dc

Laplacian of edge bc

Page 13: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Graph Laplacians

LG = §e2E Le =

2 -1 -1

-1 2 -1

-1 -1 3 -1

-1 1

a

b

c

d

a b c d

degree of node

-1 for every edge

a

b

dc

Laplacian of graph G

Page 14: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Spectrally-thin trees

A spanning tree T is ®-spectrally-thin if LT ¹ ®¢LG

Effective Resistance from s to t: Rst = voltage difference when a 1-amp current source placed between s and t

Theorem [Harvey-Olver '14]: Every n-vertex graph has an®-spectrally-thin spanning tree with ®= .

Uses matrix concentration bounds. Algorithmic.

5 -1 -1 -1 -1 -14 -1 -1 -1 -1

-1 -1 6 -1 -1 -1 -1-1 5 -1 -1 -1 -1

-1 -1 -1 7 -1 -1 -1 -1-1 -1 -1 5 -1 -1-1 -1 -1 5 -1 -1

-1 -1 -1 -1 6 -1 -1-1 -1 -1 -1 -1 5

-1 -1 -1 -1 -1 -1 6

6 -1 -55 -1 -3 -1

-1 2 -18 -8

-1 2 -11 -1

-3 -1 5 -12 -1 -1

-5 -1 -1 -1 8-1 -8 -1 10

Page 15: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Spectrally-thin trees

A spanning tree T is ®-spectrally-thin if LT ¹ ®¢LG

Effective Resistance from s to t: Rst = voltage difference when a 1-amp current source placed between s and t

Theorem: Every n-vertex graph has an®-spectrally-thin spanning tree with ®= .

Follows from Kadison-Singer solution of MSS'13. Not algorithmic.

5 -1 -1 -1 -1 -14 -1 -1 -1 -1

-1 -1 6 -1 -1 -1 -1-1 5 -1 -1 -1 -1

-1 -1 -1 7 -1 -1 -1 -1-1 -1 -1 5 -1 -1-1 -1 -1 5 -1 -1

-1 -1 -1 -1 6 -1 -1-1 -1 -1 -1 -1 5

-1 -1 -1 -1 -1 -1 6

6 -1 -55 -1 -3 -1

-1 2 -18 -8

-1 2 -11 -1

-3 -1 5 -12 -1 -1

-5 -1 -1 -1 8-1 -8 -1 10

Page 16: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Asymmetric Traveling Salesman Problem

• Recent breakthrough: [Ansari, Oveis-Gharan Dec 2014]Show how to build on the O(1)-spectrally-thin tree resultto approximate optimal weight of an ATSP solution to within poly(log log n) of optimal.

• But, no algorithm to find the actual sequence of vertices!

Page 17: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Our Main ResultLet P½[0,1]m be a matroid base

polytope (e.g., convex hull of characteristic

vectors of spanning trees)Let A1,…, Am be PSD matrices of size

nxn.Define and Q;.There is an extreme point Â(S) of P with

Page 18: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Our Main ResultLet P½[0,1]m be a matroid base polytope.Let A1,…, Am be PSD matrices of size nxn.

Define and Q;.There is an extreme point Â(S) of P with

What is dependence on ®?• Easy: ® ¸ 1.5, even with n=2.• Standard random matrix theory: ® = O(log

n).• Our result: • Ideally: ®<2. This would solve Kadison-

Singer problem.• MSS ‘13: Solved Kadison-Singer, achieve ®

= O(1).

Page 19: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Our Main ResultLet P½[0,1]m be a matroid base polytope.Let A1,…, Am be PSD matrices of size nxn.

Define and Q;.There is an extreme point Â(S) of P with ,

Furthermore,• there is a random process that starts at any

x02Q and terminates after m steps at such a point Â(S), whp.

• each step of this process can be performed algorithmically.

• the entire process can be derandomized.

Page 20: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Pipage rounding[Ageev-Svirideno ‘04, Srinivasan ‘01, Calinescu et al. ‘07, Chekuri et al. ‘09]

Let P be any matroid polytope.

Given fractional xFind coordinates a and b s.t. linez x + z ( ea – eb ) stays in current face

Find two points where line leaves P

Randomly choose one of thosepoints s.t. expectation is x

Repeat until x = ÂT is integral

x is a martingale: expectation of final ÂT is original fractional x.

ÂT1

ÂT2

ÂT3

ÂT4

ÂT5

ÂT6

x

Page 21: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Definition: “Pessimistic Estimator”Let E µ {0,1}m be an event. Let D(x) be the product distribution on {0,1}m with expectation x.Then g : [0,1]m ! R is a pessimistic estimator for E if

Example: If E is the event { x : wT x>t }

then Chernoff bounds give the pessimistic estimator

Pessimistic estimators

Page 22: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Definition:A functionf : Rm ! R is concave under swaps if z ! f( x + z(ea-eb) ) is concave 8x2P, 8a, b2[m].

Example: is concave under swaps.

Pipage Rounding:Let X0 be initial point and ÂT be final point visited by pipage rounding.

Claim: If f concave under swaps then E[f(ÂT)] · f(X0). [by Jensen]

Pessimistic Estimators:

Let E be an event and g a pessimistic estimator for E.Claim: Suppose g is concave under swaps. Then Pr[ ÂT 2

E ] · g(X0).

Concavity under swaps

Page 23: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Matrix Pessimistic Estimators

Main Technical Result: gt,µ is concave under swaps.

Special case of Tropp ‘12: Let A1,…,Am be nxn PSD

matrices.

Let D(x) be the product distribution on {0,1}m with expectation x.Let Suppose ¹¢Ai ¹ M.

Let

Then

and .

) Tropp’s bound for independent sampling also achieved by pipage rounding

Pessimistic estimator

Page 24: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Our Variant of Lieb’s Theorem:

PD

Page 25: Matroid Bases and Matrix Concentration Nick Harvey University of British Columbia Joint work with Neil Olver (Vrije Universiteit) TexPoint fonts used in

Questions

• Does Tropp’s matrix concentration bound hold in a negatively dependent scenario?

• Does our variant of Lieb’s theorem have other uses?

• O(maxe Re)-spectrally thin trees exist by MSS’13.Can they be constructed algorithmically?