Beating the Union Bound by Geometric

TechniquesRaghu Meka (IAS & DIMACS)

“When you have eliminated the impossible, whatever remains, however improbable, must be the truth”

Union Bound

Popularized by Erdos

Probabilistic Method 101• Ramsey graphs

– Erdos• Coding theory

– Shannon• Metric embeddings

– Johnson-Lindenstrauss• …

Beating the Union Bound

• Not always enough

• Constructive: Beck’91, …, Moser’09, …

Lovasz Local Lemma:, dependent.

Beating the Union Bound

I. Optimal, explicit -nets for Gaussians

• Kanter’s lemma, convex geometry

II. Constructive Discrepancy Minimization

• EdgeWalk: New LP rounding method

Geometric techniques“Truly” constructive

OutlineI. Optimal, explicit -nets for

Gaussians• Kanter’s lemma, convex geometry

II. Constructive Discrepancy Minimization

• EdgeWalk: New LP rounding method

Epsilon Nets• Discrete approximations• Applications: integration, comp.

geometry, …

Epsilon Nets for Gaussians

Discrete approximations of GaussianExplicit

Even existence not clear!

Thm: Explicit -net of size .

Nets in Gaussian space

• Optimal: Matching lower bound• Union bound: • Dadusch-Vempala’12:

10

First: Application to Gaussian Processes and

Cover Times

Gaussian Processes (GPs)

Multivariate Gaussian Distribution

Supremum of Gaussian Processes (GPs)Given want to study

• Supremum is natural: eg., balls and bins

• Union bound: .When is the supremum smaller?

Supremum of Gaussian Processes (GPs)Given want to study

𝑣1

𝑣2

¿ 𝑋𝑖

0Random

Gaussian

• Covariance matrix• More intuitive

Why Gaussian Processes?

Stochastic Processes

Functional analysis

Convex Geometry

Machine LearningMany more!

Fundamental graph parameterEg:

Aldous-Fill 94: Compute cover time deterministically?

Cover times of Graphs

• KKLV00: approximation• Feige-Zeitouni’09: FPTAS for trees

Cover Times and GPsThm (Ding, Lee, Peres 10): O(1) det. poly.

time approximation for cover time.

Transfer to GPs Compute supremum of GP

• Ding, Lee, Peres 10: approximation• Can’t beat : Talagrand’s majorizing

measures

Question (Lee10, Ding11): PTAS for computing the supremum of GPs?

Computing the Supremum

Main ResultThm: PTAS for computing the

supremum of Gaussian processes.

Heart of PTAS: Epsilon net(Dimension reduction ala JL, use exp.

size net)

Thm: PTAS for computing cover time of bounded degree graphs.

19

Construction of Net

Construction of -netSimplest possible: univariate to

multivariate𝑘 𝑘

1. How fine a net?2. How big a net?Naïve: . Union bound!

Construction of -netSimplest possible: univariate to

multivariate𝑘 𝑘

Lem: Granularity enough.Key point that beats

union bound

Even out mass in interval .

Construction of -netThis talk: Analyze ‘step-wise’

approximator

- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿

- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿

Construction of -netTake univariate net and lift to

multivariate𝑘 𝑘

𝛾 𝛾 ℓLem: Granularity enough.

- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿𝛾 𝛾 ℓ

Dimension Free Error Bounds

• Proof by “sandwiching”

• Exploit convexity critically

Thm: For , a norm,

Analysis of Error

• Why interesting? For any norm,

Def: Sym. (less peaked), if sym. convex sets K

Analysis for Univarate Case

Fact:

Proof:

- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿

Spreading away from origin!

Def: scaled down , , pdf of .

Fact: Proof: For inward push compensates earlier spreading.

Analysis for Univariate Case

Push mass towards origin.

Analysis for Univariate Case

𝛾 𝛾𝑢

Combining upper and lower:

𝛾 𝛾𝑢

𝑘 𝑘

𝛾

𝑘

𝛾𝑢

Kanter’s Lemma(77): and unimodal,

Lifting to Multivariate Case

Key for univariate: “peakedness”Dimension free!

Lifting to Multivariate Case

Dimension free: key point that beats union bound!

𝑘 𝑘

𝛾

𝑘

𝛾𝑢

Summary of Net Construction

1. Granularity enough

2. Cut points outside -ballOptimal -net

OutlineI. Optimal, explicit -nets for

Gaussians• Kanter’s lemma, convex geometry

II. Constructive Discrepancy Minimization

• EdgeWalk: New LP rounding method

1 2 3 4 5

Discrepancy• Subsets • Color with or - to minimize imbalance

1 * 1 1 ** 1 1 * 1

1 1 1 1 1

* * * 1 1

1 * 1 * 1

1 2 3 4 51 * 1 1 ** 1 1 * 1

1 1 1 1 1

* * * 1 1

1 * 1 * 1

3

1

1

0

1

Discrepancy Examples• Fundamental combinatorial

conceptArithmetic Progressions

Roth 64: Matousek, Spencer 96:

Discrepancy Examples• Fundamental combinatorial

conceptHalfspaces

Alexander 90: Matousek 95:

Why Discrepancy?Complexity theory

Communication Complexity

Computational Geometry

PseudorandomnessMany more!

Spencer’s Six Sigma Theorem

• Central result in discrepancy theory.

• Tight: Hadamard• Beats union bound:

Spencer 85: System with n sets has discrepancy at most .

“Six standard deviations suffice”

Conjecture (Alon, Spencer): No efficient algorithm can find one.

Bansal 10: Can efficiently get discrepancy .

A Conjecture and a Disproof

• Non-constructive pigeon-hole proof

Spencer 85: System with n sets has discrepancy at most .

Six Sigma Theorem

• Truly constructive• Algorithmic partial coloring lemma• Extends to other settings

Main: Can efficiently find a coloring with discrepancy

New elementary geometric proof of Spencer’s result

EDGE-WALK: New LP rounding method

Outline of Algorithm1. Partial coloring method

2. EDGE-WALK: geometric picture

Partial Coloring Method

1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

• Beck 80: find partial assignment with zeros

1 -1 1 1 -11 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

1 -1 0 0 01 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

1 1 0-1

Lemma: Can do this in randomized

time.

Partial Coloring MethodInput:

Output:

Outline of Algorithm1. Partial coloring Method

2. EDGE-WALK: Geometric picture

1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

Discrepancy: Geometric View• Subsets

• Color with or - to minimize imbalance

1-111-1

3

1101

31101

1 2 3 4 5

1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1

Discrepancy: Geometric View

1-111-1

31101

1 2 3 4 5

• Vectors • Want

Discrepancy: Geometric View• Vectors

• Want

Goal: Find non-zero lattice point inside

Gluskin 88: Polytopes, Kanter’s

lemma, ... !

Claim: Will find good partial coloring.

Edge-Walk

• Start at origin• Brownian motion

till you hit a face• Brownian motion

within the face

Goal: Find non-zero lattice point in

Edge-Walk: AlgorithmGaussian random walk in subspaces

• Subspace V, rate • Gaussian walk in V

Standard normal in V:Orthonormal basis

change

Edge-Walk AlgorithmDiscretization issues: hitting faces

• Might not hit face• Slack: face hit if

close to it.

1. For

2. Cube faces nearly hit by .

Disc. faces nearly hit by .

Subspace orthogonal to

Edge-Walk: Algorithm• Input: Vectors • Parameters:

Edgewalk: Partial Coloring

Lem: For

with prob 0.1 and

Pr [𝑊𝑎𝑙𝑘h𝑖𝑡𝑠𝑎𝑑𝑖𝑠𝑐 . 𝑓𝑎𝑐𝑒 ]≪ Pr [𝑊𝑎𝑙𝑘 h𝑖𝑡𝑠𝑎𝑐𝑢𝑏𝑒′ 𝑠 ]

Edgewalk: Analysis

1100 Hit cube more often!

Discrepancy faces much farther than cube’s

Key point that beats union bound

Six Suffice

1. Edge-Walk: Algorithmic partial coloring

2. Recurse on unfixed variables

Spencer’s Theorem

SummaryI. Optimal, explicit -nets for

Gaussians• Kanter’s lemma, convex geometry

II. Constructive Discrepancy Minimization

• EdgeWalk: New LP rounding method

Geometric techniquesOthers: Invariance principle for polytopes

(Harsha, Klivans, M.’10), …

Open Problems• FPTAS for computing supremum?

• Beck-Fiala conjecture 81? – Discrepancy for degree .

• Applications of Edgewalk rounding?Rothvoss’13: Improvements for

bin-packing!

Thank you

Edgewalk RoundingTh: Given thresholds

Can find with 1. 2.