beating the union bound by geometric techniques raghu meka (ias & dimacs)
TRANSCRIPT
Beating the Union Bound by Geometric
TechniquesRaghu Meka (IAS & DIMACS)
“When you have eliminated the impossible, whatever remains, however improbable, must be the truth”
Union Bound
Popularized by Erdos
Probabilistic Method 101
• Ramsey graphs – Erdos
• Coding theory– Shannon
• Metric embeddings– Johnson-Lindenstrauss
• …
Beating the Union Bound
• Not always enough
• Constructive: Beck’91, …, Moser’09, …
Lovasz Local Lemma:, dependent.
Beating the Union Bound
I. Optimal, explicit -nets for Gaussians
• Kanter’s lemma, convex geometry
II. Constructive Discrepancy Minimization
• EdgeWalk: New LP rounding method
Geometric techniques“Truly” constructive
Outline
I. Optimal, explicit -nets for Gaussians
• Kanter’s lemma, convex geometry
II. Constructive Discrepancy Minimization
• EdgeWalk: New LP rounding method
Epsilon Nets• Discrete approximations• Applications: integration, comp.
geometry, …
Epsilon Nets for Gaussians
Discrete approximations of Gaussian
Explicit
Even existence not clear!
Thm: Explicit -net of size .
Nets in Gaussian space
• Optimal: Matching lower bound• Union bound: • Dadusch-Vempala’12:
10
First: Application to Gaussian Processes and
Cover Times
Gaussian Processes (GPs)
Multivariate Gaussian Distribution
Supremum of Gaussian Processes (GPs)
Given want to study
• Supremum is natural: eg., balls and bins
• Union bound: .
When is the supremum smaller?
Supremum of Gaussian Processes (GPs)
Given want to study
𝑣1
𝑣2
¿ 𝑋 𝑖
0Random
Gaussian
• Covariance matrix• More intuitive
Why Gaussian Processes?
Stochastic Processes
Functional analysis
Convex Geometry
Machine Learning
Many more!
Fundamental graph parameterEg:
Aldous-Fill 94: Compute cover time deterministically?
Cover times of Graphs
• KKLV00: approximation• Feige-Zeitouni’09: FPTAS for trees
Cover Times and GPsThm (Ding, Lee, Peres 10): O(1) det. poly.
time approximation for cover time.
Transfer to GPs Compute supremum of GP
• Ding, Lee, Peres 10: approximation• Can’t beat : Talagrand’s majorizing
measures
Question (Lee10, Ding11): PTAS for computing the supremum of GPs?
Computing the Supremum
Main Result
Thm: PTAS for computing the supremum of Gaussian processes.
Heart of PTAS: Epsilon net(Dimension reduction ala JL, use exp.
size net)
Thm: PTAS for computing cover time of bounded degree graphs.
19
Construction of Net
Construction of -net
Simplest possible: univariate to multivariate𝑘 𝑘
1. How fine a net?2. How big a net?Naïve: . Union bound!
Construction of -net
Simplest possible: univariate to multivariate𝑘 𝑘
Lem: Granularity enough.
Key point that beats union bound
Even out mass in interval .
Construction of -netThis talk: Analyze ‘step-wise’
approximator
- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿
- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿
Construction of -net
Take univariate net and lift to multivariate𝑘 𝑘
𝛾 𝛾 ℓ
Lem: Granularity enough.
- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿𝛾 𝛾 ℓ
Dimension Free Error Bounds
• Proof by “sandwiching”
• Exploit convexity critically
Thm: For , a norm,
Analysis of Error
• Why interesting? For any norm,
Def: Sym. (less peaked), if sym. convex sets K
Analysis for Univarate Case
Fact:
Proof:
- 𝛿 3 𝛿2 𝛿 4 𝛿−3 𝛿−4 𝛿 −2 𝛿
Spreading away from origin!
Def: scaled down , , pdf of .
Fact: Proof: For inward push compensates
earlier spreading.
Analysis for Univariate Case
Push mass towards origin.
Analysis for Univariate Case
𝛾 𝛾𝑢
Combining upper and lower:
𝛾 𝛾𝑢
𝑘 𝑘
𝛾
𝑘
𝛾𝑢
Kanter’s Lemma(77): and unimodal,
Lifting to Multivariate Case
Key for univariate: “peakedness”Dimension free!
Lifting to Multivariate Case
Dimension free: key point that beats union bound!
𝑘 𝑘
𝛾
𝑘
𝛾𝑢
Summary of Net Construction
1. Granularity enough
2. Cut points outside -ballOptimal -net
Outline
I. Optimal, explicit -nets for Gaussians
• Kanter’s lemma, convex geometry
II. Constructive Discrepancy Minimization
• EdgeWalk: New LP rounding method
1 2 3 4 5
Discrepancy• Subsets • Color with or - to minimize imbalance
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
1 2 3 4 51 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
3
1
1
0
1
Discrepancy Examples• Fundamental combinatorial
conceptArithmetic Progressions
Roth 64: Matousek, Spencer 96:
Discrepancy Examples• Fundamental combinatorial
conceptHalfspaces
Alexander 90: Matousek 95:
Why Discrepancy?
Complexity theory
Communication Complexity
Computational Geometry
PseudorandomnessMany more!
Spencer’s Six Sigma Theorem
• Central result in discrepancy theory.
• Tight: Hadamard• Beats union bound:
Spencer 85: System with n sets has discrepancy at most .
“Six standard deviations suffice”
Conjecture (Alon, Spencer): No efficient algorithm can find one.
Bansal 10: Can efficiently get discrepancy .
A Conjecture and a Disproof
• Non-constructive pigeon-hole proof
Spencer 85: System with n sets has discrepancy at most .
Six Sigma Theorem
• Truly constructive• Algorithmic partial coloring lemma• Extends to other settings
Main: Can efficiently find a coloring with discrepancy
New elementary geometric proof of Spencer’s result
EDGE-WALK: New LP rounding method
Outline of Algorithm
1. Partial coloring method
2. EDGE-WALK: geometric picture
Partial Coloring Method
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
• Beck 80: find partial assignment with zeros
1 -1 1 1 -1
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
1 -1 0 0 0
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
1 * 1 1 *
* 1 1 * 1
1 1 1 1 1
* * * 1 1
1 * 1 * 1
1 1 0-1
Lemma: Can do this in randomized
time.
Partial Coloring Method
Input:
Output:
Outline of Algorithm
1. Partial coloring Method
2. EDGE-WALK: Geometric picture
1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1
Discrepancy: Geometric View• Subsets
• Color with or - to minimize imbalance
1-111-1
3
1
1
0
1
3
1
1
0
1
1 2 3 4 5
1 * 1 1 ** 1 1 * 11 1 1 1 1* * * 1 11 * 1 * 1
Discrepancy: Geometric View
1-111-1
31101
1 2 3 4 5
• Vectors • Want
Discrepancy: Geometric View• Vectors
• Want
Goal: Find non-zero lattice point inside
Gluskin 88: Polytopes, Kanter’s
lemma, ... !
Claim: Will find good partial coloring.
Edge-Walk
• Start at origin• Brownian motion
till you hit a face• Brownian motion
within the face
Goal: Find non-zero lattice point in
Edge-Walk: Algorithm
Gaussian random walk in subspaces
• Subspace V, rate • Gaussian walk in V
Standard normal in V:Orthonormal basis
change
Edge-Walk Algorithm
Discretization issues: hitting faces
• Might not hit face• Slack: face hit if
close to it.
1. For
2. Cube faces nearly hit by .
Disc. faces nearly hit by .
Subspace orthogonal to
Edge-Walk: Algorithm• Input: Vectors • Parameters:
Edgewalk: Partial Coloring
Lem: For
with prob 0.1 and
Pr [𝑊𝑎𝑙𝑘h𝑖𝑡𝑠𝑎𝑑𝑖𝑠𝑐 . 𝑓𝑎𝑐𝑒 ]≪ Pr [𝑊𝑎𝑙𝑘h𝑖𝑡𝑠𝑎𝑐𝑢𝑏𝑒′ 𝑠 ]
Edgewalk: Analysis
1
100 Hit cube more often!
Discrepancy faces much farther than cube’s
Key point that beats union bound
Six Suffice
1. Edge-Walk: Algorithmic partial coloring
2. Recurse on unfixed variables
Spencer’s Theorem
Summary
I. Optimal, explicit -nets for Gaussians
• Kanter’s lemma, convex geometry
II. Constructive Discrepancy Minimization
• EdgeWalk: New LP rounding method
Geometric techniquesOthers: Invariance principle for polytopes
(Harsha, Klivans, M.’10), …
Open Problems
• FPTAS for computing supremum?
• Beck-Fiala conjecture 81? – Discrepancy for degree .
• Applications of Edgewalk rounding?
Rothvoss’13: Improvements for bin-packing!
Thank you
Edgewalk Rounding
Th: Given thresholds
Can find with 1. 2.