powerpoint presentation · 2018. 4. 6. · title: powerpoint presentation author: james lee created...
TRANSCRIPT
![Page 1: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/1.jpg)
CSE P521: Applied Algorithms
Instructor: Prof. James R. LeeTAs: Evan McCarty (head), Jeffrey Hon
Office hours: TBA
Grading: 5-6 Homeworks (60%), Project (40%)
Homework will be assigned on Thursdays; due next ThursdayThere will be a homework out tomorrow.Collaboration policy on website; all homework submitted electronically
[Prefer typeset solutions; scans of neat handwriting acceptable]
Project will be described in 3rd lecture; must work in pairs
Class e-mail list: Sign up at course site if you didn’t receive “hello” email
Discussion board: Accessible from course homepage; intended for “unsupervised” discussion of course topics
![Page 2: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/2.jpg)
CSE P521: Applied Algorithms
Instructor: Prof. James R. LeeTAs: Evan McCarty (head), Jeffrey Hon
Office hours: TBA
Expected background: discrete math (CSE 311)basic probability theory (CSE 312)undergrad algs & data structures (CSE 332)“mathematical maturity” [this is a theory course]
Course materials: There is no textbookLecture notes and supplementary reading posted on course siteSome lectures will have required preparatory reading [will send email]
Questions?
![Page 3: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/3.jpg)
what is this course about?
Modern algorithms
Inputs are huge, noisy, dynamic, incomplete, high-dimensional, arrive online
Approximation, randomization
Nuanced tradeoffs: Efficiency, profit, correctness
Tools of algorithmic analysis
Course cannot be comprehensive
Goal is exposure to a sample of key ideas, techniques, philosophies
Mathematical explanations
Prove that things work when we can
Develop a theoretical framework for understanding/designing solutions
![Page 4: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/4.jpg)
what is this course about?
HashingUniversal and perfect hashingLoad balancing, the power of two choicesStreaming algorithmsLocality sensitive hashing, high-dimensional search
Spectral algorithmsSingular-value decomposition (SVD)Principal component analysisSpectral partitioning
Linear programmingFormulating LPs; relaxations and approximationDuality theoryGradient descent
Online optimizationRegret minimizationBoosting, multiplicative weights
Algorithmic game theoryAlgorithms in the face of economic incentivesExploiting selfish agents
![Page 5: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/5.jpg)
Karger’s randomized min cuts
The Global Min-cut problem
Input: An undirected graph 𝐺 = (𝑉, 𝐸)
Output: A partition of the graph into two pieces 𝑉 = 𝑆 ∪ ҧ𝑆so the number of cut edges is minimized
Contraction operation:For an edge 𝑒 ∈ 𝐸, write 𝐺/𝑒 for the new graph formed by contracting the edge 𝑒.
⇒
![Page 6: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/6.jpg)
Karger’s randomized min cuts
Contraction operation:For an edge 𝑒 ∈ 𝐸, write 𝐺/𝑒 for the new graph formed by contracting the edge 𝑒.
⇒
![Page 7: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/7.jpg)
Karger’s randomized min cuts
![Page 8: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/8.jpg)
Karger’s randomized min cuts
How many times does the while loop execute?
𝑉 − 2 times
![Page 9: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/9.jpg)
Karger’s randomized min cuts
![Page 10: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/10.jpg)
Karger’s randomized min cuts
![Page 11: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/11.jpg)
analysis
![Page 12: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/12.jpg)
analysis
![Page 13: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/13.jpg)
analysis
Theorem: For any min-cut 𝑆, ҧ𝑆 , Karger’s algorithm returns 𝑆, ҧ𝑆 with probability at least1𝑛2
=2
𝑛 𝑛 − 1
Corollary: Any graph has at most 𝑛2
global min-cuts.
![Page 14: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/14.jpg)
analysis
Theorem: For any min-cut 𝑆, ҧ𝑆 , Karger’s algorithm returns 𝑆, ҧ𝑆 with probability at least1𝑛2
=2
𝑛 𝑛 − 1
If we run the algorithm 𝐾 times and output the smallest cut from all 𝐾 runs, the probability we fail to find a min cut is at most:
![Page 15: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/15.jpg)
analysis
Theorem: If we run the algorithm 𝐾 ≈ 𝑛2 log 𝑛 times, then
Pr find a min cut ≥ 1 − 1/𝑛2
Total running time?One run can be implemented to run in time 𝑂(𝑛2), so the total running time is 𝑂 𝑛4 log 𝑛 [pretty slow]
![Page 16: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/16.jpg)
one way to implement
For fans of undergraduate algorithms:Can use Kruskal’s algorithm for minimum spanning trees.
![Page 17: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/17.jpg)
one way to implement
For fans of undergraduate algorithms:Can use Kruskal’s algorithm for minimum spanning trees.
![Page 18: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/18.jpg)
analysis
Theorem: If we run the algorithm 𝐾 ≈ 𝑛2 log 𝑛 times, then
Pr find a min cut ≥ 1 − 1/𝑛2
Total running time?One run can be implemented to run in time 𝑂(𝑛2), so the total running time is 𝑂 𝑛4 log 𝑛 [pretty slow]
Improvement:There is an algorithm that runs in time 𝑂 𝑛2 log 𝑛 3 and finds a global min-cut with probability close to 1.
![Page 19: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/19.jpg)
EXERCISE
For any 𝑘 < 𝑛: Pr ¬𝒜1 ∧ ¬𝒜2 ∧ ⋯∧ ¬𝒜𝑘 ≥
𝑘2𝑛2
Observation:
Therefore if 𝑘 ≈ 𝑛/ 2, then the probability a min-cut 𝑆, ҧ𝑆
remains after 𝑘 steps is at least
𝑘 𝑘 − 12
𝑛 𝑛 − 12
≈
𝑛
2
2
𝑛2=1
2
![Page 20: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/20.jpg)
Karger’s randomized min cuts
![Page 21: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/21.jpg)
EXERCISE
Therefore if 𝑘 ≈ 𝑛/ 2, then the probability a min-cut 𝑆, ҧ𝑆
remains after 𝑘 steps is at least
𝑘 𝑘 − 12
𝑛 𝑛 − 12
≈
𝑛
2
2
𝑛2=1
2
So things are going better early on…
How to exploit this for a much faster algorithm?
For any 𝑘 < 𝑛: Pr ¬𝒜1 ∧ ¬𝒜2 ∧ ⋯∧ ¬𝒜𝑘 ≥
𝑘2𝑛2
Observation:
![Page 22: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/22.jpg)
Karger-Stein algorithm
Probability we find a specific min-cut 𝑆, ҧ𝑆 is given by the recurrence relation:
𝑝 𝑛 = 1 − 1 −1
2𝑝 1 +
𝑛
2
2
Running time of fastmincut:
𝑇 𝑛 = 2𝑇 1 +𝑛
2+ 𝑂 𝑛2
![Page 23: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/23.jpg)
![Page 24: PowerPoint Presentation · 2018. 4. 6. · Title: PowerPoint Presentation Author: James Lee Created Date: 1/6/2017 3:52:43 AM](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034003a588d80182f7c5fc3/html5/thumbnails/24.jpg)
after the break: hashing and sketching
⇒