compsci 630 randomized algorithm - duke university › courses › spring16 › compsci630 ›...
TRANSCRIPT
![Page 1: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/1.jpg)
COMPSCI 630 Randomized Algorithm
Spring 2016
Rong Ge
![Page 2: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/2.jpg)
Basic Info
• Web Page: http://www.cs.duke.edu/courses/spring16/compsci630/
• Contact Me:LSRC D226Email: [email protected]
• Office Hour:Wednesdays after class at 3:00 – 5:00 pm.
![Page 3: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/3.jpg)
Basic Info
• Homework:3 or 4 problem sets, due 2 weeks after posted.Discussions are allowed, but must acknowledge.
• Piazza: Use Piazza for clarificationshttp://piazza.com/duke/spring2016/cps630
• Final Exam:There will be a take-home final for this course.
• Homework/Final will mostly involve proofs for correctness/running time of algorithms.
![Page 4: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/4.jpg)
Textbook
![Page 5: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/5.jpg)
Lecture 1: Overview
![Page 6: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/6.jpg)
Outline
• What are randomized algorithms?
• Why do we need randomness?
• How do we analyze randomized algorithms?
![Page 7: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/7.jpg)
Randomized Algorithm
• Input is still deterministic (worst-case analysis).
• Output is “correct” with probability at least 2/3for every input.
Input Algorithm Output
![Page 8: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/8.jpg)
Example 1
• Randomized quicksort
• Randomly pick a pivot, partition into two parts.
• Read intro in first Chapter for analysis
1 6 3 2 9 8 7 5 4 10
1 6 3 2 5 4 9 8 7 10
![Page 9: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/9.jpg)
Example 2
• Find an number that’s within top 50% in an array.
• Randomly pick two numbers, output the larger one
• Probability of Failure = 1
2×
1
2=
1
4!
1 6 3 2 9 8 7 5 4 10
![Page 10: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/10.jpg)
Average Case Analysis
• Input is randomly generated (by nature).
• Output is “correct” for most inputs.
Input Algorithm Output
![Page 11: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/11.jpg)
Average vs. Randomized Algorithm
• Average case is usually easier(If there is a randomized algorithm, there is also a good algorithm for any input distribution.)
• They are sometimes related (Min-Max Principal).
• Many tools apply to both settings.
![Page 12: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/12.jpg)
Las Vegas Algorithm
• Always terminates with correct output.
• Running time is random, usually want to analyze the expected running time.
• Example: randomized quicksort1 6 3 2 9 8 7 5 4 10
1 6 3 2 5 4 9 8 7 10
![Page 13: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/13.jpg)
Monte Carlo Algorithm
• May produce incorrect outputs.
• Sometimes finish in fixed amount of time, sometimes can also have a random running time.
• Example: Finding a top 50% number in an array.
• Think: Can we transform between Las Vegas and Monte Carlo algorithms?
![Page 14: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/14.jpg)
Why is randomness useful?
![Page 15: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/15.jpg)
Randomness can…
• Avoid Worst-Case Scenarios.
• Randomized Quicksort
• Hashing
1 6 3 2 9 8 7 5 4 10
1 6 3 2 5 4 9 8 7 10
![Page 16: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/16.jpg)
Randomness can…
• Give a rough estimate very quickly.
• Monte-Carlo algorithm for computing volume.
![Page 17: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/17.jpg)
Randomness can…
• Compress data while preserving crucial information.
• Dimension reduction
• Graph sparsification
![Page 18: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/18.jpg)
Randomness can…
• Round Linear Program/Semidefinite Programs.
![Page 19: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/19.jpg)
Tools to Analyze Randomized Algorithms
![Page 20: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/20.jpg)
Tools
• Basic properties of random variables.• Linearity of expectation
• Independence
• Union bound
• Markov/Chebyshev inequalities.
• Chapters 1, 3 in book.
![Page 21: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/21.jpg)
Tools
• Concentration Bounds, Martingales
• 𝑓 𝑋 ≈ 𝔼[𝑓 𝑋 ]
• Chapter 4 in book.
![Page 22: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/22.jpg)
Tools
• Dimension Reduction
• Useful for fast algorithms with approximate solutions.
• We will see examples in many different settings.
![Page 23: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/23.jpg)
Tools
• Markov Chains
• Powerful tool for sampling/counting.
• Chapters 6, 11 in book
![Page 24: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/24.jpg)
Tools
• Complexity, derandomization.
• Is there a problem that has a polynomial time randomized algorithm but no deterministic polynomial time algorithm?
![Page 25: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/25.jpg)
What’s not covered in detail
• Game theory
• Randomization is inherent in the solution space!
• CPS 590.4 Computational Microeconomics: Game Theory, Social Choice, and Mechanism Design
• Read Chapter 2 for a short intro.
![Page 26: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/26.jpg)
What’s not covered in detail
• Online Algorithms
• Randomization is crucial in algorithms/analysis.
• CPS590 Optimization and Decision-making under Uncertainty
• Read Chapter 13 for a short intro.
![Page 27: COMPSCI 630 Randomized Algorithm - Duke University › courses › spring16 › compsci630 › ... · 2016-01-15 · Randomized Algorithms. Tools •Basic properties of random variables](https://reader034.vdocuments.mx/reader034/viewer/2022042409/5f24dfbba5ed9a18fb088e90/html5/thumbnails/27.jpg)
What’s not covered in detail
• Geometry Algorithms
• Many elegant applications of randomness.
• CPS634 Geometric Algorithms
• Read Chapter 10 for a few examples.