lecture 13a: probabilistic method - hacettepe
TRANSCRIPT
BBM 205 Discrete MathematicsHacettepe University
Lecture 13a: Probabilistic Method
Lale Ãzkahya
Resources:Kenneth Rosen, ᅵDiscrete Mathematics and App.ᅵ
http://www.math.caltech.edu/ 2015-16/2term/ma006b
2/4/2016
2
Independent Events
Two events ðŽ, ðµ â Ω are independent ifPr ðŽ â© ðµ = Pr ð â Pr ð .
Example. We flip two fair coins.
⊠Let ðð,ð be the elementary event that coin A
landed on ð and coin ðµ on ð, where ð, ð â {â, ð¡}. Each of the four events has a probability of 0.25.
⊠The event where coin ðŽ lands on heads is
ð = ðâ,ð¡, ðâ,â . For ðµ it is ð = ðð¡,â, ðâ,â .
⊠The events are independent since
Pr ð and ð = Pr ðâ,â = 0.25 = Pr ð â Pr ð .
(Discrete) Uniform Distribution
In a uniform distribution we have a set Ωof elementary events, each occurring with
probability 1
Ω.
⊠For example, when flipping a fair die, we have a uniform distribution over the six possible results.
2/4/2016
3
Union Bound
For any two events ðŽ, ðµ, we havePr ðŽ ⪠ðµ = Pr ðŽ + Pr ðµ â Pr ðŽ â© ðµ .
This immediately implies that Pr ðŽ ⪠ðµ †Pr ðŽ + Pr ðµ ,
where equality holds iff ðŽ, ðµ are disjoint.
Union bound. For any finite set of events ðŽ1, ⊠, ðŽð, we have
Pr âªð ðŽð â€
ð
Pr ðŽð .
Recall: Ramsey Numbers
ð ð, ð is the smallest number ð such that each blue-red edge coloring of ðŸðcontains a monochromatic ðŸð.
Theorem. ð ð, ð > 2ð/2.
⊠We proved this in a previous class.
⊠Now we provide another proof, using probability.
2/4/2016
4
Probabilistic Proof
For some ð, we color the edges of ðŸð.
⊠Each edge is independently and uniformlycolored either red or blue.
⊠For any fixed set ð of ð vertices, the probability that it forms a monochromatic ðŸð
is 21âð2 .
⊠There are ðð possible sets of ð vertices. By
the union bound, the probability that there is a monochromatic ðŸð is at most
ð
21âð2 =ðð 21âð2 .
Proof (cont.)
For some ð, we color the edges of ðŸð.
⊠Each edge is colored blue with probability of 0.5, and otherwise red.
⊠The probability for a monochromatic ðŸð is
â€ðð 21âð2 .
⊠If ð †2ð/2, this probability is smaller than 1.
⊠In this case, the probability that we do not have any monochromatic ðŸð is positive, so
there exists a coloring of ðŸð with no such ðŸð.
2/4/2016
5
Non-Constructive Proofs
We proved that there exists a coloring of ðŸð with no monochromatic ðŸð, but we
have no idea how to find this coloring.
Such a proof is called non-constructive.
The probabilistic method often proves the existence of objects with surprising properties, but we still have no idea how they look like.
A Tournament
We have ð people competing in thumb wrestling.
⊠Every pair of contestants compete once.
⊠How can we decide who the overall winner is?
We build a directed graph:
⊠A vertex for every participant.
⊠An edge between every two vertices, directed from the winner to the loser.
⊠An orientation of ðŸð is called a tournament.
2/4/2016
6
The King of the Tournament
The winner can be the vertex with the maximum outdegree (the contestant winning the largest number of matches), but it might not be unique.
A king is a contestant ð¥ such that for every other contestant ðŠ either ð¥ â ðŠ or there exists ð§ such that ð¥ â ð§ â ðŠ.
Theorem. Every tournament has a king.
Proof
ð·+(ð£) â the number of vertices reachable from ð£ by a path of length †2.
Let ð£ be a vertex that maximizes ð·+ ð£ .
⊠Assume for contradiction that ð£ is not a king.
⊠Then there exists ð¢ such that ð¢ â ð£ and there is no path of length two from ð£ to ð¢.
⊠That is, for every ð€ such that ð£ â ð€, we also have ð¢ â ð€.
⊠But this implies that ð·+ ð¢ ⥠ð·+ ð£ + 1, contradicting the maximality of ð£!
2/4/2016
7
The ðð Property
We say that a tournament ð has the ððproperty if for every subset ð of ðparticipants, there exists a participant that won against everyone in ð.
⊠Formally, this is an orientation of ðŸð, such that for every subset ð of ð vertices there exists a vertex ð£ â ð â ð with an edge from ð£to every vertex of ð.
Example. A tournament with the ð1 property.
Tournaments with the ðð Property
Theorem. If ðð1 â 2âð
ðâð< 1 then
there is a tournament on ð vertices with the ðð property.
Proof.
⊠For some ð satisfying the above, we randomly orient ðŸð = ð, ðž , such that the orientation of every ð â ðž is chosen uniformly.
⊠Consider a subset ð â ð of ð vertices. The probability that a given vertex ð£ â ð â ð does not beat all of ð is 1 â 2âð.
2/4/2016
8
Proof (cont.)
Consider a subset ð â ð of ð vertices. The probability that a specific vertex ð£ â ð â ð does not beat all of ð is 1 â 2âð.
ðŽð â the event of ð not being beat by any vertex of ð â ð.
We have Pr ðŽð = 1 â 2âð ðâð, since we ask
for ð â ð independent events to hold.
By the union bound, we have
Pr ðâðð =ð
ðŽð †ðâðð =ð
Pr ðŽð =ðð 1 â 2
âð ðâð < 1.
Completing the Proof
ðŽð â the event of ð not being beat by any vertex of ð â ð.
we have
Pr ðâðð =ð
ðŽð < 1.
That is, there is a positive probability that every subset ð â ð of size ð is beat by some vertex of ð â ð. So such a tournament exists.
2/4/2016
9
Which NBA Player is Related to Mathematics?
Michael Jordan
LeBron James
Shaquille O'Neal
Random Variables
A random variable is a function from the set of possible events to â.
Example. Say that we flip five coins.
⊠We can define the random variable ð to be the number of coins that landed on heads.
⊠We can define the random variable ð to be the percentage of heads in the tosses.
⊠Notice that ð = 20ð.
2/4/2016
10
Indicator Random Variables
An indicator random variable is a random variable ð that is either 0 or 1, according to whether some event happens or not.
Example. We toss a fair die.
⊠We can define the six indicator variable ð1, ⊠, ð6 such that ðð = 1 iff the result of the roll is ð.
Expectation
The expectation of a random variable ð is
ðž ð =
ðâΩ
ð ð Pr ð .
⊠Intuitively, ðž ð is the expected value of ð in the long-run average value when repeating the experiment ð represents.
2/4/2016
11
Expectation Example
We roll a fair six-sided die.
⊠Let ð be a random variable that represents the outcome of the roll.
ðž ð =
ðâΩ
ð ð Pr ð =
ðâ{1,âŠ,6}
ð â 1
6= 3.5
2/4/2016
12
Linearity of Expectation
If ð is a random variable, then 5ð is a random variable with a value five times that of ð
Lemma. Let ð1, ð2, ⊠, ðð be a collection set of random variables over the same discrete probability. Let ð1, ⊠, ðð be constants. Then
ðž ð1ð1 + ð2ð2 +â¯+ ðððð =
ð=1
ð
ðððž ðð .
Fixed Elements in Permutations
Let ð be a uniformly chosen permutation of 1,2, ⊠, ð .
⊠For 1 †ð †ð, let ðð be an indicator variable that is 1 if ð is fixed by ð.
⊠ðž ðð = Pr ð ð = ð =ðâ1 !
ð!=1
ð.
⊠Let ð be the number of fixed elements in ð.
⊠We have ð = ð1 +â¯+ ðð.
⊠By linearity of expectation
ðž ð =
ð
ðž ðð = ð â 1
ð= 1.
2/4/2016
13
Hamiltonian Paths
Given a directed graph ðº = ð, ðž , a Hamiltonian path is a path that visits every vertex of ð exactly once.
⊠Major problem in theoretical computer science: Does there exist a polynomial-time algorithm for finding whether a Hamiltonian path exists in a given graph.
Undirected Hamiltonian paths
Hamiltonian Paths in Tournaments
Theorem. There exists a tournament ðwith ð players that contains at least
ð! 2â(ðâ1) Hamiltonian paths.
2/4/2016
14
Proof
We uniformly choose an orientation of the edges of ðŸð to obtain a tournament ð.
⊠There is a bijection between the possible Hamiltonian paths and the permutations of 1,2,⊠, ð . Every possible path defines a
unique permutation, according to the order in which it visits the vertices.
⊠For a permutation ð, let ðð be an indicator variable that is 1 if the path corresponding to ð exists in ð.
⊠We have ðž ðð = Pr ðð = 1 = 2â ðâ1 .
We uniformly choose an orientation of the edges of ðŸð to obtain a tournament ð.
⊠For a permutation ð, let ðð be an indicator variable that is 1 if the path corresponding to ð exists ð.
⊠We have ðž ðð = Pr ðð = 1 = 2âð+1 .
⊠Let ð be a random variable of the number of Hamiltonian paths in ð. Then ð = ððð.
ðž ð = ðž
ð
ðð =
ð
ðž ðð = ð! 2âð+1.
⊠Since this is the expected number of paths in a uniformly chosen tournament, there must is a orientation with at least as many paths.
2/9/2016
2
Reminder: Indicator Random Variables An indicator random variable is a random
variable ð that is either 0 or 1, according to whether some event happens or not.
Example. We toss a die.
⊠We can define the six indicator variable ð1, ⊠, ð6 such that ðð = 1 iff the result of the roll is ð.
Reminder: Expectation
The expectation of a random variable ð is
ðž ð =
ðâΩ
ð ð Pr ð .
⊠Intuitively, ðž ð is the expected value of ð in the long-run average value of repetitions of the experiment it represents.
2/9/2016
3
Reminder: Expectation Example
We roll a fair six-sided die.
⊠Let ð be a random variable that represents the outcome of the roll.
ðž ð =
ðâΩ
ð ð Pr ð =
ðâ{1,âŠ,6}
ð â 1
6= 3.5
Reminder: Linearity of Expectation
If ð is a random variable, then 5ð is a random variable with a value five times that of ð
Lemma. Let ð1, ð2, ⊠, ðð be a collection set of random variables over the same discrete probability. Let ð1, ⊠, ðð be constants. Then
ðž ð1ð1 + ð2ð2 +â¯+ ðððð =
ð=1
ð
ðððž ðð .
2/9/2016
4
Independent Sets
Consider a graph ðº = ð, ðž . An independent set in ðº is a subset ðâ² â ðsuch that there is no edge between any two vertices of ðâ².
Finding a maximum independent set in a graph is a major problem in theoretical computer science.
⊠No polynomial-time algorithm is known.
Warm Up
What are the sizes of the maximum independent sets in:
2 4
2/9/2016
5
Large Independent Sets
Theorem. A graph ðº = (ð, ðž) has an independent set of size at least
ð£âð
1
1 + deg ð£.
Proof. We uniformly choose an ordering for the vertices of ð = ð£1, ⊠, ð£ð .
⊠The set of vertices that appear before all of their neighbors is an independent set.
We uniformly choose an ordering for the vertices of ð = ð£1, ⊠, ð£ð .
⊠The set ð of vertices that appear before all of their neighbors is an independent set.
⊠ðð - indicator that is 1 if ð£ð â ð.
ðž ðð = Pr ðð = 1 =1
1 + degð£ð.
⊠ð â the random variable of the size of ð.
ðž ð = ðž
ð=1
ð
ðð =
ð=1
ð
ðž ðð =
ð=1
ð1
1 + deg ð£ð.
⊠There must exist an ordering for which |ð| has at least this value.
2/9/2016
6
Sum-free Sets
Consider a set ðŽ of positive integers. We say that ðŽ is sum-free if for every ð¥, ðŠ â ðŽ, we have that ð¥ + ðŠ â ðŽ(including the case where ð¥ = ðŠ).
What are large sum-free subsets of ð = 1,2, ⊠,ð ?
⊠We can take all of the odd numbers in ð.
⊠We can take all of the numbers in ð of size larger than ð/2.
Large Sum-Free Sets Always Exist
Theorem. For any set of positive integers ðŽ, there is a sum-free subset ðµ â ðŽ of
size |ðµ| â¥1
3|ðŽ|.
Proof. Consider a prime ð such that ð > ðfor every ð â ðŽ.
⊠From now on, calculations are ððð ð.
⊠Notice that if ðµ is sum-free ððð ð, it is also sum-free under standard addition.
⊠Thus, it suffices to find a large set that is sum-free ððð ð.
2/9/2016
7
Proof (cont.)
The calculations are ððð ð.
The set ð = ð/3 ,⊠, 2ð/3 is sum-free and ð ⥠ð â 1 /3.
We uniformly choose ð¥ â 1,2,⊠, ð â 1 and set ðŽð¥ = ð â ðŽ | ðð¥ â ð .
Consider ð, ð â ðŽð¥. Since ðð¥, ðð¥ â ð, we have that ð + ð ð¥ = ðð¥ + ðð¥ â ð. Thus, ð + ð â ðŽð¥, and ðŽð¥ is sum-free.
ðð â indicator that is 1 if ð â ðŽð¥.
ðž ðŽð¥ = ðž
ðâðŽ
ðð =
ðâðŽ
ðž ðð =
ðâðŽ
Pr ðð = 1 .
The set ð = ð/3 ,⊠, 2ð/3 is sum-free and ð ⥠ð â 1 /3.
We uniformly choose ð¥ â 1,2,⊠, ð â 1 and set ðŽð¥ = ð â ðŽ | ðð¥ â ð . ðŽð¥ is sum-free.
ðð â indicator that is 1 if ð â ðŽð¥.
ðž ðŽð¥ = ðâðŽ Pr ðð = 1 .
Recall: If ð¥â¢ð¥â² ððð ð then ðð¥â¢ðð¥â² ððð ð.
We thus have Pr ðð = 1 = |ð|/(ð â 1).
ðž ðŽð¥ =
ðâðŽ
Pr ðð = 1 =ðŽ ð
ð â 1⥠ðŽ1
3.
Thus, there exists an ð¥ for which ðŽð¥ ⥠ðŽ1
3.
2/9/2016
8
Which Super Villain is a Mathematician?
Austin PowersâDr. Evil
Sherlock HolmesâProfessor Moriarty
SpidermanâsDr. Octopus
More Ramsey Numbers
In the previous class, we used a basic probabilistic argument to prove
ð ð, ð > 2ð/2.
Theorem. For any integer ð > 0, we have
ð ð, ð > ð âðð 21âð2 .
2/9/2016
9
Proof
Consider a random red-blue coloring of ðŸð = (ð, ðž). The color of each edge is chosen uniformly and independently.
For every subset ð â ð of size ð, we denote by ðð the indicator that ð induces a monochromatic ðŸð. We set ð = ð =ððð .
ðž ðð = Pr ðð = 1 = 21âð2
By linearity of expectation, we have
ðž ð =
ð =ð
ðž ðð =ðð 21âð2 .
Completing the Proof
We proved that the in a random red-bluecoloring of ðŸð the expected number of monochromatic copies of ðŸð is
ð =ðð 21âð2 .
There exist a coloring with at mostðmonochromatic ðŸðâs.
By removing a vertex from each of these copies, we obtain a coloring of ðŸðâð with no monochromatic ðŸð.
2/9/2016
10
Recap
How we used the probabilistic method:
⊠Our first applications were simply about making random choices and showing that we obtain some property with non-zero probability.
⊠We moved to more involved proofs, where we use linearity of expectation to talk about the âexpectedâ result.
⊠In the previous proof, we used a two step method â first we randomly choose an object, and then we alter it. This method is called the alternation method.
Transmission Towers
Problem. A company wants to establish transmission towers in its large compound.
⊠Each tower must be on top of a building and each building must be covered by at least one tower.
⊠We are given the pairs of buildings such that a tower on one covers the other.
⊠We wish to minimize the number of towers.
2/9/2016
11
Building a graph
We build a graph ðº = ð, ðž .
⊠A vertex for every building.
⊠An edge between every pair of buildings that can cover each other.
⊠We need to find the minimum subset of vertices ðâ² â ð such that every vertex of ðhas at least one vertex of ðâ² as a neighbor.
Dominating Sets
Consider a graph ðº = ð, ðž . A dominating set of ðº is a subset ðâ² â ðsuch that every vertex of ð has at least one neighbor in ðâ².
It is not known whether there exists a polynomial-time algorithm for fining a minimum dominating set in a graph.
2/9/2016
12
Warm Up
Problem. Let ðº = ð, ðž be a graph with maximum degree ð. Give a lower bound for the size of any dominating set of ðº.
Answer.
⊠Every vertex covers itself and at most ð other vertices, so any dominating set is of size at least
ð
ð + 1.
The Case of a Minimum Degree
Theorem. Let ðº = ð, ðž be a graph with minimum degree ð. Then there exists a
dominating set of size at most ð â 1+lg ð
ð+1.
Proof. We consider a random subset ð â ð by independently taking each
vertex of ð with probability ð =lg ð
ð+1.
Let ð â ð â ð be the vertices that have no neighbors in ð.
⊠ð ⪠ð is a dominating set.
2/9/2016
13
ð â ð â a random subset formed by independently taking each vertex of ð with
probability ð =lg ð+1
ð+1.
ð â ð â ð â the vertices with no neighbors in ð.
⊠ð ⪠ð is a dominating set.
ðž ð = ð£âð Pr[ð£ â ð] = ð£ ð = ð ð .
A vertex is in ð if it is not in ð and none of its neighbors are in ð. The probability for this is at most 1 â ð ð+1.
ðž ð = ð£âð Pr[ð£ â ð] †ð£ 1 â ðð+1
= 1 â ð ð+1 ð .
Proof (cont.)
Completing the Proof
ð =lg ð+1
ð+1.
Famous inequality. 1 â ð †ðâð for any positive ð.
Thus, 1 â ð ð+1 †ðâð ð+1 =1
ð+1.
We provedðž ð| + |ð = ðž ð + ðž |ð|
†ð + 1 â ð ð+1 ð =lg ð + 1 + 1
ð + 1|ð|.
There must exist a dominating set of size.
2/9/2016
14
How to Choose the Probability?
In the previous problem, we knew to
choose ð =lg(ð+1)
ð+1. But how?
⊠When you solve a question and the choice is not uniform, first mark the probability as ð.
⊠At the end of the analysis you will obtain some expression with ð in it. Choose the value of ð that optimizes the expression.
The End: Professor Moriarty
Professor Moriarty is a mathematician.
⊠âAt the age of twenty-one he wrote a treatise upon the binomial theoremâ.
⊠So a combinatorist?!
⊠Dr. Octopus is a nuclear physicist.
⊠Dr. Evil is a medical doctor.