cs2210 discrete structures
TRANSCRIPT
CS2210DiscreteStructuresDiscreteProbability
Fall2017SukumarGhosh
SampleSpace
DEFINITION.ThesamplespaceSofanexperimentistheset
ofpossibleoutcomes.Anevent E isasubset ofthesample
space.
Whatisprobability?
Probabilitydistribution
Consideranexperimentwheretherearen possibleoutcomesx1,x2,x3,x4,…xn .Then
1.0≤p(xi)≤1(1≤i<n)2.p(x1)+p(x2)+p(x3)+p(x4)+…+p(xn)=1
Youcantreatp asafunction thatmapsthesetofalloutcomestothesetofrealnumbers.Thisiscalledtheprobabilitydistributionfunction.
Probabilityofindependentevents
• WhentwoeventsEandFareindependent,theoccurrenceofonegivesnoinformationabouttheoccurrenceoftheother.
• Theprobabilityoftwoindependenteventsp(E∩F)=p(E).p(F)
Exampleofdice
Whatistheprobabilityoftwo1’s ontwosix-sideddice?
ExamplefromCardgames
Thereare(13x 4)=52cardsinapack
Pokergame:Royalflush
Moreonprobability
Probabilityoftheunionofevents
Example
Whenisgamblingworth?
Disclaimer.Thisisastatisticalanalysis,notamoralorethicaldiscussion
Powerballlottery
Disclaimer.Thisisastatisticalanalysis,notamoralorethicaldiscussion
ConditionalProbability
Youareflippingacoin3times.Thefirstflipisatail.Giventhis,whatistheprobabilitythatthe3flipsproduceanoddnumberoftails?
DealswiththeprobabilityofaneventEwhenanothereventFhasalreadyoccurred.TheoccurrenceofFactuallyshrinksthesamplespace.
GivenF,theprobabilityofEisp(E|F)=p(E ⋂ F)/p(F)
ConditionalProbability
SamplespaceS={TTT,THH,THT,TTH,HTT,HHH,HHT,HTH}F={TTT,THH,THT,TTH}(thereducedsamplespace)E={TTT,THH} {thetargeteventset)
p(E ⋂ F)=2/8,p(F)=4/8.
Sop(E|F)=p(E ⋂ F)/p(F)=1/2
ExampleofConditionalProbability
Whatistheprobabilitythatafamilywithtwochildrenhastwoboys,giventhattheyhaveatleastoneboy?
F={BB,BG,GB}E={BB}
Ifthefourevents{BB,BG,GB,GG}areequallylikely,then
p(F)=¾,andp(E ⋂ F)=¼
Sotheansweris¼dividedby¾=1/3
MontyHall3-doorPuzzle
Whatisbehindthedoors?
WarmupProblem1.Asequenceof10bitsisrandomlygenerated.Whatistheprobabilitythatatleastoneofthesebitsis0?
Answer.Probabilitythatnoneofthesebitsis0is1/210
So,theprobabilitythatatleastoneofthesebitsis0is(1-1/210)=1023/1024
WarmupProblem2.Findtheprobabilityofselectingnoneofthecorrectsixintegers inalottery,(wheretheorderinwhichtheseintegersareselecteddoesnotmatter)fromthepositiveintegers1-40?
Answer.Thenumberofwaysofselectingallwrongnumbersisthenumberofwaysofselectingsixnumbersfromthe34incorrectnumbers.ThereareC(34,6)waystodothis.SincethereareC(40,6)waystochoosenumbersintotal,theprobabilityofselectingnoneofthecorrectsixintegersis
C(34,6)/C(40,6)
Bernoullitrials
Anexperimentwithonlytwooutcomes(like0,1or
T,F)iscalledaBernoullitrial.Manyproblemsneed
tocomputetheprobabilityofexactlyk successes
whenanexperimentconsistsofn independent
Bernoullitrials.
Bernoullitrials
Example.Acoinisbiased sothattheprobability
ofheadsis2/3.Whatistheprobabilitythat
exactlyfourheadscomeupwhenthecoinis
flippedexactlyseventimes?
BernoullitrialsThenumberofways4-out-of-7flipscanbeheadsisC(7,4).
HHHHTTTTHHTHHTTTTHHHH
Eachflipisanindependentflips.Foreachsuchpattern,the
probabilityof4heads(and3tails)=(2/3)4.(1/3)3. So,inall,the
probabilityofexactly4headsisC(7,4).(2/3)4.(1/3)3=560/2187
TheBirthdayProblemTheproblem.Whatisthesmallestnumberofpeoplewhoshouldbeina
roomsothattheprobabilitythatatleasttwoofthemhavethesamebirthdayisgreaterthan½?
13 2
Considerpeopleenteringtheroomoneafteranother.Assumingbirthdaysarerandomlyassigneddates,theprobabilitythatthesecondpersonhasthesamebirthdayasthefirstoneis1- 365/366
Probabilitythatthirdpersonhasthesamebirthdayasanyoneofthepreviouspersonsis1– 364/366x 365/366
TheBirthdayProblemContinuinglikethis,probabilitythatthenth personhasthesamebirthdayas
oneofthepreviouspersonsis1– 365/366x 364/366x …x (367–n)/366
13 2
Solvetheequationsothatforthenth person,thisprobabilityexceeds½.Youwillgetn =23
Alsosometimesknownasthebirthdayparadox.
RandomvariablesDEFINITION.Arandomvariableisafunction fromthesample
spaceofanexperimenttothesetofrealnumbers
Note.Arandomvariableisafunction,notavariableJExample.Acoinisflippedthreetimes.LetX(t)betherandom
variablethatequalsthenumberofheadsthatappearwhentheoutcomeist.Then
X(HHH)=3X(HHT)=X(HTH)=X(THH)=2X(TTH)=X(THT)=X(HTT)=1X(TTT)=0
ExpectedValueInformally,theexpectedvalueofarandomvariableisitsaverage
value.Like,“whatistheaveragevalueofaDie?”
DEFINITION.TheexpectedvalueofarandomvariableX(s)isequalto∑s∈S p(s)X(s)
EXAMPLE1.ExpectedvalueofaDieEachnumber1,2,3,4,5,6occurswithaprobability1/6.Sotheexpectedvalueis1/6(1+2+3+4+5+6)=21/6=7/2
ExpectedValue(continued)
EXAMPLE2.Afaircoinisflippedthreetimes.LetXbetherandomvariablethatassignstoanoutcomethenumberofheadsthatistheoutcome.WhatistheexpectedvalueofX?
Thereareeightpossibleoutcomeswhenafaircoinisflippedthreetimes.TheseareHHH,HHT,HTH,HTT,THH,THT,TTH,TTT.Eachoccurswithaprobabilityof1/8.So,
E(X)=1/8(3+2+2+2+1+1+1+0)=12/8=3/2
Geometricdistribution
Considerthis:Youflipacoinandtheprobabilityofatailisp.Thiscoinisrepeatedlyflippeduntilitcomesuptails.
Whatistheexpectednumberofflipsuntilyouseeatail?
GeometricdistributionThesamplespace{T,HT,HHT,HHHT…}isinfinite.
Theprobabilityofatail(T)isp.Probabilityofahead(H)is(1-p)Theprobabilityof(HT)is(1-p)pTheprobabilityof(HHT)is(1-p)2petc.Why?
LetXbetherandomvariablethatcountsthenumberofflipstoseeatail.Thenp (X=j)=(1-p)j-1.p
Thisisknownasgeometricdistribution.
ExpectationinaGeometricdistribution
X=therandomvariablethatcountsthenumberofflipstoseeatail.
So, andsoon
Thisinfiniteseriescanbesimplifiedto1/p.
Thus,ifp =0.2thentheexpectednumberofflipsafterwhichyouseeatailis1/0.2=5
X(T ) = 1,X(HT ) = 2,X(HHT ) = 3
E(X) = j.p(X = j)1
∞
∑ = 1.p + 2.(1− p).p + 3.(1− p)2.p + 4.(1− p)3.p + ...
Explanation
Probability Value
0.20.30.5
304020
Whatistheaveragevalue?
0.2x 30+0.3x 40+0.5x 20=28
LinearityofExpectationTheorem
IfXi,i =1,2,...,nwithnapositiveinteger,arerandomvariablesonS,andifaandb
arerealnumbers,then
(i) E(X1 +X2 +···+Xn)=E(X1)+E(X2)+···+E(Xn)
(ii) E(aX+b)=aE(X)+b.
Example1.Whatistheexpectedvalueofthesumofthenumbersthatappearwhenapairoffairdiceisrolled?
LetX1 andX2 betherandomvariablessothatX1 appearsinthefirstdieandX2 appearsontheseconddie.E(X1 +X2)=E(X1)+E(X2)=7/2+7/2=7.
UsefulFormulas
p(E) = 1− p(E)
p(E ∩ F) = p(E).p(F) (EandFaremutuallyindependent)
p(E ∪ F) = p(E)+ p(F) (EandFaremutuallyindependent)
p(E ∪ F) = p(E)+ p(F)− p(E ∩ F) (EandFarenotindependent:Inclusion- Exclusion)
p(E | F) = p(E ∩ F)
p(F) (Conditionalprobability:givenF,theprobabilityofE)
MonteCarloAlgorithms
Aclassofprobabilisticalgorithmsthatmakearandomchoiceatoneormoresteps.
Example.Hasthisbatchofn chipsnot beentestedbythechipmaker?
Randomlypickachipandtestit.Ifitisbad,thentheansweristrue(i.e.ithasnotbeentested).Ifthechipisgoodthentheansweris“don’tknow.”Thenrandomlypickanother.
Aftertheansweris“don’tknow”forKdifferentrandompicks,withyoucertifythebatchtobegood.
Whatistheprobabilityofawrongconclusion?
MonteCarloAlgorithms
Assumethatinpreviouslyuntestedbatches,theprobabilitythataparticularchipisbadhasbeenobservedtobe0.1.Sotheprobabilityofachipbeinggoodfromanuntestedbatchis(1-0.1)=0.9.
Eachtestisindependent.SotheprobabilitythatallKstepsproducetheresult“don’tknow”is0.9k.BymakingKlargeenough,onecanmaketheprobabilityassmallaspossible.Thus,ifK=66,then0.966 <0.001
ThefactthatsomanychipsareOKtellsthattheprobabilitythatthebatchhasnotbeentestedisverysmall.Sowecertifythebatch.UsuallyKisaconstant.Eachtesttakesaconstanttime– sowecancertify(ordiscard)abatchinconstanttime.
-- Certificationviarandomwitnesses-- MonteCarloalgorithmfortestingprimenumbers
Bayes’theoremThisisrelatedtoconditionalprobability.Wecanmakearealisticestimatewhensomeextrainformationisavailable.
Problem1.Therearetwoboxes.Bobfirstchoosesoneofthetwoboxesatrandom.Hethenselectsoneoftheballsinthisboxatrandom.
IfBobhasselectedaredball,whatistheprobabilitythatheselectedaballfromthefirstbox?
(Seepage469ofyourtextbook)
Bayes’theoremLetE= Bobchosearedball.SoE’=BobchoseagreenballF=BobchosefromBox1.SoF’ =BobchosefromBox2Wehavetocomputep(F|E)
p(E|F)=7/9,p(E|F’)=3/7
Wehavetofind
p(F)=p(F’)=1/2
p(F | E) = p(F∩E)p(E)
p(E∩F) = p(E | F).p(F) = (7 / 9).(1 / 2) = 7 /18
p(E∩F ' ) = p(E | F ' ).p(F ' ) = (3 / 7).(1 / 2) = 3 /14
Bayes’theorem
p(E) = p(E∩F)+ p(E∩F ' ) == 7 /18 + 3 /14 = 38 / 63
p(F | E) = p(F∩E)p(E)
= 7 /1838 / 63
= 4976
ThisistheprobabilitythatBobchosetheballfromBox1
Bayes’theoremLetEandFbeeventsfromasamplespaceSsuchthatp(E)≠0andp(F)≠0.Then
p(F | E) = p(E | F).p(F)p(E | F)p(F)+ p(E | F).p(F)
cComputethis
cgiven
Bayes’theorem
1. Supposethatonepersonin100,000hasaparticularrarediseaseforwhichthereisafairlyaccuratediagnostictest.
2.Thistestiscorrect99.0%ofthetimewhengiventoapersonselectedatrandomwhohasthedisease;
3.Thetestiscorrect99.5%ofthetimewhengiventoapersonselectedatrandomwhodoesnothavethedisease.
Findtheprobabilitythatapersonwhotestspositiveforthediseasereallyhasthedisease.(Seepage471ofyourtextbook)
Problem2
Bayes’theorem
ü 1in100,000hastheraredisease (1)ü Thistestis99.0%correctifactuallyinfected; (2)ü Thetestis99.5%correctifnotinfected (3)
LetF=eventthatarandomlychosenpersonhasthediseaseandE=eventthatarandomlychosenpersontestspositive
So,p(F)=0.00001,p(F’)=0.99999 {from(1)}Also,p(E|F)=0.99,andp(E’|F)=1- 0.99=0.01 {from(2)}Alsop(E’|F’)=0.995,andp(E|F’)=1- 0.995=0.005{from(3)}
Now,plugintoBayes’theorem.
Bayes’theorem
p(F | E) = p(E | F).p(F)p(E | F)p(F)+ p(E | F).p(F)
=
0.99 × 0.000010.99 × 0.00001+ 0.005 × 0.99999
! 0.002
So,theprobabilitythataperson“whotestspositiveforthedisease”reallyhasthediseaseisonly0.2%