pycon 2015- bayesian statistics made simple
TRANSCRIPT
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
1/145
Bayesian Statistics
Made Simple Allen B. Downey
Olin College
sites.google.com/site/simplebayes
http://sites.google.com/site/simplebayeshttp://sites.google.com/site/simplebayes
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
2/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
3/145
Follow along at home
sites.google.com/site/simplebayes
http://sites.google.com/site/simplebayeshttp://sites.google.com/site/simplebayes
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
4/145
The plan
From Bayes's Theorem to Bayesian inference.
A computational framework.
ork on e!ample problems.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
5/145
Goals
By the en"# you shoul" be rea"y to$
%ork on similar problems.
%&earn more on your own.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
6/145
Think Bayes
This tutorial is base" on my
book#
Think Bayes
Bayesian Statistics in Python
ublishe" by O'(eilly )e"ia
an" a*ailable un"er aCreati*e Commons license from
thinkbayes.com
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
7/145
Bayes’s Theorem
+igh on my list of "esert islan" algorithms$
,.-ulers metho"
.Bayess theorem
0.1aplan2)eier estimation3
4ot on the list$ 56"elsincompleteness theorem.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
8/145
Probability
p7A8$ the probability that A occurs.
p7A9B8$ the probability that A occurs# gi*en that
B has occurre".
p7A an" B8 : p7A8 p7B9A8
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
9/145
Bayes's Theorem
By "efinition of con;oint probability$
p7A an" B8 : p7A8 p7B9A8 : 7,8
p7B an" A8 : p7B8 p7A9B8
-
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
10/145
Bayes's Theorem
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
11/145
Bayes's Theorem
One way to think about it$
Bayes's Theorem is an algorithm to get
from p7B9A8 to p7A9B8.
=seful if p7B9A8# p7A8 an" p7B8 are easier
than p7A9B8.
O( ...
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
12/145
Diachronic interpretation
+$ +ypothesis
D$ Data
5i*en p7+8# the probability of the hypothesisbefore you saw the "ata.
Fin" p7+9D8# the probability of the hypothesisafter you saw the "ata.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
13/145
A cookie problem
>uppose there are two bowls of cookies.Bowl ?, has ,@ chocolate an" 0@ *anilla.
Bowl ? has @ of each.
Fre" picks a bowl at ran"om# an" then picks acookie at ran"om. The cookie turns out to be
*anilla.
hat is the probability that Fre" picke" fromBowl ?,
from ikipe"ia
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
14/145
ookie problem
+$ +ypothesis that cookie came from Bowl ,.
D$ Cookie is *anilla.
5i*en p7+8# the probability of the hypothesisbefore you saw the "ata.
Fin" p7+9D8# the probability of the hypothesisafter you saw the "ata.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
15/145
Diachronic interpretation
p7+9D8 : p7+8 p7D9+8 / p7D8
p7+8$ prior
p7D9+8$ con"itional likelihoo" of the "ata
p7D8$ total likelihoo" of the "ata
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
16/145
Diachronic interpretation
p7+9D8 : p7+8 p7D9+8 / p7D8
p7+8$ prior : ,/
p7D9+8$ con"itional likelihoo" of the "ata : 0/
p7D8$ total likelihoo" of the "ata : /
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
17/145
Diachronic interpretation
p7+9D8 : 7,/870/8 / 7/8 : 0/
p7+8$ prior : ,/
p7D9+8$ con"itional likelihoo" of the "ata : 0/
p7D8$ total likelihoo" of the "ata : /
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
18/145
A little int!ition
p7+8$ prior : @E
p7+9D8$ posterior : @E
Ganilla cookie was more likely un"er +.
>lightly increases our "egree of belief in +.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
19/145
omp!tation
mf represents a robability )ass Function
)aps from possible *alues to probabilities.
Diagram by yuml.me
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
20/145
"nstall test
+ow many of you got install_test.py running
Don't try to fi! it nowH
Instea"...
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
21/145
Partner !p
%If you "on't ha*e a working en*ironment# fin"
a neighbor who "oes.
%-*en if you "o# try pair programmingH
%Take a minute to intro"uce yourself.
%Juestions Ask your partner first 7please8.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
22/145
"cebreaker
hat was your first computer
hat was your first programming language
hat is the longest time you ha*e spent fin"ing
a stupi" bug
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
23/145
Start yo!r engines
,.Kou clone" Bayes)a"e>imple# right
2.cd into that "irectory.0.>tart the ython interpreter.
$ python>>> from thinkbayes import Pmf
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
24/145
#r "Python
1.cd into Bayes)a"e>imple..>tart Iython.
0.Create a new notebook.
$ ipython notebook
from thinkbayes import Pmf
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
25/145
Pm$
from thinkbayes import Pmf
# make an empty Pmf
d6 = Pmf()
# outcomes of a six-sided die
for x in [1,,!,",,6$% d6.&et(x, 1)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
26/145
Pm$
d6.Print()
d6.'ormaie()
d6.Print()
d6.*andom()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
27/145
The Bayesian $ramework
,8 Buil" a mf that maps from each hypothesis
to a prior probability# p7+8.
8 )ultiply each prior probability by thelikelihoo" of the "ata# p7D9+8.
08 4ormaliLe# which "i*i"es through by the totallikelihoo"# p7D8.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
28/145
Prior
pmf = Pmf()
pmf.&et(+o 1+, .)
pmf.&et(+o +, .)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
29/145
%pdate
p(/ania 0 o 1) = !"
p(/ania 0 o ) = "
pmf.2ut(+o 1+, .3)pmf.2ut(+o +, .)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
30/145
&ormalie
pmf.'ormaie()
.6 # return 4aue is p(5)
print pmf.Prob(+o 1+).6
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
31/145
()ercise
hat if we select another cookie# an" its
chocolate
The posterior 7after the first cookie8 becomesthe prior 7before the secon" cookie8.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
32/145
()ercise
hat if we select another cookie# an" its
chocolate
pmf.2ut(+o 1+, .)pmf.2ut(+o +, .)
pmf.'ormaie()
pmf.Print()o 1 ."!
o .3!
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
33/145
S!mmary
Bayes's Theorem#
Cookie problem#
mf class.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
34/145
The dice problem
I ha*e a bo! of "ice that contains a 2si"e" "ie#
a 2si"e" "ie# an 2si"e" "ie# a ,2si"e" "ie
an" a @2si"e" "ie.
>uppose I select a "ie from the bo! at ran"om#
roll it# an" get a . hat is the probability that I
rolle" each "ie
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
35/145
*ypothesis s!ites
A suite is a mutually e!clusi*e an" collecti*ely
e!hausti*e set of hypotheses.
(epresente" by a &uite that maps
hypothesis M probability.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
36/145
S!ite
cass &uite(Pmf)%
*epresents a suite of hypotheses and
their probabiities.
def 77init77(sef, hypos)% 8nitiaies the distribution.
for hypo in hypos%
sef.&et(hypo, 1)
sef.'ormaie()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
37/145
S!ite
def 9pdate(sef, data)%
9pdates the suite based on data.
for hypo in sef./aues()% ike = sef.Likelihood (data, hypo)
sef.2ut(hypo, ike)
sef.'ormaie()
self.&ikelihoo"
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
38/145
S!ite
:ikeihood is an abstract metho".
Chil" classes inherit 9pdate#
pro*i"e :ikeihood.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
39/145
+ikelihood
Outcome$ %hat is the likelihoo" of this outcome on a
si!2si"e" "ie
%On a ten2si"e" "ie%On a four2si"e" "ie
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
40/145
+ikelihood
# hypo is the number of sides on the die
# data is the outcome
cass 5ice(&uite)%
def :ikeihood(sef, data, hypo)%
# rite this method;
rite your solution in dice.py
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
41/145
+ikelihood
# hypo is the number of sides on the die
# data is the outcome
cass 5ice(&uite)%
def :ikeihood(sef, data, hypo)%
if hypo < data%
return ese%
return 1.hypo
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
42/145
Dice
# start ith eua priors
suite = 5ice([", 6, >, 1, $)
# update ith the data
suite.9pdate(6)
suite.Print()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
43/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
44/145
Dice
for ro in [>, 3, 3, , "$%
suite.9pdate(ro)
suite.Print()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
45/145
Dice
osterior "istribution$
" .
6 .
> .?
1 .>
.!>
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
46/145
S!mmary
Dice problem#
&ikelihoo" function#
>uite class.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
47/145
http$//images.forwallpaper.com/files/images//N,a/N,a0e,N//coffee2break2yet.;pg
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
48/145
Trains
The trainspotting problem$%Kou belie*e that a freight carrier operates
between ,@@ an" ,@@@ locomoti*es with
consecuti*e serial numbers.%Kou spot locomoti*e ?0,.%+ow many locomoti*es "oes the carrier
operate
)o"ify train.py to compute your answer.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
49/145
Trains
%If there are m trains# what is the chance of
spotting train ?n
%hat "oes the posterior "istribution look
like
%+ow woul" you summariLe it
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
50/145
Train
print suite.2ean()
print suite.2aximum:ikeihood()
print suite.@redibe8nter4a(?)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
51/145
Trains
%hat if we spot more trains
%hy "i" we "o this e!ample
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
52/145
Trains
%ractice using the Bayesian framework# an"
figuring out &ikelihoo"78.
%-!ample that uses sparse "ata.
%Its a non2tri*ial# real problem.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
53/145
Tanks
The 5erman tank problem.en.wikipe"ia.org/wiki/5ermanPtankPproblem
http://en.wikipedia.org/wiki/German_tank_problemhttp://en.wikipedia.org/wiki/German_tank_problem
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
54/145
Good time $or ,!estions
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
55/145
A (!ro problem
QWhen spun on edge 250 times, a Belgian one-
euro coin came up heads 140 times and tails
110. 't loo!s "ery suspicious to me,' said Barry
Blight, a statistics lecturer at the #ondon Schoolo$ %conomics. '$ the coin &ere uniased, the
chance o$ getting a result as e(treme as that
&ould e less than )*.' Q
From QThe 5uar"ianQ
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
56/145
A (!ro problem
)ac1ay asks# But do these data gi"e e"idence
that the coin is iased rather than $air
Assume that the coin has probability ( of
lan"ing hea"s.
7Forget that ( is a probabilityR ;ust think of it as
a physical characteristic.8
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
57/145
A (!ro problem
-stimation$ Base" on the "ata 7,@ hea"s# ,,@
tails8# what is (
+ypothesis testing$ hat is the probability that
the coin is fair
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
58/145
(!ro
e can use the >uite template again.
e ;ust ha*e to figure out the likelihoo"
function.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
59/145
+ikelihood
# hypo is the prob of heads (1-1)
# data is a strinA, either +B+ or +C+
cass Duro(&uite)%
def :ikeihood(sef, data, hypo)%
# one more, pease;
)o"ify euro.py to compute your answer.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
60/145
+ikelihood
# hypo is the prob of heads (1-1)
# data is a strinA, either +B+ or +C+
cass Duro(&uite)%
def :ikeihood(sef, data, hypo)%
x = hypo 1.
if data == +B+% return x
ese%
return 1-x
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
61/145
Prior
hat "o we belie*e about ( before seeing the
"ata
>tart with something simpleR we'll come backan" re*iew.
=niform prior $ any *alue of ( between @E an"
,@@E is e
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
62/145
Prior
suite = Duro(ranAe(, 11))
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
63/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
64/145
%pdate
>uppose we spin the coin once an" get hea"s.
suite.9pdate(+B+)
hat "oes the posterior "istribution look like
+int$ what is p7!:@E 9 D8
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
65/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
66/145
%pdate
>uppose we spin the coin again# an" get hea"s
again.
suite.9pdate(+B+)
hat "oes the posterior "istribution look like
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
67/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
68/145
%pdate
>uppose we spin the coin again# an" get tails.
suite.9pdate(+C+)
hat "oes the posterior "istribution look like
+int$ what's p7!:,@@E 9 D8
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
69/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
70/145
%pdate
After ,@ spins# N hea"s an" 0 tails$
for outcome in +BBBBBBBCCC+%
suite.9pdate(outcome)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
71/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
72/145
%pdate
An" finally# after ,@ hea"s an" ,,@ tails$
e4idence = +B+ E 1" F +C+ E 11
for outcome in e4idence% suite.9pdate(outcome)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
73/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
74/145
Posterior
%4ow what%+ow "o we summariLe the information in the
posterior >uite
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
75/145
Posterior
5i*en the posterior "istribution# what is the
probability that ! is @E
suite.Prob()
An" the answer is... @.@,
+mm. )aybe that's not the right
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
76/145
Posterior
+ow about the most likely *alue of !
pmf.2aximum:ikeihood()
An" the answer is E.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
77/145
Posterior
Or the e!pecte" *alue
suite.2ean()
An" the answer is .E.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
78/145
Posterior
Cre"ible inter*al
suite.@redibe8nter4a(?)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
79/145
Posterior
The th percentile is ,.
The th percentile is ,.
These *alues form a @E cre"ible inter*al.
>o can we say$ QThere's a @E chance that ( is
between , an" ,Q
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
80/145
Fre,!entist response
Thank you smbc2comics.com
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
81/145
Bayesian response
Kes# ( is a ran"om *ariable#
Kes# 7,# ,8 is a @E cre"ible inter*al#
Kes# ( has a @E chance of being in it.
ro$ Bayesian stats are amenable to "ecision
analysis.
Con$ The prior is sub;ecti*e.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
82/145
The prior is s!b-ecti.e
(emember the prior
e chose it pretty arbitrarily# an" reasonable
people might "isagree.
Is ( as likely to be ,E as @E
5i*en what we know about coins# I "oubt it.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
83/145
Prior
+ow shoul" we capture backgroun" knowle"ge
about coins
Try a triangle prior.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
84/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
85/145
Posterior
hat "o you think the posterior "istributions
look like
I was going to put an image here# but then I5oogle" QposteriorQ. 4e*er min".
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
86/145
Swamp the prior
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
87/145
Swamp the prior
ith enough "ata#reasonable people con*erge.
But if any p7+i8 : @# no "ata
will change that.
Swamp the prior
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
88/145
Swamp the prior
riors can be arbitrarily low#but a*oi" @.
>ee wikipe"ia.org/wiki/
Cromwell'sPrule
eseech you, in the o&els
o$ /hrist, thin! it possile thatyou may e mista!en.
http://en.wikipedia.org/wiki/%20Cromwell's_rulehttp://en.wikipedia.org/wiki/%20Cromwell's_rulehttp://en.wikipedia.org/wiki/%20Cromwell's_rulehttp://en.wikipedia.org/wiki/%20Cromwell's_rule
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
89/145
S!mmary o$ estimation
,.Form a suite of hypotheses# +i.
.Choose prior "istribution# p7+i8.
0.Compute likelihoo"s# p7D9+i8.
.Turn off brain.
.Compute posteriors# p7+i9D8.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
90/145
*ypothesis testing
(emember the original
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
91/145
*ypothesis testing
D is e*i"ence in fa*or of + if
p7+9D8 S p7+8
which is true if
p7D9+8 S p7D9+8or e
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
92/145
*ypothesis testing
This term
p7D9+8 / p7D9+8
is calle" the likelihoo" ratio# or Bayes factor.
It measures the strength of the e*i"ence.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
93/145
*ypothesis testing
F$ hypothesis that the coin is fair
B$ hypothesis that the coin is biase"
p7D9F8 is easy.p7D9B8 is har" because B is un"erspecifie".
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
94/145
Bogosity
Tempting$ we got ,@ hea"s out of @ spins#
so B is the hypothesis that ( : ,@/@.
But#,.Doesn't seem right to use the "ata twice.
.By this process# almost any "ata woul" be
e*i"ence in fa*or of B.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
95/145
/e need some r!les
,.Kou ha*e to choose your hypothesis before
you see the "ata.
.Kou can choose a suite of hypotheses# but
in that case we a*erage o*er the suite.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
96/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
97/145
+ikelihood
def G4eraAe:ikeihood(suite, data)% tota =
for hypo, prob in suite.8tems()%
ike = suite.:ikeihood(data, hypo) tota F= prob E ike
return tota
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
98/145
*ypothesis testing
F$ hypothesis that ( : @E.
B$ hypothesis that ( is not @E# but might be
any other *alue with e
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
99/145
Prior
fair = Duro()
fair.&et(, 1)
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
100/145
Prior
bias = Duro()
for x in ranAe(, 11)%
if x ;= %
bias.&et(x, 1) bias.'ormaie()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
101/145
Bayes $actor
data = 1", 11
ike7fair = G4eraAe:ikeihood(fair, data)
ike7bias = G4eraAe:ikeihood(bias, data)
ratio = ike7bias ike7fair
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
102/145
*ypothesis testing
(ea" euro2.py.
4otice the new representation of the "ata# an"
correspon"ing &ikelihoo" function.
(un it an" interpret the results.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
103/145
*ypothesis testing
An" the answer is$
p7D9B8 : . U ,@2N
p7D9F8 : . U ,@2N
&ikelihoo" ratio is about @.N.
>o this "ataset is e*i"ence against B.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
104/145
Fair comparison0
%)o"ify the co"e that buil"s biasR try out a"ifferent "efinition of B an" run again.
bias = Duro()
for x in ranAe(, "?)% bias.&et(x, x)
for x in ranAe(1, 11)%
bias.&et(x, 1-x)
bias.'ormaie()
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
105/145
oncl!sion
%The Bayes factor "epen"s on the "efinitionof B.
%Depen"ing on what Vbiase"W means# the"ata might be e*i"ence for or against B.
%The e*i"ence is weak either way 7between
@. an" 8.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
106/145
S!mmary
-uro problem#Bayesian estimation#
Bayesian hypothesis testing.
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
107/145
h t t
p $ / / n a o l i t o . " e * i a n t a r t . c o m / a r t / T e a 2 T i m e 2 0 , @ D 0 F
N N
/ord problem $or geeks
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
108/145
p g
A&IC-$ hat "i" you get on the math >AT
BOB$ N@
A&IC-$ Oh# well I got a N@. I guess that means I'm
smarter than you.
4A((ATO($ (eally hat is the probability that Alice is
smarter than Bob
Ass!me1 de$ine1 ,!anti$y
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
109/145
1 1 , y
Assume$ each person has some probability# ( # ofanswering a ran"om >AT
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
110/145
y
Treat ( as a ran"om tart with a prior "istribution.
=p"ate it.
Compare posterior "istributions.
Prior0
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
111/145
Distribution of raw scores.
+ikelihood
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
112/145
def Likelihood(self, data, hypo) ! " hypo score " data ra# " self.e!am.e%erse(score)
yes, no " ra#, self.e!am.ma!_score & ra# like " !''yes ' (1&!)''no return like
Posterior
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
113/145
Pm$ProbGreater
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
114/145
def PmfProbreater(pmf1, pmf2) ***eturns the prob that a %alue from pmf1
is +reater than a %alue from pmf2.***
Pm$ProbGreater
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
115/145
def PmfProbreater(pmf1, pmf2) ***eturns the prob that a %alue from pmf1
is +reater than a %alue from pmf2.***
terate throu+h all pairs of %alues. -heck #hether the %alue from pmf1 is +reater. dd up total probability of successful pairs.
Pm$ProbGreater
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
116/145
def PmfProbreater(pmf1, pmf2)
for !1, p1 in pmf1.tems() for !2, p2 in pmf2.tems()
/ 0LL 3 45
Pm$ProbGreater
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
117/145
def PmfProbreater(pmf1, pmf2)
total " 6.6 for !1, p1 in pmf1.tems()
for !2, p2 in pmf2.tems() if !1 > !2 total 7" p1 ' p2
return total
And the answer is222
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
118/145
Alice$ N@
Bob$ N@
robability that Alice isQsmarterQ$ ,E
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
119/145
Modeling
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
120/145
Modeling
%This result is base" on the simplification that
all >AT
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
121/145
Modeling
%For most real worl" problems# there are
se*eral reasonable mo"els.
%The best choice "epen"s on your goals.
%)o"eling errors often "ominate.
Modeling
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
122/145
Modeling
Therefore$
%Don't mistake the map for the territory.
%Don't sweat appro!imations smaller thanmo"eling errors.
%Iterate.
3ecess4
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
123/145
h t t p $ / / i m a g e s . w
i k i a . n o
c o o k i e . n e t /PP
c b @ , @ , , F @ , 0 C @ N / r e c e s s
/ i m a g e s
/ B / B c / ( e c e s sP
' i cP
f o rP
t h eP
I n t e r n e t . p n g
ase st!dy
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
124/145
ase st!dy
roblem$ stu"ents sign up to participate in acommunity ser*ice pro;ect. >ome fraction#
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
125/145
ase st!dy
If we can get a smaller sample of stu"entswhere we know who participate" an" who
reporte"# we can use that to impro*e the
estimates of < an" r.
An" we can use that to compute the posterior
"istribution of the number of stu"ents who
participate".
.ol!nteer2py
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
126/145
probs " numpy.linspace(6, 1, 161)
hypos " 89 for : in probs for r in probs hypos.append((:, r))
suite " ;olunteer(hypos)
.ol!nteer2py
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
127/145
/ students #ho si+ned up and reported data " 1
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
128/145
class ;olunteer(thinkbayes.3uite)
def Likelihood(self, data, hypo) if len(data) "" 2 return self.Likelihood1(data, hypo) elif len(data) "" ? return self.Likelihood2(data, hypo) else
raise ;alue@rror()
.ol!nteer2py
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
129/145
def Likelihood1(self, data, hypo) :, r " hypo p " : ' r si+ned_up, reported " data yes " reported no " si+ned_up & reported
like " p''yes ' (1&p)''no
return like
.ol!nteer2py
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
130/145
def Likelihood2(self, data, hypo)
:, r " hypo si+ned_up, participated, reported " data yes " participated no " si+ned_up & participated
like1 " :''yes ' (1&:)''no
yes " reported no " participated & reported
like2 " r''yes ' (1&r)''no
return like1 ' like2
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
131/145
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
132/145
S!mmary
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
133/145
S!mmary
%The Bayesian approach is a "i*i"e an"con
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
134/145
Think Bayes
This tutorial is base" on mybook#
Think Bayes
Bayesian Statistics ade Simple
ublishe" by O'(eilly )e"ia
an" a*ailable un"er a
Creati*e Commons license fromthinkbayes.com
ase st!dies
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
135/145
ase st!dies
%-uro%>AT%(e" line
%rice is (ight%Boston Bruins%aintball%Gariability hypothesis
%1i"ney tumor growth%5eiger counter %=nseen species
Think Stats
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
136/145
Think Stats
Kou might also likeThink Stats, 2nd edition
%(ploratory ata nalysis
ublishe" by O'(eilly )e"ia
an" a*ailable un"er a
Creati*e Commons license from
thinkstats.com
More reading
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
137/145
More reading
)ac1ay#n$ormation +heory, n$er
ence, and #earning lgori
thmsFree DF.
More reading
http://www.inference.phy.cam.ac.uk/itila/http://www.inference.phy.cam.ac.uk/itila/http://www.inference.phy.cam.ac.uk/itila/http://www.inference.phy.cam.ac.uk/itila/http://www.inference.phy.cam.ac.uk/itila/http://www.inference.phy.cam.ac.uk/itila/
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
138/145
More reading
Da*i"son2ilon#Bayesian ethods
$or 3ac!ers
On 5ithub.
More reading 5not $ree6
https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackershttps://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackershttps://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackershttps://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
139/145
More reading 5not $ree6
+owson an" =rbach# Scienti$ic easoning +heBayesian pproach
>i*ia# ata nalysis Bayesian +utorial
5elman et al# Bayesian ata nalysis
/here does this $it0
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
140/145
/here does this $it0
=sual approach$% Analytic "istributions.%)ath.
%)ulti"imensional integrals.%4umerical metho"s 7)C)C8.
/here does this $it0
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
141/145
/here does this $it0
roblem$%+ar" to get starte".%+ar" to "e*elop solutions incrementally.
%+ar" to "e*elop un"erstan"ing.
My theory
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
142/145
My theory
%>tart with non2analytic "istributions.%=se backgroun" information to choose
meaningful priors.
%>tart with brute2force solutions.%If the results are goo" enough an" fast
enough# stop.%Otherwise# optimiLe 7where analysis is one
kin" of optimiLation8.%=se your reference implementation for
regression testing.
&eed help0
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
143/145
&eed help0
I am always looking for interesting pro;ects.%>abbatical [une @, to August @,.
Thank yo!4
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
144/145
Thank yo!4
lease fill out the sur*ey at
tinyurl.com/bayes@@
http://tinyurl.com/bayes0409http://tinyurl.com/bayes0409
-
8/18/2019 PyCon 2015- Bayesian Statistics Made Simple
145/145