doing bayesian data analysis, chapter 5
DESCRIPTION
03/AUG/2013 @Matsuo lab, the University of Tokyo.TRANSCRIPT
![Page 1: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/1.jpg)
www.***.com
Part2: Chapter 5 Inferring a Binominal Proportion via Exact Mathematical Analysis�
Haru Negami
03/08/2013�
![Page 2: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/2.jpg)
summary�
! binomial proportion " the likelihood function
! the Bernoulli likelihood function " the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+z) " comparison ---- best model <-p(D|M)
![Page 3: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/3.jpg)
Binomial Proportion�
! dlcW�Gp�W[W ! binomial or dichotomous ! ��
" 50��m 8nKTW.#X&,�6
![Page 4: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/4.jpg)
ahbbC�
p(θ|q) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
![Page 5: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/5.jpg)
Likelihood function�
! Example : coin flipping " � y = {0,1} ex) (0:1, 1:2)
" p(y=1|θ)=f(θ)=θA;mθX�8[0,1]S7/n " p(y=0|θ) = 1-θ
" the Bernoulli distribution " p(y|θ) = θy(1-θ)(1-y)
! �MBθX��CyX��SD^C ! *'��SD^C (ΣyAp(y|θ) = 1)�
![Page 6: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/6.jpg)
Likelihood function�
! Bernoulli likelihood function " p(y|θ) = θy(1-θ)(1-y)
" y`��Bθ`��T3^C ! ��9�XθG�O^KTV7/W�`T^C ! gkela��XyKTV:�)U�`�^C
" ��9�X*'��SXUEJTV$�O^C ! p(y|θ) = θy(1-θ)(1-y)(y=i)`θS+�C
![Page 7: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/7.jpg)
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done�
![Page 8: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/8.jpg)
belief (to make a model)�
! ��o " p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ) " p(θ)Tp(y|θ)p(θ)G�N��S!I^C
! �N�`(ER50��`�\OJTG�"^C
! ��p " denominator Σyp(y|θ) G4-SH^C
" J_]`%POp(θ)` a conjugate prior for p(y|θ) TEFC
![Page 9: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/9.jpg)
belief (to make a model)�
! p(θ)A= θa(1-θ)b Xgkela��9�WYTQW���*'��9�
>
p(y|θ)×p(θ)A= θy(1-θ)(1-y) × θa(1-θ)b
= θy+a(1-θ)(1-y+b)
! JW�S1L_^*'��9�`beta distributionT�ZC�
![Page 10: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/10.jpg)
belief (to make a model)�
! beta distribution " a, bW2QWfjilcGD^ (a,b > 0) " p(θ|a,b) = beta(θ|a,b) = θ(a-1)(1-θ)(b-1)/B(a,b) ! B(a,b)Xbeta functionT�ZC
" beta distribution W normalizer ! B(a,b) = ∫01dθAθ(a-1)(1-θ)(b-1)
![Page 11: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/11.jpg)
belief (to make a model)�
! Beta Distribution�
b�
a�
![Page 12: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/12.jpg)
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done�
![Page 13: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/13.jpg)
belief in detail (prior)�
! beta distribution : beta(θ|a,b)
" two parameters ! mean :
! standard deviation :
![Page 14: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/14.jpg)
belief in detail (prior)�
! beta distribution : beta(θ|a,b) " guess the values of a and b
! from (observed) data " ex) a=b=1, a=b=4, etc…
! m = a/(a+b), n=(a+b)TO^T a = mn, b = (1-m)n
! from mean and standard deviation
<a@1 & b@1 G�EGB a<1 &/or b<1W� [
![Page 15: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/15.jpg)
ahbbC�
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
done� done�
![Page 16: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/16.jpg)
belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)
! posterior distribution : beta(θ|z+a, N-z+b)
![Page 17: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/17.jpg)
belief in detail(posterior)�! supposition : N flips, z heads ! prior distribution : beta(θ|a,b)
! posterior distribution : beta(θ|z+a, N-z+b)
one of the beauties of mathematical approach to
Bayesian inference!�
![Page 18: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/18.jpg)
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
![Page 19: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/19.jpg)
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
![Page 20: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/20.jpg)
belief in detail (updated parameters)�
! probability distribution : " prior : beta(θ|a,b)
N flips, z heads
" posterior : beta(θ|z+a, N-z+b)
! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)]
= (z+a)/(N+a+b) !!!
0� z/N� a/(a+b)�
1-α� α�
α = N/(N+a+b)TO^C�
![Page 21: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/21.jpg)
Discussion(?) Three inferential goals�
! from chapter 4 " estimating the binominal proportion " predicting Data " comparing models
![Page 22: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/22.jpg)
estimation�
! uncertainty of the prior distribution " From the posterior distribution
! HDI : the highest density interval (chapter 3)
HDI L : broad R : narrow
prior dist. L : more uncertain�
![Page 23: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/23.jpg)
estimation�
! reasonable credibility of a value concerned " From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5 credible?
ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible
![Page 24: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/24.jpg)
estimation�
! reasonable credibility of a value concerned " From the posterior distribution
! ROPE : region of practical equivalence
coin flipping
θ = 0.5 credible?
ROPE = [0.48,0.52] if 95% HDI ∩ ROPEA= = then θ is incredible
includes many extra assumptions!�
![Page 25: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/25.jpg)
prediction�
! p(y) = ∫dθp(y|θ)p(θ) <-posterior�
![Page 26: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/26.jpg)
prediction�
! p(y) = ∫dθp(y|θ)p(θ) <-posterior�
0� z/N� a/(a+b)�
1-α� α�
α = N/(N+a+b)TO^C�
![Page 27: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/27.jpg)
prediction (example 1)�
! 1st beta(θ|1,1) mean 1/2 (= p(y)) observation : head (N=1,z=1)
! 2nd beta(θ|2,1) mean 2/3 (= p(y)) observation : head (N=1,z=1)
! 3rd beta(θ|3,1) mean 3/4 (= p(y))
![Page 28: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/28.jpg)
prediction (example 2)�
! 1st beta(θ|100,100) : 1/2 (= p(y)) observation : head (N=1,z=1)
observation : head (N=1,z=1)
! 3rd beta(θ|102,100) : 102/202(?50%)
![Page 29: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/29.jpg)
comparison�
to compare the models,
p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence
prior probability�
observation�posterior probability�
![Page 30: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/30.jpg)
comparison�
! Calculation of evidence " p(D|M) = p(z,N)�
![Page 31: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/31.jpg)
comparison�
uniform strongly peaked� uniform strongly peaked�
N = 14, z = 11� N = 14, z = 7�
p(D|M)=0.000183>p(D|M)=6.86×10-5� p(D|M)=1.94×10-5<p(D|M)=5.9×10-5�
![Page 32: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/32.jpg)
comparison�
! both are important " the prior distribution " the likelihood function
" in detail, see chapter 4
The best model (so far) is not a good model.
![Page 33: Doing Bayesian Data Analysis, Chapter 5](https://reader038.vdocuments.mx/reader038/viewer/2022102813/5482f85cb4af9f690d8b4906/html5/thumbnails/33.jpg)
summary�
! binomial proportion " the likelihood function
! the Bernoulli likelihood function " the prior/posterior distribution
! beta distribution
" estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+b) " comparison ---- best model <-p(D|M)