Download - Basics on Probability
![Page 1: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/1.jpg)
Basics on ProbabilityJingrui He
09/11/2007
![Page 2: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/2.jpg)
Coin Flips You flip a coin
Head with probability 0.5
You flip 100 coins How many heads would you expect
![Page 3: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/3.jpg)
Coin Flips cont. You flip a coin
Head with probability p Binary random variable Bernoulli trial with success probability p
You flip k coins How many heads would you expect Number of heads X: discrete random variable Binomial distribution with parameters k and p
![Page 4: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/4.jpg)
Discrete Random Variables Random variables (RVs) which may take on
only a countable number of distinct values E.g. the total number of heads X you get if you
flip 100 coins
X is a RV with arity k if it can take on exactly one value out of E.g. the possible values that X can take on are 0,
1, 2,…, 100
1, , kx x
![Page 5: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/5.jpg)
Probability of Discrete RV Probability mass function (pmf): Easy facts about pmf
if if
P X X 0i jx x
i j
1 2P X X X 1kx x x P X X P X P Xi j i jx x x x
i j
P X 1iix
P X ix
![Page 6: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/6.jpg)
Common Distributions Uniform
X takes values 1, 2, …, N E.g. picking balls of different colors from a box
Binomial X takes values 0, 1, …, n
E.g. coin flips
X 1, ,U N
P X 1i N
X ,Bin n p
P X 1n iin
i p pi
![Page 7: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/7.jpg)
Coin Flips of Two Persons Your friend and you both flip coins
Head with probability 0.5 You flip 50 times; your friend flip 100 times How many heads will both of you get
![Page 8: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/8.jpg)
Joint Distribution Given two discrete RVs X and Y, their joint
distribution is the distribution of X and Y together E.g. P(You get 21 heads AND you friend get 70
heads)
E.g.
P X Y 1x y
x y
50 100
0 0P You get heads AND your friend get heads 1
i ji j
![Page 9: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/9.jpg)
Conditional Probability is the probability of ,
given the occurrence of E.g. you get 0 heads, given that your friend gets
61 heads
P X Yx y
P X YP X Y
P Y
x yx y
y
X xY y
![Page 10: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/10.jpg)
Law of Total Probability Given two discrete RVs X and Y, which take
values in and , We have 1, , mx x 1, , ny y
P X P X Y
P X Y P Y
i i jj
i j jj
x x y
x y y
![Page 11: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/11.jpg)
Marginalization
Marginal Probability Joint Probability
Conditional Probability
P X P X Y
P X Y P Y
i i jj
i j jj
x x y
x y y
Marginal Probability
![Page 12: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/12.jpg)
Bayes Rule X and Y are discrete RVs…
P Y X P XP X Y
P Y X P X
j i ii j
j k kk
y x xx y
y x x
P X YP X Y
P Y
x yx y
y
![Page 13: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/13.jpg)
Independent RVs Intuition: X and Y are independent means that
neither makes it more or less probable that
Definition: X and Y are independent iff P X Y P X P Yx y x y
X xY y
![Page 14: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/14.jpg)
More on Independence
E.g. no matter how many heads you get, your friend will not be affected, and vice versa
P X Y P X P Yx y x y
P X Y P Xx y x P Y X P Yy x y
![Page 15: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/15.jpg)
Conditionally Independent RVs Intuition: X and Y are conditionally
independent given Z means that once Z is known, the value of X does not add any additional information about Y
Definition: X and Y are conditionally independent given Z iff
P X Y Z P X Z P Y Zx y z x z y z
![Page 16: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/16.jpg)
More on Conditional Independence
P X Y Z P X Z P Y Zx y z x z y z
P X Y , Z P X Zx y z x z
P Y X , Z P Y Zy x z y z
![Page 17: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/17.jpg)
Monty Hall Problem You're given the choice of three doors: Behind one
door is a car; behind the others, goats. You pick a door, say No. 1 The host, who knows what's behind the doors, opens
another door, say No. 3, which has a goat. Do you want to pick door No. 2 instead?
![Page 18: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/18.jpg)
Host mustreveal Goat B
Host mustreveal Goat A
Host revealsGoat A
orHost reveals
Goat B
![Page 19: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/19.jpg)
Monty Hall Problem: Bayes Rule : the car is behind door i, i = 1, 2, 3 : the host opens door j after you pick door i
iC
ijH
1 3iP C
0
0
1 2
1 ,
ij k
i j
j kP H C
i k
i k j k
![Page 20: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/20.jpg)
Monty Hall Problem: Bayes Rule cont. WLOG, i=1, j=3
13 1 11 13
13
P H C P CP C H
P H
13 1 11 1 1
2 3 6P H C P C
![Page 21: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/21.jpg)
Monty Hall Problem: Bayes Rule cont.
13 13 1 13 2 13 3
13 1 1 13 2 2
, , ,
1 11
6 31
2
P H P H C P H C P H C
P H C P C P H C P C
1 131 6 1
1 2 3P C H
![Page 22: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/22.jpg)
Monty Hall Problem: Bayes Rule cont.
1 131 6 1
1 2 3P C H
You should switch!
2 13 1 131 2
13 3
P C H P C H
![Page 23: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/23.jpg)
Continuous Random Variables What if X is continuous? Probability density function (pdf) instead of
probability mass function (pmf) A pdf is any function that describes the
probability density in terms of the input variable x.
f x
![Page 24: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/24.jpg)
PDF Properties of pdf
Actual probability can be obtained by taking the integral of pdf E.g. the probability of X being between 0 and 1 is
0,f x x
1f x
1
0P 0 1X f x dx
1 ???f x
![Page 25: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/25.jpg)
Cumulative Distribution Function Discrete RVs
Continuous RVs
X P XF v v
X P Xi
ivF v v
X
vF v f x dx
X
dF x f x
dx
![Page 26: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/26.jpg)
Common Distributions Normal
E.g. the height of the entire population
2X ,N
22
1exp ,
2 2
xf x x
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
f(x)
![Page 27: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/27.jpg)
Common Distributions cont. Beta
: uniform distribution between 0 and 1 E.g. the conjugate prior for the parameter p in
Binomial distribution
X ,Beta
111; , 1 , 0,1
,f x x x x
B
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
x
f(x)
![Page 28: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/28.jpg)
Joint Distribution Given two continuous RVs X and Y, the joint
pdf can be written as
X,Y ,f x y
X,Y , 1x y
f x y dxdy
![Page 29: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/29.jpg)
Multivariate Normal Generalization to higher dimensions of the
one-dimensional normal
1X 1 22
1
1, ,
2
1exp
2
d d
T
f x x
x x
Covariance Matrix
Mean
![Page 30: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/30.jpg)
Moments Mean (Expectation):
Discrete RVs:
Continuous RVs: Variance:
Discrete RVs:
Continuous RVs:
XE X P X
ii iv
E v v XE xf x dx
2X XV E 2X P X
ii iv
V v v 2XV x f x dx
![Page 31: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/31.jpg)
Properties of Moments Mean
If X and Y are independent,
Variance If X and Y are independent,
X Y X YE E E
X XE a aE XY X YE E E
2X XV a b a V
X Y (X) (Y)V V V
![Page 32: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/32.jpg)
Moments of Common Distributions Uniform
Mean ; variance Binomial
Mean ; variance Normal
Mean ; variance Beta
Mean ; variance
X 1, ,U N 1 2N 2 1 12N
X ,Bin n pnp 2np
2X ,N 2
X ,Beta 2
1
![Page 33: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/33.jpg)
Probability of Events X denotes an event that could possibly happen
E.g. X=“you will fail in this course” P(X) denotes the likelihood that X happens,
or X=true What’s the probability that you will fail in this
course? denotes the entire event set
X,X
![Page 34: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/34.jpg)
The Axioms of Probabilities 0 <= P(X) <= 1 , where are
disjoint events Useful rules
P 1
1 2P X X P Xii Xi
1 2 1 2 1 2P X X P X P X P X X
P X 1 P X
![Page 35: Basics on Probability](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56813a55550346895da24c8c/html5/thumbnails/35.jpg)
Interpreting the Axioms
1X2X