![Page 1: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/1.jpg)
Winter 2006 EE384x 1
Review of Probability Theory
Review Session 1EE384X
![Page 2: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/2.jpg)
Winter 2006 EE384x 2
A random experiment with set of outcomes
Random variable is a function from set of outcomes to real numbers
Random Variable
![Page 3: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/3.jpg)
Winter 2006 EE384x 3
Example
Indicator random variable:
A : A subset of is called an event
![Page 4: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/4.jpg)
Winter 2006 EE384x 4
CDF and PDF Discrete random variable:
The possible values are discrete (countable)
Continuous random variable: The rv can take a range of values in R
Cumulative Distribution Function (CDF):
PDF and PMF:
![Page 5: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/5.jpg)
Winter 2006 EE384x 5
Expectation and higher moments
Expectation (mean):
if X>0 :
Variance:
![Page 6: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/6.jpg)
Winter 2006 EE384x 6
Two or more random variables
Joint CDF:
Covariance:
![Page 7: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/7.jpg)
Winter 2006 EE384x 7
Independence
For two events A and B:
Two random variables
IID : Independent and Identically Distributed
![Page 8: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/8.jpg)
Winter 2006 EE384x 8
Useful Distributions
![Page 9: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/9.jpg)
Winter 2006 EE384x 9
Bernoulli Distribution
The same as indicator rv:
IID Bernoulli rvs (e.g. sequence of coin flips)
![Page 10: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/10.jpg)
Winter 2006 EE384x 10
Binomial Distribution Repeated Trials:
Repeat the same random experiment n times. (Experiments are independent of each other)
Number of times an event A happens among n trials has Binomial distribution (e.g., number of heads in n coin tosses, number of arrivals in n time
slots,…)
Binomial is sum of n IID Bernouli rvs
![Page 11: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/11.jpg)
Winter 2006 EE384x 11
Mean of Binomial
Note that:
![Page 12: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/12.jpg)
Winter 2006 EE384x 12
1 2 3 4 5 6 7 8 9 10 11 120
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Binomial - Example
n=4
n=40
n=10
n=20
p=0.2
![Page 13: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/13.jpg)
Winter 2006 EE384x 13
Binomial – Example (ball-bin)
There are B bins, n balls are randomly dropped into bins.
: Probability that a ball goes to bin i : Number of balls in bin i after n drops
![Page 14: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/14.jpg)
Winter 2006 EE384x 14
Multinomial Distribution Generalization of Binomial Repeated Trails (we are interested in
more than just one event A) A partition of into A1,A2,…,Al
Xi shows the number of times Ai occurs
among n trails.
![Page 15: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/15.jpg)
Winter 2006 EE384x 15
Poisson Distribution
Used to model number of arrivals
![Page 16: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/16.jpg)
Winter 2006 EE384x 16
0 5 10 150
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Poisson Graphs
=10
=.5
=1
=4
![Page 17: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/17.jpg)
Winter 2006 EE384x 17
Poisson as limit of Binomial
Poisson is the limit of Binomial(n,p) as
Let
![Page 18: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/18.jpg)
Winter 2006 EE384x 18
0 1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Poisson and Binomial
Poisson(4)
n=5,p=4/5
n=20, p=.2
n=10,p=.4
n=50,p=.08
![Page 19: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/19.jpg)
Winter 2006 EE384x 19
Geometric Distribution
Repeated Trials: Number of trials till some event occurs
![Page 20: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/20.jpg)
Winter 2006 EE384x 20
Exponential Distribution
Continuous random variable Models lifetime, inter-arrivals,…
![Page 21: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/21.jpg)
Winter 2006 EE384x 21
Minimum of Independent Exponential rvs
: Independent Exponentials
![Page 22: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/22.jpg)
Winter 2006 EE384x 22
Memoryless property
True for Geometric and Exponential Dist.:
The coin does not remember that it came up tails l times
Root cause of Markov Property.
![Page 23: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/23.jpg)
Winter 2006 EE384x 23
Proof for Geometric
![Page 24: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/24.jpg)
Winter 2006 EE384x 24
Moment Generating Function (MGF) For continuous rvs (similar to Laplace
xform)
For Discrete rvs (similar to Z-transform):
Characteristic Function
![Page 25: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/25.jpg)
Winter 2006 EE384x 25
Characteristic Function
Can be used to compute mean or higher moments:
If X and Y are independent and T=X+Y
![Page 26: Winter 2006EE384x1 Review of Probability Theory Review Session 1 EE384X](https://reader035.vdocuments.mx/reader035/viewer/2022062300/56649e705503460f94b6e22f/html5/thumbnails/26.jpg)
Winter 2006 EE384x 26
Useful CFs
Bernoulli(p) :
Binomial(n,p) :
Multinomial:
Poisson: