intro to probability - stat.ubc.cabouchard/courses/stat302-sp2017-18//files/... · intro to...

Intro to Probability Instructor: Alexandre Bouchard

www.stat.ubc.ca/~bouchard/courses/stat302-sp2017-18/

Announcements

• Webwork out

• Graded midterm available after lecture

Regrading policy• IF you would like a partial regrading, you should,

BEFORE or ON Friday March 15, hand in to me at the beginning of a lecture:

• your exam

• a clean piece of paper stapled to it that clearly (i) explains the question(s) you would like us to regrade AND (ii) the issue(s) you would like to raise

• NOTE: for fairness, the new grade for the question could stay the same, increase, or, in certain cases, decrease

Plan for today

• Multivariate distributions, continued

• Independence of continuous random variables

Review: joint and marginal densities

Today: density (for two random variables)

Example: A = [a, b] x [c, d] height = density

The function f(x, y) is a ‘joint density’ for X, Y if for any subset A of the plane:

ab

c d

volume = probability

Notation for rectangle with one side equal to [a, b] and the other

equal to [c,d]

P ((X,Y ) 2 A) =

Z

(x,y)2A

f(x, y) dx dy

xy

Def 23

Example: uniform density on a subset B of the plane

density

x yx

yBheight = density

= 1/ area(B)

Example:

Ex 60

f(x, y) =1B(x, y)

area(B)

Recall:1B(x, y) =

⇢1 if (x, y) 2 B

0 o.w.

Motivating problemEx 59

A man and a woman try to meet at a certain place between 1:00pm and 2:00pm. Suppose each person pick an arrival time between 1:00pm and 2:00pm uniformly at random, and waits for the other at most 10 minutes. What is the probability that they meet?

Example of marginal densities

0.0

0.2

0.4

0.6

−1.0 −0.5 0.0 0.5 1.0x

density

●

−1.0

−0.5

0.0

0.5

1.0

−1.0 −0.5 0.0 0.5 1.0X

Y

−1.0

−0.5

0.0

0.5

1.0

0.0 0.2 0.4 0.6density

y

‘Marginal of X’fX(x)

‘Marginal of Y’fY(y)

Height of the marginal at x = 0 obtained by

integrating the joint density over y at x = 0:

fX(x) =

Z +1

�1f(x, y) dy

Def 24

Independence vs. dependence for continuous

random variables

Equivalent definitionsDef 25

X and Y are independent

For all intervals, A1, A2:

f(x, y) = h(x)k(y)

The joint density of (X, Y) can be written as:

Useful to show that r.v.’s are NOT indep

Useful to show that r.v.’s are indep

P (X 2 A1, Y 2 A2) = P (X 2 A1)P (Y 2 A2)

Example: two random variables that are independent

x

y why?

Ex 65

f(x, y) =1B(x, y)

area(B)

a bc

d

=

✓1[a,b](x)

area(B)

◆�1[c,d](y)

�

h(x) k(y)

f(x, y) = h(x)k(y)

The joint density of (X, Y) can be written as:

Example: two random variables that are NOT independent

y

x

Ex 66

why?For some intervals, A1, A2:

P (X 2 A1, Y 2 A2) = P (X 2 A1)P (Y 2 A2)

A1

A2

Pick A1, A2 as shown on the left

Which one(s) of these are zero? (use material from earlier today)

P (X 2 A1, Y 2 A2) = P (X 2 A1)P (Y 2 A2)

P (X 2 A1, Y 2 A2) = P (X 2 A1)P (Y 2 A2)

P (X 2 A1, Y 2 A2) = P (X 2 A1)P (Y 2 A2)

Examples of non-uniform joint density

• P(X > 1, Y < 1) ?

f(x, y) = 2e�x�2y

Suppose (X,Y) has joint density:

for x > 0 and y > 0

Ex 67a

Example

Example

• P(X > 1, Y < 1) ?

• P(X < Y) ?

• X indep of Y?

f(x, y) = 2e�x�2y

Suppose (X,Y) has joint density:

for x > 0 and y > 0

Ex 67b

e-1(1 - e -2)

A. 1/2B. 1/3C. 1/4D. 1/5

A useful trick

Known facts: • Densities integrate to 1

• For any λ > 0, λ exp(-λx) 1[0,∞)(x) is a density (the exponential density)

Note: • We can we use these two facts to get, without any effort:

Z 1

0e

�5.2x dx =1

5.2

Review: transformations

• Suppose I tell you that is the distribution of Richter scales

• What is the distribution of the amplitudes?

• For simplicity:

• Assume Richter scale X ~ Uniform(0, 1)

• What is the distribution of Y = exp(X) ?

Ex 53

Review: recipe for transformations

Recipe for finding the distribution of transforms of r.v.’s

1

2

Find the CDF

Differentiate to find the density

Density fX

Richter:

Amplitude:

0 1

1 102 3 4 5 6 7 8 9



• For simplicity:

• Assume Richter scale X ~ Uniform(0,1)

• What is the distribution of exp(X) ?


1 Find the CDF



• For simplicity:



FY (y) = P (exp(X) y)

= P (X log(y))

= FX(log(y)) = 1[1,e](y) log(y)

Why?

Why P(exp(X)≤y) = P(X≤log(y))

• Because (exp(X)≤y) = (X≤log(y)), which is true because:

• log is increasing, i.e. x1≤x2 iff log(x1)≤log(x2)

• this means I can take log on both sides of the inequality: (exp(X)≤y) = (log(exp(X))≤log(y))

• log/exp are invertible: log(exp(z)) = z, so(log(exp(X))≤log(y)) = (X≤log(y))


2



• For simplicity:




fY (y) =dFY (y)

dy

= 1[1,e](y)1

y

at points where FY is differentiable

Sums of independent discrete random variables

(exact method)

Sum of independent r.v.s: summary

• Approximations:

• Central limit theorem (Normal approximation)

• Use software/PPL

• Exact methods:

• Binomial distribution (works only for sum of Bernoullis)

• Today: general, exact method CONVOLUTIONS

Simple example

• X: outcome of white dice

• Y: outcome of black dice

• Example: computing P(X + Y = 4)

Ex 68

Simple example

Application

• Not convinced? Play this game:

Settler of Catan

General formula for discrete r.v.s

If:

Sum of Independent Random Variables

Consider two integer-valued independent r.v. X and Y of respectivep.m.f. pX (x) and pY (y).

Consider Z = X + Y , we want to compute the p.m.f. of Z denotedpZ (z).

Assume Y = y then Z = z if and only if X = z y and

P (X = z y Y = y) = pX (z y) pY (y)

so, as Y can take integer values and the events

(X = z y) (Y = y) and (X = z y ) (Y = y ) are mutuallyexclusive for y = y , we have

pZ (z) =�

⇥y=�

pX (z y) pY (y) .

AD () March 2010 9 / 13

Then:


Consider two integer-valued independent r.v. X and Y of respectivep.m.f. pX (x) and pY (y).

Consider Z = X + Y , we want to compute the p.m.f. of Z denotedpZ (z).

Assume Y = y then Z = z if and only if X = z y and

P (X = z y Y = y) = pX (z y) pY (y)

so, as Y can take integer values and the events

(X = z y) (Y = y) and (X = z y ) (Y = y ) are mutuallyexclusive for y = y , we have

pZ (z) =�

⇥y=�

pX (z y) pY (y) .

AD () March 2010 9 / 13

Prop 16

Sums of independent continuous random variables

Sum of continuous r.v.s• X: a continuous r.v. with density fX

• Y: a continuous r.v. with density fY

• Assume they are indep: f(x, y) = fX(x) fY(y)

• What is the density fZ of the sum Z = X + Y?

Recipe for finding the distribution of transforms of r.v.’s

1

2

Find the CDF


Density fX

Richter:

Amplitude:

0 1

1 102 3 4 5 6 7 8 9

Example

• Let X and Y be independent and both uniform on [0, 1]


x

y

Ex 69

Example



x

y

1 Find the CDF

P( Z ≤ 1 ) = P( X + Y ≤ 1 )

= ?

FZ(z) = P(Z ≤ z) example: z = 1

Example



x

y

1 Find the CDF

P( Z ≤ 1 ) = P( X + Y ≤ 1 )

= P( (X, Y) ∈ A )

x

y

P(Z ≤ z) for all zexample: z = 1

=

Z

Af(x, y) dx dy

= 1/2

=

Z 1

�1

✓Z 1�x

�1f(x, y) dy

◆dx

A = {(x,y) : x + y ≤ 1}

Example



x

y

1 Find the CDF

P( Z ≤ z ) = P( X + Y ≤ z )

P(Z ≤ z) for all z

=

Z 1

�1

✓Zz�x

�1f

X

(x)fY

(y) dy

◆dx

=

Z 1

�1f

X

(x)

✓Zz�x

�1f

Y

(y) dy

◆dx

=

Z 1

�1fX(x) (FY (z � x)) dx

Definition of the CDF F(y)

Example



x

y

1 Find the CDF

FZ(z) = P( Z ≤ z )

2 Differentiate to find the density

=

Z 1

�1fX(x)FY (z � x)dx

fZ(z) =dFZ(z)

dz=

Z 1

�1fX(x)

dFY (z � x)

dzdx

=

Z 1

�1fX(x)fY (z � x)dx

=

Z 1

�1fX(x)fY (z � x)

✓d

dz(z � x)

◆dx

Under regularity conditions, you can interchange

integrals and derivatives

Chain rule of calculus

Sum of continuous r.v.s• X: a continuous r.v. with density fX

• Y: a continuous r.v. with density fY



In numerous scenarios, we have to sum independent continuous r.v.;signal + noise, sums of dierent random eects etc.

Assume that X ,Y are continuous r.v. of respective pdf fX (x) andfY (y) then Z = X + Y admits the pdf

fZ (z) = �

�fX (z y) fY (y) dy

= �

�fX (x) fY (z x) dx

The pdf fZ (z) is the so-called “convolution” of fX (x) and fY (y).

AD () March 2010 11 / 13

Terminology: ‘convolution’

Prop 16b



Note: Not equal to the sum of the densities !!!

Ex 69

x

y

intro to probability - stat.ubc.cabouchard/courses/stat302-sp2017-18//files/... · intro to...

Documents