probability theory presentation 14

8/8/2019 Probability Theory Presentation 14

1/26

BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational BiologyUniversity of Rochester

October 15, 2009

Qiu, Lee BST 401
http://find/http://goback/


2/26

Outline

Qiu, Lee BST 401


3/26

Expectation (I)

If Y = (X), then EY =(x)dP.

Special case: (X) = Xk. EXk is called the kth moment ofX.

E|X|k: the kth absolute momentof X (remember the Lp

norm?).

E(X EX)k: the kth central momentof X.

E|X EX|k: the kth absolute central momentof X.

The second central moment (which is also the second

absolute central moment) of X is called the varianceof X.

Denote m = EX.var(X) = E(X m)2 = E(X2 2mX + m2) = E(X2) m2.

Qiu, Lee BST 401


4/26

Expectation (I)




norm?).






Qiu, Lee BST 401


5/26

Expectation (I)




norm?).






Qiu, Lee BST 401


6/26

Expectation (I)




norm?).






Qiu, Lee BST 401


7/26

Expectation (I)




norm?).






Qiu, Lee BST 401


8/26

Expectation (I)




norm?).






Qiu, Lee BST 401


9/26

Expectation (I)




norm?).






Qiu, Lee BST 401


10/26

Expectation (II)

The existing of a higher order moment implies the existing

of a lower order moment.

A special case of Chebyshevs inequality: (m = EX,=STD)

P(|X m| > k) 1

k2

Qiu, Lee BST 401


11/26

Expectation (II)

The existing of a higher order moment implies the existing

of a lower order moment.

A special case of Chebyshevs inequality: (m = EX,=STD)

P(|X m| > k) 1

k2

Qiu, Lee BST 401


12/26

Covariance/correlation

cov(X, Y) = E(X EX)(Y EY).corr(X, Y) = cov(X, Y)/

X

Y.

Why correlation is always in [1, 1]? Cauchy-Schwarz.

Variance of summation.

var(X1 + X2 + . . .) =

varXi + 2

i,j=1;i


13/26


cov(X, Y) = E(X EX)(Y EY).corr(X, Y) = cov(X, Y)/

X

Y.



var(X1 + X2 + . . .) =

varXi + 2

i,j=1;i


14/26


cov(X, Y) = E(X EX)(Y EY).corr(X, Y) = cov(X, Y)/XY.



var(X1 + X2 + . . .) =

varXi + 2

i,j=1;i


15/26

Independence (I))

Two events A, B are independent if P(A B) = P(A)P(B).

Two random variables X, Y are independence if for any

A,B B, P(X A,Y B) = P(X A)P(Y B).

Two -algebras F and G are independent if all events ineach algebra are independent.

The second definition is a special case of 3 because X

induces a -algebra FX = X1(B) on . Note that

FX F (coarser).

For more than one variables: X1,X2, . . .. We have to makesure for all B1,B2, . . . B, X

1(B1),X1(B2), . . . are

independent.

Qiu, Lee BST 401


16/26

Independence (I))







FX F (coarser).


1(B1),X1(B2), . . . are

independent.

Qiu, Lee BST 401


17/26

Independence (I))







FX F (coarser).


1(B1),X1(B2), . . . are

independent.

Qiu, Lee BST 401


18/26

Independence (I))







FX F (coarser).


1(B1),X1(B2), . . . are

independent.

Qiu, Lee BST 401


19/26

Independence (I))







FX F (coarser).


1(B1),X1(B2), . . . are

independent.

Qiu, Lee BST 401


20/26

(Independence (II))

Distribution function and independence: X1,X2, . . . areindependent iff F(x1, x2, . . . ) = F(x1)F(x2) . . ..

The part is just a special case of the definition. The

part relies on the Carathodory extension theorem.In terms of density function: the same multiplication rule.

If X1,X2, . . . are independent, then f1(X1), f2(X2), . . . areindependent. This is because fi(X) can only induce a

-algebra that is coarser than FX. Corollary (4.5) is aslightly stronger version of this proposition.

Qiu, Lee BST 401


21/26

(Independence (II))





-algebra that is coarser thanF

X. Corollary (4.5) is aslightly stronger version of this proposition.

Qiu, Lee BST 401


22/26

(Independence (II))







Qiu, Lee BST 401


23/26

(Independence (II))







Qiu, Lee BST 401


24/26

Independence and Expectation

Being independent implies being uncorrelated

EXY = EX EY but not vice versa.

This is because covariance is a summary statistic which is

sort of the average dependence and being dependent

means independence almost everywhere. Example: X,Y are two discrete random variables. X Y meansP(X = t,Y = s) = P(X = t)P(Y = s) for every t, s, sothere are many, many constraint equations. On the other

hand, EXY = EXEY is just one constraint equation

t X(t)Y(t) =

t

s XtYs.

The convolution formula for the sum of two independent

r.v.s.

Qiu, Lee BST 401


25/26








t X(t)Y(t) =

t

s XtYs.


r.v.s.

Qiu, Lee BST 401


26/26








t X(t)Y(t) =

t

s XtYs.


r.v.s.

Qiu, Lee BST 401

probability theory presentation 14

Documents