8/8/2019 Probability Theory Presentation 01

1/29

BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational BiologyUniversity of Rochester

September 2, 2010

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

2/29

Outline

1 Set and Functions

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

3/29

Set Operations

The whole set , the empty set ; an element x in a set A:x A.

A B, A B; A B, A B.

Set operations: A B, A B, Ac

(w.r.t. ).De Morgans laws: (A B)c = AcBc, (A B)c = AcBc.

For more than two sets:

n

Anc

=

nAcn,

n

Anc

=

nAcn.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

4/29

Set Operations

The whole set , the empty set ; an element x in a set A:x A.

A B, A B; A B, A B.

Set operations: A B, A B, Ac

(w.r.t. ).De Morgans laws: (A B)c = AcBc, (A B)c = AcBc.

For more than two sets:

n

Anc

=

nAcn,

n

Anc

=

nAcn.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

5/29

Set Operations

The whole set , the empty set ; an element x in a set A:x A.

A B, A B; A B, A B.

Set operations: A B, A B, Ac

(w.r.t. ).De Morgans laws: (A B)c = AcBc, (A B)c = AcBc.

For more than two sets:

n

Anc

=

nAcn,

n

Anc

=

nAcn.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

6/29

Set Operations

The whole set , the empty set ; an element x in a set A:x A.

A B, A B; A B, A B.

Set operations: A B, A B, Ac

(w.r.t. ).De Morgans laws: (A B)c = AcBc, (A B)c = AcBc.

For more than two sets:

n

Anc

=

nAcn,

n

Anc

=

nAcn.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

7/29

Set Operations

The whole set , the empty set ; an element x in a set A:x A.

A B, A B; A B, A B.

Set operations: A B, A B, Ac

(w.r.t. ).De Morgans laws: (A B)c = AcBc, (A B)c = AcBc.

For more than two sets:

n

Anc

=

nAcn,

n

Anc

=

nAcn.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

8/29

Disjoint atoms: the classical three-circle-diagram.

Figure: Three sets can generate 7 atoms.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

9/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

10/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

11/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

12/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

13/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

14/29

Functions

Please review the basic definitions of a function. Pay

attention to a functions domain and its image.

I assume you know the definition and basic properties ofthe following elementary functions:

1 Power functions, xa, a R. When a is a non-integerrational number, without confusion we take the principlebranch of nth root operation; when a is irrational, itsdefinition is given by the principle branch of a log function.

2 Exponential and logarithmic functions, ex and log(x).3

Trigonometric functions and their inverse functions. Theirdomain, image, etc.4 The combination of the above by +,,, and functional

composition.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

15/29

Limit of a sequence of real numbers (I)

For simplicity, we are going to use increasing to meannon-decreasing. decreasing to mean non-increasing.

A sequence of real numbers (ai) = (a1, a2, . . .) convergesto a if for any given precision criterion > 0, there exists

an integer N such that the error, defined as

dist(ai a) = |ai a

|, is smaller than for all i N.

An increasing, bounded sequence of real numbers

a1 a2 . . . always converges to a limit. (if you consider as a valid limiting point, the boundedness part can be

omitted.)Similarly, a decreasing sequence of real numbers always

converges to a limit (if you dont like , you can add thebounded from below condition).

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

16/29

Limit of a sequence of real numbers (I)

For simplicity, we are going to use increasing to meannon-decreasing. decreasing to mean non-increasing.

A sequence of real numbers (ai) = (a1, a2, . . .) convergesto a if for any given precision criterion > 0, there exists

an integer N such that the error, defined as

dist(ai a) = |ai a

|, is smaller than for all i N.

An increasing, bounded sequence of real numbers

a1 a2 . . . always converges to a limit. (if you consider as a valid limiting point, the boundedness part can be

omitted.)Similarly, a decreasing sequence of real numbers always

converges to a limit (if you dont like , you can add thebounded from below condition).

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

17/29

Limit of a sequence of real numbers (I)

For simplicity, we are going to use increasing to meannon-decreasing. decreasing to mean non-increasing.

A sequence of real numbers (ai) = (a1, a2, . . .) convergesto a if for any given precision criterion > 0, there exists

an integer N such that the error, defined as

dist(ai a) = |ai a

|, is smaller than for all i N.

An increasing, bounded sequence of real numbers

a1 a2 . . . always converges to a limit. (if you consider as a valid limiting point, the boundedness part can be

omitted.)Similarly, a decreasing sequence of real numbers always

converges to a limit (if you dont like , you can add thebounded from below condition).

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

18/29

Limit of a sequence of real numbers (I)

For simplicity, we are going to use increasing to meannon-decreasing. decreasing to mean non-increasing.

A sequence of real numbers (ai) = (a1, a2, . . .) convergesto a if for any given precision criterion > 0, there exists

an integer N such that the error, defined as

dist(ai a) = |ai a

|, is smaller than for all i N.

An increasing, bounded sequence of real numbers

a1 a2 . . . always converges to a limit. (if you consider as a valid limiting point, the boundedness part can be

omitted.)Similarly, a decreasing sequence of real numbers always

converges to a limit (if you dont like , you can add thebounded from below condition).

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

19/29

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1, a2, . . .) alwayshas a subsequence which approaches the upper limit

(including as a possible limit) of this sequence.

Similarly, it contains a subsequence which approaches its

lower limit. Once the upper limit equals the lower limit, we

say this sequence converges to this limit.

Two companion subsequences, denoted as (b1, b2, . . .)and (c1, c2, . . .), can be quite useful:

bi = supin ai, ci = infinai.

(bi) is decreasing and (ci) is increasing and they convergeto lim supi ai and lim infi ai, respectively.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

20/29

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1, a2, . . .) alwayshas a subsequence which approaches the upper limit

(including as a possible limit) of this sequence.

Similarly, it contains a subsequence which approaches its

lower limit. Once the upper limit equals the lower limit, we

say this sequence converges to this limit.

Two companion subsequences, denoted as (b1, b2, . . .)and (c1, c2, . . .), can be quite useful:

bi = supin ai, ci = infinai.

(bi) is decreasing and (ci) is increasing and they convergeto lim supi ai and lim infi ai, respectively.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

21/29

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1, a2, . . .) alwayshas a subsequence which approaches the upper limit

(including as a possible limit) of this sequence.

Similarly, it contains a subsequence which approaches its

lower limit. Once the upper limit equals the lower limit, we

say this sequence converges to this limit.

Two companion subsequences, denoted as (b1, b2, . . .)and (c1, c2, . . .), can be quite useful:

bi = supin ai, ci = infinai.

(bi) is decreasing and (ci) is increasing and they convergeto lim supi ai and lim infi ai, respectively.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

22/29

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1, a2, . . .) alwayshas a subsequence which approaches the upper limit

(including as a possible limit) of this sequence.

Similarly, it contains a subsequence which approaches its

lower limit. Once the upper limit equals the lower limit, we

say this sequence converges to this limit.

Two companion subsequences, denoted as (b1, b2, . . .)and (c1, c2, . . .), can be quite useful:

bi = supin ai, ci = infinai.

(bi) is decreasing and (ci) is increasing and they convergeto lim supi ai and lim infi ai, respectively.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

23/29

Limit of a sequence of real numbers (IV)

For a sequence (an), If its upper limit equals its lower limit

(lim supnan = lim infnan = a), then (an) converges to a.The distance function which quantifies error is important.

For real numbers, there is essentially one way to measure

the error term: |ai a|. This is because a distance

function needs to satisfy several axioms (use wikipedia).

For a sequence of n-dimensional points(vectors), the

natural way to measure the error term is the Euclidean

distance. But other distance functions do exist, such as the

Manhattan distance (google it). Fortunately, a sequence of

vectors is convergent in one distance implies it isconvergent in all other distances.

Unfortunately, you can define quite a few non-compatible

distances of random numbers. So there are many different

convergences of random numbers.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

24/29

Limit of a sequence of real numbers (IV)

For a sequence (an), If its upper limit equals its lower limit

(lim supnan = lim infnan = a), then (an) converges to a.The distance function which quantifies error is important.

For real numbers, there is essentially one way to measure

the error term: |ai a|. This is because a distance

function needs to satisfy several axioms (use wikipedia).

For a sequence of n-dimensional points(vectors), the

natural way to measure the error term is the Euclidean

distance. But other distance functions do exist, such as the

Manhattan distance (google it). Fortunately, a sequence of

vectors is convergent in one distance implies it isconvergent in all other distances.

Unfortunately, you can define quite a few non-compatible

distances of random numbers. So there are many different

convergences of random numbers.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

25/29

Limit of a sequence of real numbers (IV)

For a sequence (an), If its upper limit equals its lower limit

(lim supnan = lim infnan = a), then (an) converges to a.The distance function which quantifies error is important.

For real numbers, there is essentially one way to measure

the error term: |ai a|. This is because a distance

function needs to satisfy several axioms (use wikipedia).

For a sequence of n-dimensional points(vectors), the

natural way to measure the error term is the Euclidean

distance. But other distance functions do exist, such as the

Manhattan distance (google it). Fortunately, a sequence of

vectors is convergent in one distance implies it isconvergent in all other distances.

Unfortunately, you can define quite a few non-compatible

distances of random numbers. So there are many different

convergences of random numbers.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

26/29

Limit of a sequence of real numbers (IV)

For a sequence (an), If its upper limit equals its lower limit

(lim supnan = lim infnan = a), then (an) converges to a.The distance function which quantifies error is important.

For real numbers, there is essentially one way to measure

the error term: |ai a|. This is because a distance

function needs to satisfy several axioms (use wikipedia).

For a sequence of n-dimensional points(vectors), the

natural way to measure the error term is the Euclidean

distance. But other distance functions do exist, such as the

Manhattan distance (google it). Fortunately, a sequence of

vectors is convergent in one distance implies it isconvergent in all other distances.

Unfortunately, you can define quite a few non-compatible

distances of random numbers. So there are many different

convergences of random numbers.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

27/29

Limit of a function

We can now define the limit of a function f(x) as xapproaches x0. x0 could be , for the sake of simplicitywe assume x0 is finite for the following definition.

limxx0 f(x) = y if and only if

> 0, > 0 such that dist(f(x)y

) < for all x Ball(x0, ).

The logic negation of a sequence/function converges to a

value is that this sequence/function breaks the precision

rule infinitely often. More precisely, a sequence is not

convergent if for a given > 0, we have

dist(ai, a) > i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

28/29

Limit of a function

We can now define the limit of a function f(x) as xapproaches x0. x0 could be , for the sake of simplicitywe assume x0 is finite for the following definition.

limxx0 f(x) = y if and only if

> 0, > 0 such that dist(f(x)y

) < for all x Ball(x0, ).

The logic negation of a sequence/function converges to a

value is that this sequence/function breaks the precision

rule infinitely often. More precisely, a sequence is not

convergent if for a given > 0, we have

dist(ai, a) > i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401

http://find/http://goback/

8/8/2019 Probability Theory Presentation 01

29/29

Limit of a function

We can now define the limit of a function f(x) as xapproaches x0. x0 could be , for the sake of simplicitywe assume x0 is finite for the following definition.

limxx0 f(x) = y if and only if

> 0, > 0 such that dist(f(x)y

) < for all x Ball(x0, ).

The logic negation of a sequence/function converges to a

value is that this sequence/function breaks the precision

rule infinitely often. More precisely, a sequence is not

convergent if for a given > 0, we have

dist(ai, a) > i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401

http://find/http://goback/