probability theory presentation 03

8/8/2019 Probability Theory Presentation 03

http://slidepdf.com/reader/full/probability-theory-presentation-03 1/36

BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational BiologyUniversity of Rochester

September 9, 2010

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Outline

1 σ-algebras

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra),

which is a fundamentally useful tool in modern probability

and statistics theory.

We will define notions such as probability, probability

space and random variables formally later. Some informal

knowledge about probability is useful for this lecture,

though.

At the intuitive level, a probability is an assessment of how

“likely” a particular set of outcomes might be. In otherwords, a probability is a set function: it takes a set as input,

and then churns out a real number as its output.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Motivation I







though.




Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Motivation II

Again by our intuition, a probability P should have thefollowing properties:

1 If A, B are two well defined sets of outcomes, we should be

able to “talk about” the probability of A ∩ B , A ∪ B , and Ac orB c .

2 Other important properties such as non-negativity andadditivity which will be introduced later.

In other words, the domain of P is not just any collection of

sets, it must have certain algebraic properties.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Motivation II







Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Motivation III

If a set function satisfies certain properties, it will be called

a measure. Later we will learn that all valid probabilities

are measures.

An important family of non-probability measure is the

family of Lebesgue measures, which is the usual length for

R1, area for R2, volume for R3, etc.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Set with infinite number of subsets

Def: An forms an increasing sequence of sets with limit A:

A1 ⊂ A2 ⊂ . . . and ∪∞n =1An = A. Denote as An ↑ A.

Similarly, we can define An ↓ A.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of realnumbers.

The analogy in set theory:

lim supn

An =

∞

n

∞

k =n

Ak .

ω ∈ lim inf n An iff ω ∈ An for infinite many times.

Similarly

lim inf n An =

∞n

∞k =n

Ak .

ω ∈ lim inf n An iff ω /∈ An for only finite many times.

If lim supn An = lim inf n An = A, we say A = limn An .

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/






lim supn

An =

∞

n

∞

k =n

Ak

.


Similarly

lim inf n An =

∞

n

∞

k =n

Ak .



Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Algebras

Def: algebra of collection of subsets: closure under Ac and

A ∪ B , which implies closure under ∩.Finite set algebra: always atomizable. So it is easy to

make it close under set operations.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A

to N.

For N,·

· is not well defined. (Partial) solution: extend N toQ.

Q is not closed under the limit operation. Solution: extend

Q to R.

Strictly speaking, R is not closed under division sincea

0(singular points) is undefined. It creates a lot of trouble!

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why algebras? (I)



to N.

For N,·



Q to R.

Strictly speaking, R is not closed under division since

a


Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why algebras? (I)



to N.

For N,·



Q to R.


a


Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of setoperations.

Real/complex number example: f (x ) = (x ).

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite

unions/intersections.

The minimum σ-algebras.The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is defined

to be all subsets of finitely many numbers. Is the set of

even numbers a member of F ?

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Σ-algebras







Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why σ-algebras?

Closure under countable infinite union makes it easy to use∞

n =1, or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure

taking limit is a valid operation.

Without this we still can talk about the finite step arithmetic

(for Q) or set (for sets) operations, yet we can not utilize

most of the modern mathematical tools (that is, pretty

much every theorem since calculus).

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



Why σ-algebras?









Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/



σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind ofinformation: Some sets are valid, some sets are

“unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum

one), and a finer one (row algebra, or the max algebra).The column σ-algebra and the row σ-algebra represents

different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed

with functional transformations, a subject we will brieflytouch when we discuss the characteristic functions.)

Infinite case, stock price prediction as a function of days.

Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/





“unspeakable”.








Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/





“unspeakable”.








Qiu, Lee BST 401

l b d “i f i ”

http://goforward/

http://find/

http://goback/





“unspeakable”.








Qiu, Lee BST 401

http://goforward/

http://find/

http://goback/

probability theory presentation 03

Documents