# ece353: probability and random processes lecture fuxia/lec2-settheoryjan10.pdf · ece353:...

Post on 30-Aug-2018

214 views

Embed Size (px)

TRANSCRIPT

ECE353: Probability and Random Processes

Lecture 2 - Set Theory

Xiao Fu

School of Electrical Engineering and Computer ScienceOregon State University

E-mail: xiao.fu@oregonstate.edu

January 10, 2018

Set Theory

Set theory is the mathematical basis of proabability.

A set contains elements, e.g.,

A = {1, 2, 3}, B = {h, t}.

In an experiment, S is a set of elementary outcomes.

A subset of S, i.e., A S, is an event, where A S reads A is a subset of orequal to S.

We use lower-case letters to denote the elements in sets, e.g., x A means xbelongs to A.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 1

Union

Union of A and B is

A B = {x S | x A or x B},

where A and B are events and S is the entire sample space.

x A B if and only if x A or x B.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 2

Intersection

Intersection of A and B is

A B = {x S | x A and x B},

where A and B are events and S is the entire sample space.

x A B if and only if x A and x B.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 3

Complement

Complement of A is denoted as

Ac = {x S | x / A}.

note that (Ac)c = A.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 4

Empty Set

Empty set, denoted as , is defined through its properties

1. A = A.2. A = .3. c = S.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 5

Set Difference

Set difference of A and B is

AB = {x S | x A and x / B }= {x S | x A and x Bc }

Note that AB 6= B A

A symmetrical set difference notion of A and B is

A\B = {x S | (x A or x B) and (x / A B)}

Note that A\B = B\A

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 6

De Morgans Law

Theorem: De Morgans Law

(A B)c = Ac Bc.

a Theorem is something that we need to prove from Axioms and Definations.

Proof: the basic idea is to show (A B)c Ac Bc and Ac Bc (A B)chold simultaneously.

Assume x (A B)c. This means that x / A B, which means x / Aand x / B. Together, this means that x Ac and x Bc, which leads tox Ac Bc.

Assume x Ac Bc. Then we know that x Ac and x Bc. This isequivalent to x / A and x / B. By definition it means that x / A B, whichmeans that x (A B)c.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 7

Disjoint Sets

Sample Space, S, is the set of all elementary outcomes.

Event, E, is a subset of S.

Definition: Two sets are disjoint if and only if the intersection is empty.

Definition: Sets A1, . . . , An are mutually disjoint if and only if AiAj = , forall i, j.

Definition: If A1, . . . , AN are collectively exhaustive if their union is S, i.e.,A1 . . . An = S or ni=1Ai = S.

The above definition of collectively exhaustive can be generalized to countablyinfinite number of Ais, i.e., using

i=1Ai = S.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 8

Partition of Sample Space

If {Ai}i=1 are both mutually disjoint and collectively exhaustive, then we callthem a partition of S.

A puzzle pattern, or a mosaic of the sample space S.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 9

Remarks

The book calls a partition an event space.

We will not follow this terminology for several reasons.

We will stick with partition.

Why do we care about partition?

Basic idea is that using partition we can express any event as a union of disjointevents, which may make the calculation of probability much easier.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 10

Axioms of Probability Theory

mathematics cannot tell you what it is, but what would be if

Probability Theory is based on several Axioms.

Axiom 1: Probability of any event A S is greater than or equal to 0, i.e.,

P [A] 0, A S.

Axiom 2: P [S] = 1. (something will happen).

Axiom 3: For any countable collection of mutually disjoint events A1, A2, . . ., wehave

P [A1 A2 . . .] = P [A1] + P [A2] + . . .

Theorems, propositions, lemmas, and corollaries need proof.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 11

Basic Theorems of Probability Theory

Theorem: P [] = 0.

Proof: Since P [S ] = P [S]. Using Axiom 3, we know that

P [S] + P [] = P [S] = P [] = 0.

Theorem: P [Ac] = 1 P [A].

Proof: Note that A and Ac are disjoint and that A Ac = S. Hence, we have

1 = P [S] = P [A Ac]= P [A] + P [Ac]

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 12

Basic Theorems of Probability Theory

Theorem: A B = P [A] P [B].

Proof: First note thatB = (B A) A.

Then, we have, by Axiom 3, that

P [B] = P [B A] + P [A].

Also note that by Axiom 1 we have P [B A] 0. Therefore, it is obvious that

P [B] P [A].

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 13

Basic Theorems of Probability Theory

Theorem: For every partition {B1, B2, B3, . . .} of S, it holds that

P [A] =i

P [A Bi], A S.

This is called Law of Total Probabilityextremely useful in engineering.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 14

Conditional Probability

Example: somebody is rolling a fair die behind a curtain.

Sample SpaceS = {1, 2, 3, 4, 5, 6}

P [{1}] = P [{2}] = . . . = P [{6}] = 1/6 is now the probability model.

Q: what is the probability of P [{2, 3}] (or, equivalently, P [{2} {3}])?

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 15

Conditional Probability

In practice, we sometimes can get side information from somewhere, whichcould change our probability model.

Suppose that there is a person who tells you that this time you have an evennumber.

Then, what is the probability of getting 3?

P [{3}|Even] = P [{3}|{2, 4, 6}] = 0.

What is the probability of getting 2?

P [{2}|Even] = P [{2}|{2, 4, 6}] = 1/3.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 16

More on Conditional Probability

Consider two events A and B.

Assume that we have the side information that B has already happened, we wouldlike to know what is the probability of A conditioned on that B has happened,i.e., P [A|B].

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 17

More on Conditional Probability

Consider two events A and B.

Assume that we have the side information that B has already happened, we wouldlike to know what is the probability of A conditioned on that B has happened,i.e., P [A|B].

Conjecture: P [A|B] = P [A B]?

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 18

More on Conditional Probability

P [A|B] = P [A B] is not correct for a couple of reasons:

P [A|B] should be larger than P [A B] by intuition. When A is a superset of B, P [A|B] = 1 but P [A B] = P [B].

Intuition: we should scale (normalize) P [A B] to represent P [A|B].

Definition: the probability of A conditioned on B is P [A|B] = P [AB]P [B] .

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 19

More on Conditional Probability First, 0 < P [B] 1. Then,

P [A|B] = P [A B]P [B]

P [A B];

i.e., the first bug is fixed.

Second, when B A, then

P [A|B] = P [A B]P [B]

=P [B]

P [B]= 1.

The second bug is fixed.

Example: Lets go back to the rolling dice example.

P [{4}|Even] = P [{4 Even}]P [Even]

=1/6

1/2= 1/3.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 20

Remarks on Conditional Probability

Remark 1: The definition of conditional probability automatically assumes that

P [B] > 0.

Remark 2: P [A|B] itself is a respectable probability measure, i.e., P [A|B]satisifes Axioms 1-3 of Probability Theory.

Remark 3: If A B = , then we have

P [A|B] = 0.

ECE353 Probability and Random Processes X. Fu, School of EECS, Oregon State University 21

Recommended