Introduction to Random Processes (1): Probability
Luiz DaSilva Professor of Telecommunications [email protected] +353-1-8963660
Brief review of set theory: a set q A set is a collection of elements
q May be finite, or countably or uncountably infinite
q Examples
q A finite set:
q A countably infinite set:
q An uncountably infinite set
2
Question q Consider an N-element set
q How many subsets of this set are there?
3
Set algebra q Union
q Intersection
q Complement
q Difference (reduction)
4
Set algebra example A = {odd integers between 0 and 10} B = {multiples of 3 between 1 and 10} A ∩ B = A U B = A \ B =
5
Disjoint and equal sets q Two sets are disjoint if:
q Two sets are equal if:
6
De Morgan’s Laws
7
Random experiments q A random experiment H is an experiment for
which the outcome is not known a priori
q The sample space Ω is the set of all possible outcomes of H
q An event E is a subset of Ω q The null set denotes the impossible event
8
Example A random experiment:
H = roll a fair die once
The sample space: An event:
9
Probability Defn: Probability is a set function P[•] that assigns to each event E in a σ-field F a number P[E] such that (i) (ii) (iii)
10
Axiom #1 P[Ø] = 0 Proof:
11
Axiom #2 P[E] = 1 – P[Ec] Proof:
12
Axiom #3 When Ei’s are disjoint: Proof: [A special case of property (iii) in the definition]
13
][][11∑==
=n
ii
n
ii EPEP∪
Axiom #4 P[E U F] = P[E] + P[F] – P[EF] Proof:
14
Conditional probability Defn: The probability of event E conditional on event F, with P[F] > 0, is Interpretation: Provability of event E given that event F has occurred
15
][][]|[
FPEFPFEP =
joint probability
Example: binary communication system
16
X Y
noisy channel
0
1
0
1
1 - ε
1 - ε
ε
ε
Theorem on total probability Thm: Let E1, …, En be disjoint and exhaustive (their union yields Ω), with P[Ei] ≠ 0. Then, for any event B, Proof:
17
][]|[][1
i
n
ii EPEBPBP ∑
=
=
Bayes’ theorem Thm: Let Eis be a partition of Ω (disjoint, exhaustive), with P[Ei] ≠ 0. Then, for any event B, with P[B] ≠ 0,
18
∑=
== n
jjj
iiii
EPEBP
EPEBPBPBEPBEP
1][]|[
][]|[][][]|[
Binary channel example revisited q If receiver decodes a ‘1,’ what is the probability
that a ‘1’ was sent?
19
Independence Defn: (i) Events E and F are independent if P[EF] = P[E]P[F] (ii) Events E1, …, En are independent if
20
][]...[
][][][][
][][][
121 ∏
=
=
≠≠∀=
≠∀=
n
iin
kjikji
jiji
EPEEEP
kjiEPEPEPEEEPjiEPEPEEP
!
