chapter 1 random signals and systems
TRANSCRIPT
Random
Signals and S
ystems
Chapter 1
Jitendra K T
ugnaitJam
es B D
avis Professor
Departm
ent of Electrical &
Com
puter Engineering
Auburn U
niversity
2A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
3A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
4A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Descriptions of P
robability
•R
elative frequency approach»
Physical approach
»Intuitive
»L
imited to relatively sim
ple problems
•A
xiomatic approach
»M
athematical approach
»T
heoretical
»C
an handle complicated problem
s
»E
LE
C 3800 is based on this approach
5A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Elem
entary Set T
heory
•A
setis a collection of elements
Ex:A
={1,2,3,4,5,6}
•set B
is a subsetof Aif all elem
ents of Bare in A
Ex:B
={1,2,3}
We denote this as
empty set:
equality: A=
B iff
and
BA
B A
A
B
BA
6A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Elem
entary Set T
heory cont’d
•union»
sum
»L
ogical OR
•intersection»
product
»L
ogical AN
D
•m
utually exclusive if
AB
A
B
AB
AB
B
A
AB
7A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Elem
entary Set T
heory cont’d
•com
plement
•difference
AB
AB
AB
AB
A
A
AB
AB
AA
B
AB
8A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach
•A
probability space
•A
n event is a subset of Sis the set of all possible
outcomes of an experim
ent
Ex:rolling a six-sided die
S=
{1,2,3,4,5,6}
S
Sthe space
is the certain event
is the impossible event
an event consisting of a single element is called
an elementary event
9A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
•T
o each event, we assign a probability
()
PA
Axiom
s of Probability
1) (
)0
PA
2)
1P
S
3)If
,thenA
B
PA
BP
AP
B
•A
ll probability theory can be derived from these
axioms.
10A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach -
Exam
ple
•T
he third axiom describes how
to compute the
probability of the sum of m
utually exclusive events:
•H
ow do w
e add non-mutually exclusive events?
PA
BP
AP
B
11A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach -
Exam
ple
Suppose
andare not m
utually exclusiveA
B
PA
BP
AA
B
PA
PA
B
also
BA
BA
B
PB
PA
BP
AB
PA
BP
BP
AB
PA
BP
AP
BP
AB
A
B
12A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Ex:
six-sided dieS
={1,2,3,4,5,6}
16
iP
Ri=
1,…,6(m
utually exclusive){1,3}
{1}{3}
A
11
1{1}
{3}6
63
PA
PP
13A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Ex:six-sided die
S=
{1,2,3,4,5,6}
{1,3}A
{3,5}B
{3}A
B
PA
BP
AP
BP
AB
{1,3,5}A
B
11
11
33
62
or
{1}
{3}{5}
PA
BP
PP
11
11
66
62
14A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Ex:
A dodecahedron is a solid w
ith twelve sides and is often used
to display the twelve m
onths of the year. When this object is
rolled, let the outcome be taken as the m
onth appearing on the upper face. A
lso let A=
{January}, B=
{Any m
onth with exactly
30 days}, and C=
{Any m
onth with exactly
31 days}. Find
Months w
ith 31 days: January, March, M
ay, July, August, O
ctober,Decem
ber (7)M
onths with 30 days: A
pril, June, September, N
ovember (4)
PA
C
PA
C
PB
C
PB
C
15A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Ex:
A dodecahedron is a solid w
ith twelve sides and is often used
to display the twelve m
onths of the year. When this object is
rolled, let the outcome be taken as the m
onth appearing on the upper face. A
lso let A=
{January}, B=
{Any m
onth with 30
days}, and C=
{Any m
onth with 31 days}. F
ind
71
112
1212
712 ()
()
()
PA
CP
AP
CP
AC
112(January)
PA
CP
16A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Ex:
A dodecahedron is a solid w
ith twelve sides and is often used
to display the twelve m
onths of the year. When this object is
rolled, let the outcome be taken as the m
onth appearing on the upper face. A
lso let A=
{January}, B=
{Any m
onth with 30
days}, and C=
{Any m
onth with 31 days}. F
ind
7412
12
1112 ()
()
()
0
PB
CP
BP
CP
BC
()
0P
BC
P
17A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Axiom
atic Approach cont’d
Three non-m
utually exclusive events
PA
BC
PA
PB
PC
A
BC
PA
BP
AC
PB
C
PA
BC
18A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability of a C
omplem
ent
()
?
()
()
1(
)(
)
()
1(
)
PBS
BB
PS
PB
B
PB
PB
PB
PB
19A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hom
ework
•1-7.1
•1-7.2
•1-7.5
20A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Conditional P
robability
|P
AB
PA
BP
B
0P
B
•W
e can show that a conditional probability
is a reallyprobability satisfying:
|0
|1
PA
B
PB
S
If,
||
|P
AC
BP
AB
PC
B
AC
21A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Conditional P
robability cont’d
Ex:six-sided die
{1,2}
{2,4,6}
AB
find the probability of rolling a 1 or 2 giventhat the outcom
e is an even number
{2}1
61
/1
23
PA
BP
PA
BP
BP
B
Sim
ilarly, probability of even number given 1 or 2
16
1/
13
2
PA
BP
BA
PA
Note that, except in special cases, P
(A|B
) ≠P(B
|A)
22A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:
A m
anufacturer of electronic equipment purchases 1000 IC
s from
supplier A, 2000 IC
s from supplier B
, and 3000 ICs from
supplier C
. Testing reveals that the conditional probability of an
IC
failing during
burn-in is,
for devices
from
each of
the suppliers
|0.1
PF
A
|0.05
PF
B
|0.08
PF
C
The IC
s from all suppliers are m
ixed together and one device isselected at random
.
a)W
hat is the probability that it will fail during burn-in?
b)G
iven that the device fails, what is the probability that the
device came from
supplier A?
23A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Total P
robability
1n
AA
S
Suppose w
e have n mutually
exclusive eventsthat span the probability space
12
nA
AA
S
Consider an event
B
S
1
2
12
n
n
BB
AB
AB
A
PB
PB
AP
BA
PB
A
S
/i
ii
PB
AP
BA
PA
(from conditional probability)
Then,
11
22
||
|n
nP
BP
BA
PA
PB
AP
AP
BA
PA
(total probability)
24A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple•
A candy m
achine has 10 buttons»
1 button never works
»2 buttons w
ork half the time
»7 buttons w
ork all the time
•A
coin is inserted and a button is pushed at random•
What is the probability no candy is received?
•N
o candy (NC
) can happen two w
ays:»
Push button that never w
orks (100%)
»P
ush one of the buttons that work half the tim
e (50%)
•W
e must w
eight each possibility by the chance it occurs»
Button that never w
orks (10%)
»O
ne of the buttons that work half the tim
e (20%)
25A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
NC
never
half
(|
)1.00
(|
)0.50
PN
CB
PN
CB
Exam
ple
nevernever
halfhalf
()
(|
)(
)(
|)
()
(1.00)(0.10)(0.50)(0.20)
0.20
PN
CP
NC
BP
BP
NC
BP
B
never
half
()
0.10
()
0.20
PB
PB
never
B
halfB
goodB
26A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:
A m
anufacturer of electronic equipment purchases 1000 IC
s from
supplier A, 2000 IC
s from supplier B
, and 3000 ICs from
supplier C
. Testing reveals that the conditional probability of an
IC
failing during
burn-in is,
for devices
from
each of
the suppliers
|0.1
PF
A
|0.05
PF
B
|0.08
PF
C
The IC
s from all suppliers are m
ixed together and one device isselected at random
.
a)W
hat is the probability that it will fail during burn-in?
b)G
iven that the device fails, what is the probability that the
device came from
supplier A?
27A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Total P
robability cont’d
PA
PB
PC
ABC
F
are known a prioriprobabilities because they are
known before the experim
ent is performed
|P
AF
is called an a posterioriprobability because it is applied after the experim
ent is performed
28A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Total P
robability cont’d
//
ii
ii
PA
BP
AB
PB
PB
AP
A
//
ii
i
PB
AP
AP
AB
PB
We can relate the a priori probability to the a posteriori probability by:
This is called B
ayes Theorem
.
11
22
//
/n
nP
BP
BA
PA
PB
AP
AP
BA
PA
(from total probability)
11
22
//
//
/i
ii
nn
PB
AP
AP
AB
PB
AP
AP
BA
PA
PB
AP
A
29A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Total P
robability cont’dL
et’s return to the problem:
A m
anufacturer of electronic equipment
purchases 1000 ICs from
supplier A,
2000 ICs from
supplier B, and 3000
ICs from
supplier C. T
esting reveals that the conditional probability of an IC
failing
during burn-in
is, for
devices from each of the suppliers
/0.1
PF
A
/0.05
PF
B
/0.08
PF
C
The
ICs
from
all suppliers
are m
ixed together and one device is selected at random
.a) W
hat is the probability that it will fail
during burn-in?b) G
iven that the device fails, what is the
probability that the device came from
supplier A
?
16P
A
a)
26
PB
36
PC
//
/P
FP
FA
PA
PF
BP
BP
FC
PC
12
30.1
0.050.08
0.07336
66
b)
/
/P
FA
PA
PA
FP
F1
0.16
0.22740.0733
30A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•A
test for cancer has the following properties
»If a person has cancer, the test w
ill state they have cancer 95%
of the time.
–M
edical literature: sensitivity=95%
–
»If a person does not have cancer, the test w
ill state they do not have cancer 95%
of the time.
–M
edical literature specificity=95%
–
•T
he cancer rate in 20-29 year olds is 0.2076%*
»
test(
|)
0.95P
CC
test(
|)
0.95P
CC
()
0.002076P
C
* All cancers, N
IH N
CI
SE
ER
Cancer S
tatistics Review
31A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
•Q
UE
STIO
N: Should this test be used to screen
20 year olds for cancer?
•R
eally what w
e are interested in is»
If the test says I have cancer, what is the probability
I really have cancer?–
Positive predictive value (PPV
)
–
»If the test says I do not have cancer, w
hat is the probability I really do not have cancer?
–N
egative predictive value (NP
V)
–
test(
|)
PC
C
test(
|)
PC
C
32A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
•If a 20-29 year old tests positive for cancer, there is only a 3.8%
chance they actually have cancer.•
CO
NC
LU
SIO
N: D
o not use test for screening 20-29 year olds.
testtest
test
test
testtest
(|
)(
)(
|)
()(
|)
()
(|
)(
)(
|)
()
0.950.002076
0.950.002076
0.05(1
0.002076)
0.038
PC
CP
CP
CC
PC
PC
CP
C
PC
CP
CP
CC
PC
33A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
Age
Group
PP
V (%
)
20-293.8023
30-399.2326
40-4919.2343
50-5936.8524
60-6956.8987
70-7969.1181
80+69.2864
34A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
•10,000 people
•P
(cancer)=0.002 (2%
)»
20 have cancer
»9980 do not have cancer
19(
|)
0.037(3.7%
)518
testP
CC
19499
518
19481
9482
209980
CC
testC
testC
35A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hom
ework
•1-8.1
•1-8.2
36A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence
•E
xample: Flip a coin tw
o times
»W
hat is the probability of two heads?
»P
(H,H
)=P
(H)P
(H)=
(0.5)(0.5)=0.25
•P
reviously, we found that if tw
o events are independent, their joint probability is the product of their m
arginal probabilities.
37A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence
•T
wo events A
and Bare independentif and only
if (iff)
•In som
e cases, independence is assumed or
determined by the physics of the situation
•In other cases, independence can be established m
athematically by show
ing that the above equation holds.
P
AB
PA
PB
38A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence
•T
hree events, A1 , A
2 , A3 , are independent iff
•pair w
ise comparison is not sufficient. F
or nevents, com
parisons are required
12
12
13
13
23
23
12
31
23
PA
AP
AP
A
PA
AP
AP
A
PA
AP
AP
A
PA
AA
PA
PA
PA
21
nn
39A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
Ex:rolling a pair of dice:
event A: getting a total of 7
event B: getting a total of 11
Are A
and Bindependent?
NO
! -m
utu
ally exclusive even
ts are never
mu
tually exclu
sive events are n
ever in
dep
end
ent!
ind
epen
den
t!(if one occurs, the other can not).
40A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
Ex:rolling a pair of dice
event A: getting a total of odd num
ber
event B: getting a total of 11
(these events are not mutually exclusive)
AB
B
since ,
BA
21
3618
PA
BP
B
12
PA
11
1
182
18P
AB
PA
PB
Thus, A
and B are n
otstatistically independent.
41A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
Ex:rolling single die
{1,2,3}
{3,4}
AB
1213
PA
PB
11
1{3}
62
3P
AB
PP
AP
B
Aand B
are independent, although the physical significance of thisis not intuitively clear.
42A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
Ex:A
card is selected at random from
a standard deck of 52 cards. L
et A be the event of selecting an ace, and let B
the event of selecting a spade. A
re these events statistically independent?
452P
A
1352P
B
152P
AB
413
1
5252
52P
AP
BP
AB
Aand B
are independent.
43A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence
•C
omplem
ents»
If Aand B
are independent, then so are
»P
roof:
and and
and A
BA
BA
B
()
([]
[])
()
()
()
()
()
()
()
()
()
()[1
()]
()
()
PA
PA
BA
B
PA
PA
BP
AB
PA
BP
AP
AB
PA
PA
PB
PA
PB
PA
PB
44A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability of a D
ifference
•F
or any events Aand B
,
()
()
()
()
PA
BP
AB
PA
PA
B
Note:
()
()
()
()
()
()
PA
PA
BP
AB
PA
BP
AP
AB
45A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
Ex:In the sw
itching circuit shown below
, the switches are
assumed to operate random
ly and independently.
A
BD
C
If each switch has a probability of 0.2 of being closed, find the
probability that there is a complete path through the circuit.
(Path 1: A
and D are closed
Path 2: A
, B and C
are closed)
Path 2
Path 1
46A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Independence cont’d
event P1: path 1 complete
event P2: path 2 complete
12
12
12
PP
PP
PP
PP
PP
11
11
55
251
11
12
55
5125
PP
PP
independence used
11
11
11
25
55
5625
PP
P
11
11
20.0464
25125
625P
PP
47A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hom
ework
•1-9.2
48A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
bined Experim
ents
•S
o far, the probability space, S,has been associated w
ith a single experiment.
•N
ow consider tw
o experiments:
»R
olling a die and flipping a coin
»R
olling a die 2 times
•W
e call these experiments com
bined experiments
49A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
bined Experim
ents
•S
uppose that there are two experim
ents:
•T
he probability space of the combined experim
entis the C
artesian product of the two spaces:
•T
he elements of S
are the ordered pairs
12
12
{,
,}
{,
,} nm
12
SS
12
S=
SS
1
12
2,
,,
,,n
m
50A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
bined Experim
ents cont’d
Ex:L
et’s take the experiments of rolling a die and tossing a coin
{1,2,3,4,5,6}
{H,T
}
12
SS
1,H
,1,T
,2,H
,6,T
1
2S
=S
S
Sam
e things hold for subsets (events):If subset
is an event inIf subset
is an event in 1
A1
S
2A
2S
thenis an event in
12
AA
A
S
51A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
bined Experim
ents cont’d
Ex:
12
12
{1,3}
{H}
1,H,
3,H
AAAA
A
If the two experim
ents are independent, then:
12
12
PA
PA
AP
AP
A
1
13P
A
2
12P
A
12
11
1
32
6P
AP
AP
A
53A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials•
The sam
e experiment is repeated n
times and it is desired to
find the probability that a particular event occurs exactly kof
these times.
•P
(exactly two 4’s in any order in 3 rolls)=
?
Let
rolling a 4A
A
not rolling a 4
16P
A
56P
A
Total of 8 possible outcom
es, but only 3 of them have exactly
two 4’s
11
5prob
66
6
AA
A1
51
prob6
66
A
AA
51
1prob
66
6
P(exactly tw
o 4’s in any order in 3 rolls)2
11
53
66
(the three situations are m
utually exclusive so we can sum
them)
AA
A
54A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials cont’d
•In general, probability event A
occurs ktim
es in ntrials can be
given as:
kn
kn
nP
kp
qk
w
here
1
pP
A
qP
Ap
Assum
ptions:1) independent events2) p
and qare sam
e for each event
!
!!
nn
kk
nk
56A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Binom
ial Coefficients
1
11
12
1
13
31
14
64
1
15
1010
51
()
na
b
Pascal’s T
riangle
57A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials cont’d
Ex:32-bit “w
ords”in m
emory
310
probability of an incorrect bit
0.001
0.999
pq
P(1 error in a w
ord)
1
31
32
321
0.0010.999
1
0.031
P
P(no error)
0
32
32
320
0.0010.999
0
0.9685
P
58A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials cont’d
Ex: toss a coin 4 tim
es and what is P(at least tw
o heads)?
P(at least tw
o heads)
44
42
34
PP
P
22
4
31
4
40
4
41
13
22
22
8
41
11
33
22
4
41
11
44
22
16
PPP
P(at least tw
o heads)3
11
11
84
1616
59A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials cont’d
Ex:toss a pair of dice 8 tim
es
a) Probability of a 7 exactly 4 tim
es
A: rolling 7
A=
{(1,6),(6,1),(2,5),(5,2),(3,4),(4,3)}
44
8
61
()
366
15
16
6
81
54
0.026054
66
PA
p
PA
q
P
b) An 11 occurs 2 tim
esB
: rolling 11B
={(5,6),(6,5)}
26
8
21
3618
117
118
18
81
172
0.06132
1818
PB
p
PB
q
P
60A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Bernoulli T
rials cont’d
1361
351
3636
PC
p
q
8
81
01
PP
8
8
17
8
350
0.798236
81
351
0.18251
3636
PP
c) Probability of a 12 m
ore than onceC
: rolling 12B
={(6,6)}
P(m
ore than one 12)
P(m
ore than one 12)
1
0.79820.1825
0.0193
61A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•S
uppose you are a design engineer for a Mars
spacecraft.»
Need to com
municate w
ith Earth
»U
se 12-bit words
»T
he probability of a single bit error is 0.001
•C
an use two types of codes
»N
o code–
Send 12 data bits in a 12-bit codew
ord
»E
xtended Golay code
–E
ncode 12 data bits in a 24-bit codeword
–C
an correct errors of 3 bits or less
•W
hich code is more reliable?
62A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
•N
o code (12 data bits in 12 code bits)»
Let A
= probability of a one-bit error in transm
ission–
p = 0.001
–q =
0.999
•P
robability of a transmission error
012
12
012
12(error)
1(0)
10
1(0.001)
(0.999)
10.9881
0.012
Pp
pq
63A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple cont’d
•E
xtended Golay code (12 data bits in 24 code bits, can
correct 3-bit errors)
•P
robability of a transmission error
024
123
222
321
2424
2424
-4-6
2424
2424
(0)(1)
(2)(3)
01
23
0.97630.0235
=2.70
10=
1.9810
pp
qp
pq
pp
qp
pq
2424
2424
(error)1
[(0)
(1)(2)
(3)]P
pp
pp
8(error)
1.0410
P
←U
se Exten
ded
Golay cod
e!
This code w
as used by the Voyager spacecraft to send back im
agesof Saturn and Jupiter.
64A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
DeM
oivre-Laplace T
heorem
1npq
knp
npq
If1)2)
is on the order of or less than
2
21
2k
npnpq
kn
kn
nP
kp
qe
knpq
This theorem
is used to simplify the evaluation of binom
ial coefficients and the large pow
ers of pand q
by approximating them
.
•When n
gets large, it is difficult to compute P
n (k).•U
se the DeM
oivre-Laplace approxim
ation
65A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
DeM
oivre-Laplace T
heorem cont’d
Ex:8000 character file transfer
0.001 chance of one character error
8000
0.001
0.999
npq
a) P(no error)
80004
10.001
3.3410
b) P(exactly 10 errors)
2
107990
108
160.999
80000.001
0.99910
1
28000
0.0010.999
0.1099
e
c) What should be p such that
P(no error)
0.99
8000log0.99
8000
log0.99
80006
10.99
8000log
1log
0.99
110
110
1.25610
p
p
p
p