chapter 4 random signals and systemstugnajk/elec3800_ch4_11s.pdf · random signals and systems...
TRANSCRIPT
Random
Signals and S
ystems
Chapter 4
JitendraK
Tugnait
James B
Davis P
rofessor
Departm
ent of Electrical &
Com
puter Engineering
Auburn U
niversity
2A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Statistics
•P
robability and Random
Variables
»A
ssume w
e know how
the RV
behaves –either by
assumption or physics of the problem
.
•S
tatistics»
Analysis of data (specific values of an R
V)
3A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Applications of S
tatistics
•S
ampling T
heory»
How
do we select sam
ples from a large population?
»E
xamples
–P
ublic opinion polls
–IC
manufacturing
•H
ypothesis Testing
»H
ow do w
e decide which of tw
o hypotheses are true?
»D
oes my electronic fuel injector really im
prove gas mileage?
•L
inear Regression
»D
erive a linear equation that describes the data.
»D
oes my autom
atic target detector work as w
ell as a manual
one?
4A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Term
inology
•Population (N
)»
Collection of data being studied–
All registered voters
–A
ll IC’s from
a manufacturing line
•S
ample (n)
»P
art of the population that is selected at random–
Voters contacted in a survey
–IC
’s selected for test
•N
ote:»
N=
population size»
n = sam
ple size
5A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple –A
bin with 1000 R
esistors
0100
200300
400500
600700
800900
10000
200
400
600
800
1000
1200
1400
Resistor N
umber
Resistor Values ()
What is the m
ean of these resistors?
Random
ly select 3 resistors
We w
ant to estimate the population m
ean.T
he average of these three values is called the sample m
ean.W
e use the sample m
ean to estimate the population m
ean.
6A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•L
et Xbe the value of a resistor from
this bin of 1000 resistors
•S
elect 3 resistors at random
•T
he mean of this sam
ple (called the sample
mean) is:
3
11
11
11963
n
ii
ii
xx
xn
123
1133
1205
1250
xxx
7A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•H
ow m
any resistors do we need in our sam
ple to ensure that the sam
ple mean that is w
ithin ±1
of the true mean?
8A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
More T
erminology / N
otation
Notation: (capital letters; R
V’s, low
er case: particular value)
random variable representing the population
an arbitrary sample taken from
the population
a particular value of
a particular sample taken from
the population
true (population)
ii
XXxX
xXm
ean (an unknown constant,
an RV
)
ˆ theoreticalsam
ple mean
actual sample m
ean (the mean of a particular sam
ple)
not
Xx
9A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•U
sually we are interested in the m
ean of an arbitrary set of sam
ples, Xi
»N
ote that is itself a random variable
•S
uppose we picked another set of 3 resistors from
the batch
•T
he mean of a particular sam
ple depends on which 3
resistors you pick.
1
1ˆ
n
ii
XX
n
X̂
123
1275
1198
1185
xxx
the mean of this sam
ple is 1219.33
(previousm
eanw
as1196) x
10A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
010
2030
4050
6070
8090
1000
200
400
600
800
1000
1200
1400
Trial
Sample Mean ()S
ample m
eans for n=3
11A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Tw
o Expressions for the S
ample M
ean
•N
ote we have tw
o expressions for the sample m
ean
•T
he mean of n
arbitrary samples
»A
random variable used for theoretical analysis
•T
he mean of n
actual samples
»A
number com
puted from the data
•is a particular value of
»Just like in random
variables, e.g. P(X
<x)
1
1ˆ
n
ii
XX
n
1
1n
ii
xx
n
X̂x
12A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
•W
hat is the mean of the sam
ple mean ?
•O
n average, the sample m
ean equals the true m
ean»
This is called an unbiased
estimator
Mean of the S
ample M
ean
1
11
11
1ˆ
nn
n
ii
ii
i
EX
EX
EX
XX
nn
n
population mean (true m
ean)
where
is an arbitrary sample from
the population i
XX
13A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
010
2030
4050
6070
8090
1000
200
400
600
800
1000
1200
1400
Trial
Sample Mean ()
Sam
ple means for n=
3
Exam
ple
Population m
ean =
1200
How
precisely can we estim
ate the mean?
How
precisely can we estim
ate the mean?
14A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Variance of the S
ample M
ean
•O
n average the sample m
ean equals the true m
ean
•H
ow m
uch does the sample m
ean vary about the true m
ean?»
How
precise is the sample m
ean?
•W
e need to compute the variance of the sam
ple m
ean
15A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Variance of the S
ample M
ean
Assum
e N>
>n
(removing n
samples does not alter the pop. m
ean)
2
2
1
1ˆ
Var
n
ii
XE
XX
n
22
22
11
11
2
2
11
assuming
and are independent
for
for
nn
nn
ij
ij
ij
ij
ij
ij
EX
XX
EX
XX
nn
XX
Xi
jE
XX
Xi
j
22
22
2
1ˆ
Var
Xn
Xn
nX
Xn
22
2X
X
nn
16A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
True variance =
σ2
= 50
2=
2500
22
2500ˆ
Var(
)833.33
3
ˆˆ
Std(
)V
ar()
28.87
Xn
XX
22
2500ˆ
Var(
)83.33
30
ˆˆ
Std(
)V
ar()
9.13
Xn
XX
010
2030
4050
6070
8090
1000
200
400
600
800
1000
1200
1400
Trial
Sample Mean ()
Sam
ple means for n=
30
010
2030
4050
6070
8090
1000
200
400
600
800
1000
1200
1400
Trial
Sample Mean ()
Sam
ple means for n=
3
17A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
•T
he variance of the sample m
ean is
»T
he sample m
ean gets closer to the true mean as the
sample size increases
•T
he above formula w
orks when N
>>
n.»
If the population size is small, it also w
orks when
the samples can be returned to the population after
testing.–
Called sam
pling with replacem
ent.
Variance of the S
ample M
ean
2
ˆV
ar=
X
n
18A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Sam
pling Without R
eplacement
•W
hen the population size is small and the
samples are not replaced, the m
ean of the rem
aining population may be different from
the original population. In this case
2
-ˆ
Var
=-1
Nn
Xn
N
19A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:endless production line produces diodes random
ly tested forreverse current and forw
ard current. If
has truem
ean and variance of , how
many diodes m
ust betested to get a sam
ple mean w
ith 5% of the true m
ean?
1I
1I
1I
612
10&
10A
61
128
Var
0.0510
105
10
400 I
nn
2ˆ
,souse
Var(
)N
nX
n
20A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:A
production line produces 30 diodesrandom
ly tested without
replacementfor reverse current I-1
and forward current I1 . If I-1
has true mean and variance of 10
-6and 10
-12A
, how m
any diodes m
ust be tested to get a sample m
ean with 5%
of the true mean?
61
128
0.0510
1030
510
301
27.9728
Var
I
n
n
n
2ˆ
issm
all,souse
Var(
)1
Nn
NX
nN
Note that
when N
=n, the
variance of the sam
ple m
ean is zero.
21A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple –A
bin with 1000 R
esistors
0100
200300
400500
600700
800900
10000
200
400
600
800
1000
1200
1400
Resistor N
umber
Resistor Values ()
We know
how to estim
ate the mean.
What is the variance of these resistors?
22A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Sam
ple Variance
221
1
1ˆ
n
ii
SX
Xn
•S
ince we can estim
ate the population mean, w
e can estimate the
population variance with the sam
ple variance
How
ever,
2
22
1
1ˆ
XX
nE
Sn
This estim
ate of the population variance is biased.
Instead, we use
22
211
1ˆ
11
n
ii
nS
SX
Xn
n
This is called the sam
ple variance.
2
2XE
S
This estim
ate of the sample variance is unbiased.
23A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple (n=3)
010
2030
4050
6070
8090
1000
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Trial
Sample Variance (2)
Sam
ple variances for n=3
Population
variance =
25002
24A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple (n=30)
010
2030
4050
6070
8090
1000
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Trial
Sample Variance (2)
Sam
ple variances for n=30
Population
variance =
25002
25A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Variance of the S
ample V
ariance
•T
he variance of the sample variance is
»W
here μ4
is the fourth central mom
ent of the population.
•T
his formula is valid only w
hen N>
>n
•A
measure of how
much the sam
ple variance fluctuates around the population variance
44
22
Var
1
nS
n
26A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Higher-O
rder Central M
oments
•G
aussian
•U
niform»
Can assum
e, without loss of generality that:
–the range is -Δ
x/2 to Δx/2
–n
is even (odd central mom
ents are zero)
11
11
12
2
1
22
12
(1)
(1)
22
(1)2
(1)2
xx
nn
nn
nn
nn
nx
x
n
n
xx
xx
xE
XX
EX
xdx
xn
xn
xn
x
x
n
0 odd
13
5(
1) even
n
n
nE
XX
nn
27A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
putational Formula for S
ample V
ariance
2
2
21
12
1
1
22
12
1ˆ
11
1w
here
Or,
ˆ
1
nn
ii
ni
ii
i
n
ii
n
ii
nx
x
sx
xn
nn
xx
n
xn
x
sn
N
ote sample m
ean, not Σx
i
28A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:
207202184204206198197213191201
101
102
1
2
22
11
2
Sam
ple mean:
12003
200.310
10
401,825
Sam
ple variance:
11
11
110
401,825200.3
101
101
69.34 ii
ii
nn
ii
ii
xx
x
ns
xx
nn
n
29A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:
207202184204206198197213191201
If the sample size is sm
all compared to the
population size, the true variance is 75, and the true 4
thcentral m
oment is 16875,
2
Variance of the sam
ple mean:
75ˆ
Var(
)7.5
10X
n
4
22
42
2
Variance of the sam
ple variance:
()
10(16,87575
)V
ar()
1,388.9(
1)(10
1)
nS
n
30A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:S
uppose we sam
ple a random w
aveform of infinite duration
at equally spaced intervals.A
ssume the w
aveform has a true m
ean of 10 and true variance of9, and the sam
ples are independent.
What is the variance of the sam
ple mean ?
2
9ˆ
Var
Xn
n
29
100.02
0.04225
nn
How
many sam
ples do we need to estim
ate the mean w
ithin 2%of its true value?
31A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•S
uppose we use n=
225, and compute a sam
ple m
ean. Are w
e guaranteed that our value is w
ithin 2% of the true m
ean?»
NO
!!
•W
hat is the probability that is within 2%
of the true m
ean?»
We need the P
DF
ofto answ
er this question
X̂
X̂
32A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•In this case, the sam
ple is large (n=225) and the
central limit theorem
tells us that is Gaussian
regardless of what the P
DF’s of X
i are.»
Recall from
a previous slide that
10.210
9.810
ˆ9.8
10.210.2
9.80.2
0.2
11
21
12
0.84131
0.6826
PX
FF
2
ˆ10
9ˆ
Var(
)0.04
225
XXn
68% chance that the
sample m
ean is w
ithin 2% of the
true mean
X̂
33A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•T
o answer questions like this, w
e need to know
the probability distributions of:»
The sam
ple mean,
»T
he normalized sam
ple mean:
X̂
ˆ (true variance know
n)X
XZ
n
ˆ (true variance unknow
n)X
XT
Sn
Recall that norm
alized random variables have zero m
ean and unit variance. We
normalize a random
variable by subtracting the mean and dividing
by the standard deviation.
34A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•If the sam
ples are independent, σ2
known, and
»T
he sample size is large
»O
r, the Xi are G
aussian
•T
he sample m
ean is Gaussian w
ith
•T
he normalized
sample m
ean is also Gaussian
(30)
n
2ˆ
ˆV
arE
XX
Xn
X̂X
Zn
35A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•W
hat if the sample size is large and σ
2
unknown?
»T
he sample m
ean is Gaussian
»C
an use as an estimate of σ
2
–W
hen nis large, should be sm
all
•T
he normalized sam
ple mean is approxim
ately G
aussian
2S
2V
ar()
S X̂X
ZS
n
(30)
n
36A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:100 bipolar transistors
Want the m
ean value of the current gaintrue m
ean and variances are:
What is the probability that the sam
ple mean is betw
een 119 and 121?
2
2ˆ
120and
25A
S
ˆ119
120120
121120
ˆ(119
121)0.5
0.50.5
(2
2)
(2)(
2)
2(2)
1
2(0.9772)1
0.9544
XP
XPP
Z
37A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•W
hat do we do w
hen the sample size is sm
all and σ
2is unknow
n?
•T
he normalized sam
ple mean
»is not G
aussian even if the Xi are G
aussian because one R
V, , is divided by another,
X̂X
TS
n
X̂2
S
38A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•W
hat do we do w
hen the sample size is sm
all and σ
2is unknow
n?
39A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Probability D
istribution of the Sam
ple Mean
•If »
nis sm
all (n<30)
»σ
2is unknow
n
»X
i are Gaussian
•T
he normalized sam
ple mean
has a student’s t distributionw
ith n-1 degrees of freedom
X̂X
TS
n
40A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Student’s t D
istribution
12
2
12
12
v
T
vt
ft
vv
v
1
. gam
ma function
vn
12
1 1
2
As n
gets large, the tdistribution approaches the Gaussian distribution.
any 1
!integer
kk
kk
kk
41A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:C
ompute student’s t density for t=
1 and 4 degrees of freedom
52
22.5
11
14
42
T f
2.5
1.51.5
1.50.5
0.5
1.5
0.51.3293
2.5
1.32931
1.250.2147
41
T f
43A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
What does this have to do w
ith beer?
•T
he Student’s tdistribution was
discovered by William
Gossett for
use in statistical quality control
•G
ossett worked for G
uinness B
rewery, w
ho had a strict policy against em
ployees publishing under their ow
n names
•G
ossett published the paper under the pen nam
e “Student”
44A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
PD
F of the N
ormalized S
ample M
ean
small
large
known
unknown
are Gaussian
iX
are not Gaussian
iX
small
large
known
unknown
ZZ
TZw
ith S w
ith S Z Z
**
*special cases
2
n2
n
Z: G
aussian distributionT
: Student’s tdistribution
45A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Confidence Intervals
•S
uppose we are random
ly checking the values of resistors on a m
anufacturing line»
True m
ean is 100 and the true variance is 16
2
•If w
e randomly sam
ple a run of resistors and the sam
ple mean is 99
, is there a problem?
•T
o answer this question, w
e need to know the
expected range, or confidence interval, for the sam
ple mean
46A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Confidence Intervals
Define the q-percent confidence intervalas the interval w
ithin w
hich the sample m
ean will lie w
ith q% probability.
The q%
confidence interval for the sample m
ean is:
kis a constant that depends on q
and density of .
ˆk
kX
XX
nn
confidence limits
X̂
ˆ100
kX
n
Xk
Xn
qf
xdx
47A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:large population of resistors
Find 95%
confidence limits for n =
100 100
4
X
951.96 (T
able 4.1 in page 172 of the textbook)q
k
1.96
40.78
100ˆ
99.22100.78
k
n
X
A sam
ple mean of 99
could m
ean a problem w
ith the m
anufacturing line
48A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:large population of resistors
Find 95%
confidence limits for n =
9
100
4
X
951.96 (T
able 4.1 in page 172 of the textbook)q
k
1.96
42.61
9ˆ
97.39102.61
k
n
X
A w
ider confidence interval corresponds to a poorer
estimate
of the mean
49A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Confidence Intervals
•T
he confidence interval can also be expressed in terms of the
probability distribution function
•F
or small sam
ple size and unknown variance, use F
Tn-1
instead of Φ
100
ˆ100
ˆ
100
()
()
2(
)1
1(
)2
nn
nn
nn
n
q qP
Xk
XX
k
Xk
XX
kX
XX
qP
Pk
Zk
kk
k
k
50A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
•W
hat is kfor a 90%
confidence interval when
n=100?
•W
hat is kfor a 90%
confidence interval when
n=9?
0.901
()
0.951.64
2k
k
8
0.901
()
0.951.86
2T
Fk
k
51A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:random
ly sampled G
aussian waveform
The 95%
confidence interval is:
2
1094
Xsn
3.1829
3.1829
ˆ10
104
4ˆ
5.22714.773
XX
3
0.951
()
0.9753.182
2T
Fk
k
52A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hypothesis T
esting•
A particular car m
odel gets 26 mpg w
ith a standard deviation of 5 m
pg.
•A
new electronic fuel injection system
is thought to improve
gas mileage.
•36 gas m
ileage tests are performed w
ith mean 28.04 m
pg.»
Assum
e σis unaffected
•D
id the electronic fuel injector increase the gas mileage?
•T
o answer this and sim
ilar questions, we need to perform
a hypothesis test.
53A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hypothesis T
esting
•W
e make tw
o hypotheses:
•W
e assume that H
0is true (that is w
hy it needs equality), and test if the data is consistent w
ith this assum
ption.
•If not consistent, w
e reject hypothesis H0
and accept hypothesis H
1 .
01
: 26 (called the null hypothesis)
: 26 (called the alternative or research hypothesis)
HX
HX
54A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
.05
Hypothesis T
esting
•If w
e assume H
0is true, then the sam
ple mean has a
Gaussian distribution w
ith mean 26.
•W
e compute a critical value x
c such that
•It is easier to w
ork with the norm
alized sample m
ean.
ˆ0.95
cP
Xx
5From
table, 1.64
2627.36
36c
cc
zx
z
ˆ26
536
0.95c
XZP
Zz
56A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
.05
Hypothesis T
esting
•S
o if the gas mileage is unchanged, 95%
of the sam
ple means w
ill be ≤27.36 m
pg
•T
wo possibilities:
»W
e get lucky (i.e. 28.04 is one of the 5%)
»T
he true mean is higher than 26
•W
e choose the second possibility»
We reject H
0 and accept H1 (the gas m
ileage im
proved)
57A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Types of T
ests
Right-tailed
test(L
eft-sided test)H
1 : mean >
value
Left-tailed
test(R
ight-sided test)H
1 : mean <
value
Tw
o-tailed test
(Tw
o-sided test)H
1 : mean ≠
value
58A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Norm
alized Sam
ple Mean
•In the previous exam
ple, we com
puted xc ,
which w
e compared to the sam
ple mean,
•In practice, how
ever, it is usually easier to norm
alize
and then compare z
to zc
•T
he normalized sam
ple mean is called a test
statistic.
x
xx
Xz
n
If
is unknown, use
.
If the sample size is sm
all, the
normalized sam
ple mean w
ill
have a student's t distribution.
s
59A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Hypothesis T
esting Procedure
•C
onstruct two hypotheses: H
0 and H1
»H
1should be the negation of H
0
»E
quality should be with H
0
•C
ompute test statistic (norm
alized sample m
ean) , assum
ing H0
is true
•C
ompute critical value of test statistic.
•If
(Gaussian)
xX
zn
(Student's
)
xX
ts
nt
~G
aussian
for large
xX
zs
nn
111
accept
(right-tailed)
accept
(left-tailed)
or <
accept (tw
o-tailed)
cc
cc
zz
H
zz
H
zz
zz
H
60A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
puting Critical V
alues
1)R
ight-tailed
2)L
eft-tailed
3)T
wo-tailed
100c
qP
Zz
100c
qz
100
11
1100
100
c
cc
qP
Zz
zz
100c
qz
100
1100
100
cc
cc
cc
qP
zZ
z
zz
zz
21
100c
qz
1
1002c
q
z
*For S
tudent’s t, replace Φw
ith Fγ and z
c with t
γ
61A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Com
puting Critical V
alues cont’d
Right-tailed
Left-tailed
Tw
o-tailed
ZT
100c
qz
100c
qz
1
1002c
q
z
100T
c
qF
t
100
Tc
qF
t
1
1002T
c
q
Ft
degrees of freedom
62A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:A
manufacturer claim
s the breakdown voltage of his capacitors is
300 V or greater.
100 capacitors tested:Is this claim
valid at the 99% confidence level?
290Vx
40Vs
01
Left-tailed test
:300 (claim
valid)
:300 (claim
invalid)
Sam
ple size is large (>
30) use w
ith =
290300
2.540
10
0.99
10.99
11
0.99
0.99
2.332.33
2.5
c
cc
c
cc
c
HX
HX
nZ
s
zPZ
zzz
z
zz
zz
0reject
(claim invalid)
H
cz
0accept
H1
acceptH
64A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:S
ame as before but w
ith n =9
Sam
ple size is small and σ
2is unknow
n so use:
8
8
8
8
8 290300
0.75(t distribution w
ith 8 degrees of freedom)
409
0.99
10.99
11
0.99
0.992.896
2.896
0.75accept hypothesis (claim
invalid)
c
TcT
c
Tc
cc
c
tPT
t
FtF
t
Ft
tt
zz
cz
0accept
H1
acceptH
66A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:(tw
o-tailed test)Z
ener diodes with m
ean breakdown voltage of 10V
. Deviations on
either side are undesirable because they are used as voltage regulators.T
est 100 diodes:Is claim
valid at 95% confidence level?10.3V
x
1.2Vs
01
: 10V
(claim valid)
: 10V
(claim invalid)10.3
10large sam
ple sizeuse
2.51.2
100
compute
such thatc
HX
HX
z
z
0.95c
cP
zZ
z
0.95c
cz
z
1
0.95c
cz
z
21
0.951.95
20.975
1.96c
cc
zz
z
Since 2.5 does not lie in the rangeclaim
not valid. -1.96
z1.96
0accept
H
1accept
H
67A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
Ex:sam
e as before but with n
=9
8
8
8
10.310
0.751.2
9
0.95
21
0.95
0.9752.306
cc
Tc
Tc
c
tPt
Tt
Ft
Ft
t
Since t =
0.75 lies in the intervalclaim
valid.2.306
2.306t
0accept
H
1accept
H
69A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Curve F
itting and Linear R
egression
Ex:F
our light bulbs are tested to determine the relationship
between lifetim
e and operation voltage.
iV
Hrs
1105
1400
2110
1200
3115
1120
4120
950
ix
iy
70A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Curve F
itting and Linear R
egression
•W
e would like a m
athematical relationship
between the operation voltage and lifetim
e so w
e could estimate lifetim
e for other operation voltages.»
For exam
ple, 90V, 112V
, 130V
•O
ne way to do this is to fit a straight line to the
data. This is called linear regression.
»W
e can fit other curves as well polynom
ials, F
ourier series.
71A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Curve F
itting and Linear R
egression
•O
ur data model is y
i = m
xi +
b
•O
nly need two points to solve for m
and b, but we w
ould like to use all the data available.
•D
efine di =
yi –
mx
i –b
•C
hoose mand b to m
inimize the sum
of squared differences
2
1
2
11
1
2
11
1
0
0
n
ii
i
nn
n
ii
ii
ii
i
nn
n
ii
ii
ii
i
SSDy
mx
bm
m
xy
mx
bx
xy
mx
bx
2
1
11
11
0
0
n
ii
i
nn
ii
ii
nn
ii
ii
SSDy
mx
bb
b
ym
xbn
ym
xnb
22
11
nn
ii
ii
i
SSDd
ym
xb
72A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
11
12
2
11
11
11
nn
n
ii
ii
ii
i
nn
ii
ii
n
iin
ii
nx
yx
y
m
nx
x
by
mx
xx
n
yy
n
Curve F
itting and Linear R
egression
Solving for m
and byields:
slope
y-intercept
73A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Curve F
itting and Linear R
egression cont’d
For our exam
ple:
111
4
450
4670
521800
n
iin
iin
ii
i n
xyxy
112.5
1167.5
xy
2
1
2
1
50750
5556900
n
iin
ii
xy
2
4521800
4504670
28.64
50750450
1167.528.6
112.54385
28.64385
mbyx
90V-28.6
904385
1811 hrs
112V-28.6
1124385
1181.8 hrs
130V-28.6
1304385
667 hrs
74A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Correlation C
oefficient
Can com
pute correlation coefficient for dataH
ow correlated is operation voltage w
ith lifetime?
11
1
22
22
11
11
nn
n
ii
ii
ii
i
nn
nn
ii
ii
ii
ii
nx
yx
y
r
nx
xn
yy
Com
pare to the theoretical formula:
For our example:
X
Y
EX
YX
Y
2
2
4521800
4504670
0.98834
50750450
4556900
4670r
highly negatively correlated
75A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
0.05r0.83
r
0.50r
0.80r
From
Denney Jr., T
S, et al, M
agnetic Resonance in M
edicine, 2003
76A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
0.09r0.91
r
0.63r
0.86r
77A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple
0.08r0.74
r
0.58r
0.60r
78A
U E
LE
CT
RIC
AL
AN
D C
OM
PU
TE
R E
NG
INE
ER
ING
Exam
ple