chapter 7: point estimationvucdinh.github.io/files/lecture07.pdf · week 1 chapter 1: descriptive...
Post on 13-Jul-2020
19 Views
Preview:
TRANSCRIPT
Chapter 7: Point Estimation
MATH 450
September 19th, 2017
MATH 450 Chapter 7: Point Estimation
Where are we?
Week 1 · · · · · ·• Chapter 1: Descriptive statistics
Week 2 · · · · · ·• Chapter 6: Statistics and SamplingDistributions
Week 4 · · · · · ·• Chapter 7: Point Estimation
Week 7 · · · · · ·• Chapter 8: Confidence Intervals
Week 10 · · · · · ·• Chapter 9: Test of Hypothesis
Week 13 · · · · · ·• Two-sample inference, ANOVA, regression
MATH 450 Chapter 7: Point Estimation
Overview
7.1 Point estimate
unbiased estimatormean squared errorbootstrap
7.2 Methods of point estimation
method of momentsmethod of maximum likelihood.
7.3 Sufficient statistic
7.4 Information and Efficiency
Large sample properties of the maximum likelihood estimator
MATH 450 Chapter 7: Point Estimation
Questions for this chapter
given population parameter θ (e.g., population mean µ,population variance σ2)
use a random sample sample X1,X2, . . . ,Xn to find a goodestimate θ̂ of θ
MATH 450 Chapter 7: Point Estimation
Point estimate
Definition
A point estimate θ̂ of a parameter θ is a single number that can beregarded as a sensible value for θ.
⇒ a point estimate are computed from a sample⇒ a point estimate is a statistic⇒ a point estimate is random
MATH 450 Chapter 7: Point Estimation
Example 1: population mean and standard deviation
Last week, we saw a lot of:“Let X1,X2, . . . ,Xn be a random sample from a distributionwith population mean µ and standard deviation σ”
What are the population parameters?
What is a good estimate of µ?
What is a good estimate of σ?
MATH 450 Chapter 7: Point Estimation
Example 2: Linear regression
Model:Yi = β0 + β1Xi + εi , εi ∼ N (0, σ2)
MATH 450 Chapter 7: Point Estimation
Example 3: Phylogenetics
MATH 450 Chapter 7: Point Estimation
How should the next vaccines be designed?
MATH 450 Chapter 7: Point Estimation
Point estimate
population parameter =⇒ sample =⇒ estimate
θ =⇒ X1,X2, . . . ,Xn =⇒ θ̂
MATH 450 Chapter 7: Point Estimation
Mean Squared Error
Measuring error of estimation
|θ̂ − θ| or (θ̂ − θ)2
The error of estimation is random
Definition
The mean squared error of an estimator θ̂ is
E [(θ̂ − θ)2]
MATH 450 Chapter 7: Point Estimation
Bias-variance decomposition
Problem
Recall that the variance of random variable X is computed by
V (X ) = E [X 2]− (EX )2
Prove that
MSE (θ̂) = E [(θ̂ − θ)2] = V (θ̂) +(E (θ̂)− θ
)2Bias-variance decomposition
Mean squared error = variance of estimator + (bias)2
MATH 450 Chapter 7: Point Estimation
Unbiased estimators
Definition
A point estimator θ̂ is said to be an unbiased estimator of θ if
E (θ̂) = θ
for every possible value of θ.
Unbiased estimator
⇔ Bias = 0
⇔ Mean squared error = variance of estimator
MATH 450 Chapter 7: Point Estimation
Sample mean as an unbiased estimator
Proposition
If X1,X2, . . . ,Xn is a random sample from a distribution with meanµ, then X̄ is an unbiased estimator of µ.
Proof: E (X̄ ) = µ.
LetT = a1X1 + a2X2 + . . .+ anXn,
then the mean and of T can be computed by
E (T ) = a1E (X1) + a2E (X2) + . . .+ anE (Xn)
Let a1 = a2 = . . . an = 1/n
MATH 450 Chapter 7: Point Estimation
Sample variance
Problem
Let
S2 =1
n − 1
[(∑X 2i
)− 1
n
(∑Xi
)2]Compute E (S2)
Step 1:
S2 =1
n − 1
[(∑X 2i
)− n(X̄ )2
]Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]
MATH 450 Chapter 7: Point Estimation
Sample variance
Step 1:
S2 =1
n − 1
[(∑X 2i
)− n(X̄ )2
]Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]Recall:
Var [Y ] = E (Y 2)− (EY )2
MATH 450 Chapter 7: Point Estimation
Sample variance
Step 2:
E [S2] =1
n − 1
[(∑E [X 2
i ])− nE [(X̄ )2]
]Recall:
Var [Y ] = E (Y 2)− (EY )2
Step 3:
E [S2] =1
n − 1
[(∑Var [Xi ] + (E [Xi ])
2)− n
(E [(X̄ )2]
)]=
1
n − 1
[(∑σ2 + µ2
)− n
(Var [X̄ ] + (E [X̄ ])2
)]
MATH 450 Chapter 7: Point Estimation
Sample variance
Step 3:
E [S2] =1
n − 1
[(∑Var [Xi ] + (E [Xi ])
2)− n
(E [(X̄ )2]
)]=
1
n − 1
[(∑σ2 + µ2
)− n
(Var [X̄ ] + (E [X̄ ])2
)]Recall:
Var [X̄ ] =σ2
n, E [X̄ ] = µ
Step 4:
E [S2] =1
n − 1
[nσ2 + nµ2 − n
(σ2
n+ µ2
)]=
1
n − 1
[(n − 1)σ2
]= σ2
MATH 450 Chapter 7: Point Estimation
Sample variance as an unbiased estimator
Theorem
The sample variance
S2 =1
n − 1
[(∑X 2i
)− 1
n
(∑Xi
)2]is an unbiased estimator of the population variance σ2.
MATH 450 Chapter 7: Point Estimation
Minimum variance unbiased estimator (MVUE)
Definition
Among all estimators of θ that are unbiased, choose the one thathas minimum variance. The resulting θ̂ is called the minimumvariance unbiased estimator (MVUE) of θ.
Recall:
Mean squared error = variance of estimator + (bias)2
unbiased estimator ⇒ bias =0
⇒ MVUE has minimum mean squared error among unbiasedestimators
MATH 450 Chapter 7: Point Estimation
Example 7.1 and 7.4
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
What is a good estimate of p?
MATH 450 Chapter 7: Point Estimation
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Denote by Xi the result of test i th, where Xi = 1 when thetest success and Xi = 0 if not, then
Each Xi is distributed by
x 0 1p(x) 1-p p
E [X ] =?Moreover,
Y =n∑
i=1
Xi
E [Y ] =?
MATH 450 Chapter 7: Point Estimation
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Let
p̂ =Y
n
the E [p̂] = p, i.e., p̂ is an unbiased estimator
MATH 450 Chapter 7: Point Estimation
Sample proportion
A test is done with probability of success p
n independent tests are done, denote by Y the number ofsuccesses
Let
p̂ =Y
n
the E [p̂] = p, i.e., p̂ is an unbiased estimator
Crazy idea: How about using
p̃ =Y + 2
n + 4
What is the bias of p̃?
MATH 450 Chapter 7: Point Estimation
Example 7.1 and 7.4
MATH 450 Chapter 7: Point Estimation
top related