finitesample.pdf
TRANSCRIPT
-
Finite Sample Properties of Finite Sample Properties of S2
Finite-Sample Properties of OLS in the ClassicalLinear Model
Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009
January 21, 2009
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Classical linear model:
1 Linearity: Y = X + u.2 Strict exogeneity: E(u|X) = 03 No Multicollinearity: (X) = K.4 No heteroskedasticity/ serial correlation: V (u|X) = 2X.
OLS Estimator: = (X X)1X YEstimator of 2: S2 =
2i=1 e
2i /(nK).
Finite sample properties: properties of and S2 that can beverified for any fixed sample size n.
The goal is to derive some basic properties that hold under theclassical linear model.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Finite Sample Properties of
Properties of
1 Unbiasedness: E(|X) = .2 Variance expression: V (|X) = 2(X X)1.3 Gauss/Markov Theorem: is the best linear unbiased
estimator.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Unbiasedness: E(|X) =
= (X X)1X Y= (X X)1X (X + u)= + (X X)1X u
E(|X) = + E[(X X)1X u|X
]= + (X X)1X E [u|X]= (Since E(u|X) = 0)
How does heteroskedasticity affect unbiasedness?Which assumptions do we use and which ones we dont?
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Can we prove unconditional unbiasedness: E() = ?
Using the LIE:
E() = E[E(|X)
]= E() =
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Variance expression: V (|X) = 2(X X)1.
V (|X) = V ( |X) ( is not-random)= V
[(X X)1X u|X
](from previous proof...)
= E[(X X)1X uuX(X X)1 |X
]= (X X)1X E(uu|X)X(X X)1
= (X X)1X 2InX(X X)1 (by Assumption 4)= 2(X X)1
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Can we derive an expression for the unconditional variance, V ().?Sure, but we need an extra result
Result: V () = E[V (|X)] + V[E(|X)
]By unbiasedness, E(|X) = , so V () = 0. Also, by our previousresult V (|X) = 2(X X)1. Replacing above:
V () = E[2(X X)1
]= 2E
[(X X)1
]
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Gauss/Markow Theorem: is the best linear unbieased estimator.
Formally: For the classical linear model, for any linear unbiasedestimator ,
V (|X) V (|X) 0
that is, V (|X) V (|X) is a positive semidefinite matrix.
Before attacking the proof: better stands for smaller variance.So the GMT says that among all linear and unbiased estimators of for the classical linear model, is the best
It is a rather restrictive notion.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Proof:
linear: there is AKn that depends on X, with rank K, suchthat = AY .Under the classical linear model
E(|X) = E(AY |X) = E (A(X + u)|X) = AX (1)
unbiased:E(|X) = (2)
linear and unbiased: (1) and (2) hold simultaneouly. Thisrequires AX = I.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Trivially, = + + , with .
Note that: V (|X) = V (|X) + V (|X) iff Cov(, |X) = 0.
So if we prove Cov(, |X) = 0, we have the result. Why?
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
First note that trivially E(|X) = 0 (Why?)
Hence: Cov(, | X) = E[( ) | X]
Note that
= AY (X X)1X Y= (A (X X)1X )Y= (A (X X)1X )(X + u)= (A (X X)1X )u (since AX = I)
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Now replace to get:
Cov(, | X) = E[( ) | X]= E[(X X)1X )uu(A (X X)1X ) | X]= 2[(X X)1X (A X(X X)1)]= 2[(X X)1X A (X X)1X X(X X)1)]= 0
where we used V (u|X) = E(uu|X) = 2In, and, again, AX = I.So, by our previous argument we get:
V ( | X) V ( | X) = V ( | X)
which is by construction positive semidefinite.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Can we obtain an unconditional version of the Gauss-MarkowTheorem? Yes!
Ill leave it as an exercise. See Problem 4b) (pp.32) in Hayashistext.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Finite Sample Properties of S2
Remember that we proposed:
S2 =e2i
nK=
ee
nK
as an estimator for 2.
Result: S2 is unbiased (E(S2|X) = 2)
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Trace: Let A be a square mm matrix. Its trace of A is the sumof all its principal diagonal elements: tr(A) =
mi=1Aii.
Simple properties:
If A is a scalar, trivially tr(A) = Atr(AB) = tr(BA)tr(AB) = tr(A) + tr(B)
The M matrix: M In X(X X)1X
Some properties (check as homework)
M = M (symmetric), M = MM (idempotent)tr(M) = nKe = Mu.
e = Y X = Y X(XX)1XY = (I X(XX)1X)Y = MY = MX +Mu. Nownote that MX = (I X(XX)1X)X = X X(XX)1X)X = 0.
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
-
Finite Sample Properties of Finite Sample Properties of S2
Proof:
E(S2 | X) = E(ee | X)
nK=
E(uM Mu | X)nK
=E(uMu | X)
nK
=E(tr(uMu) | X)
nK
=E(tr(uuM) | X)
nK
=tr(E(uuM | X))
nK
=tr(2InM)nK
=2tr(M)nK
= 2
Walter Sosa-Escudero Finite-Sample Properties of OLS in the Classical Linear Model
Finite Sample Properties of Finite Sample Properties of S2