theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2....
TRANSCRIPT
![Page 1: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/1.jpg)
Estimation
Econometría. ADE.
![Page 2: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/2.jpg)
Estimation
• We assume we have a sample of size T of:– The dependent variable (y)– The explanatory variables (x1,x2, x3, …, xk), with x1being an intercept (a column vector of 1s)
• We also have a vector of unknown parameters 1, 2, …, k)’
• We have a perturbation vector (u) of dimension T• A set of hypothesis about the relationship between X, and u.
![Page 3: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/3.jpg)
Estimation
In these circumstances, we can link the dependent and the explanatory variables as follows:
Yt = 1 + 2 x2t + … + k xkt + ut
Matrix form: Y = X + u Compact form: Yt = xt’ + ut
Our key problem is to provide value to the components of the vector
![Page 4: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/4.jpg)
Estimation
To assign values to the parameters, we have some options.
‐We can invent them.
‐We can phone Rappel and ask him for the more appropriate values according to his “view”.
‐We can use a random procedure.
‐We can estimate them.
![Page 5: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/5.jpg)
Estimation
Estimation theory is a branch of statistics that deals with providing values to a set of
parameters based on measured/empirical data that has a random component.
An estimator attempts to approximate the unknown parameters using the measurements.
![Page 6: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/6.jpg)
Estimation
Estimation methods
• Maximum likelihood
• Ordinary Least Squares• Non‐linear methods
• GMM
Which one will we use along the course?
![Page 7: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/7.jpg)
Estimation
• Do you know this John Ford’s film?
• There is a very famous scene where cowboys escape from redskins by crossing the Rio Grande river. (28:05)
• Do they use a least squares principle?
![Page 8: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/8.jpg)
Estimation
We have the following the relationship:
Y = X + uThus, we can express it as follows:
ˆ ˆˆ ˆˆ
ˆ
Y X u Y u
Y is theestimated part of Yu is the residual part
![Page 9: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/9.jpg)
![Page 10: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/10.jpg)
Estimation
The OLS estimator
ˆ arg minb
y Xb
![Page 11: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/11.jpg)
Estimation
• Then,
• The projection matrix is: PX = X(X’X)‐1X’
• The residual matrix is: M = I ‐ PX = I ‐ X(X’X)‐1X’
• PX and M are orthogonal
• PX is symmetric (PX= P’X )
• PX is idempotent (PX PX= PX )
![Page 12: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/12.jpg)
Estimation
• Consequently, we have that:
1
1
ˆˆ ' '
ˆ ' '
X
X
Y P Y X X X X Y Xand
u M Y I P Y I X X X X Y
![Page 13: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/13.jpg)
Estimation
• By simply comparing the equation, we obtain the expression of the ordinary least squares (OLS) estimator:
1ˆ ' 'X X X Y
![Page 14: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/14.jpg)
Estimation
We can alternatively obtain this same result by simply applying the least squares principle:
• We need to get the combination of values of the estimation vector that minimizes the sum of the squared residuals.
• Analytically, this implies to solve this:
ˆ ˆˆ ˆ min ( ) '( )minu u Y X Y X
![Page 15: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/15.jpg)
Estimation
1
ˆ ˆˆ ˆ min ( ) '( )ˆ ˆ ˆ0 2 ' 2 'ˆ
ˆ' 'ˆ ' '
minu u Y X Y Xu u X y X X
X y X X
X X X Y
![Page 16: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/16.jpg)
Properties of the LS Estimator
We have obtained the LS estimator. But, is good enough this estimator?
Properties
- Unbiased
- BLUE
- Consistent
- Efficient
![Page 17: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/17.jpg)
Properties of the LS EstimatorIt is advisable to express the LS estimator as follows:
1 1
1 1
1
ˆ ' ' ' '
' ' ' '
' '
X X X y X X X X u
X X X X X X X u
X X X u
![Page 18: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/18.jpg)
Properties of the LS Estimator
Thus, we have that:
The OLS estimator vector is unbiased
1
1
ˆ( ) ( ) ' '
' '
E E E X X X u
X X X E u
![Page 19: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/19.jpg)
Properties of the LS Estimator
The variance of the estimator vector is
12
1121T
2111
1111
11
X'X
X'XX'XX'XX'XXI'XX'XX'XX'uuE'XX'X
X'XX'uu'XX'XE'u'XX'Xu'XX'XE
'u'XX'Xu'XX'XE'ˆEˆˆEˆEˆVar
![Page 20: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/20.jpg)
Properties of the LS Estimator
The OLS estimator is BLUE
• Best Linear Unbiased Estimator
• This implies that it has the lowest variance, as compared to other unbiased, linear estimators.
• Gauss-Markov theorem helps us to prove it.
![Page 21: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/21.jpg)
Properties of the LS Estimator
The LS estimator is consistent if we can prove that:
Or equivalently
ˆlim Pr 1T
ˆlimp
![Page 22: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/22.jpg)
Properties of the LS Estimator
The use of plim’s can help us to prove consistency, given that:
1
1
1
ˆlim lim '
lim lim '
'lim lim
0X
p p X X X u
p p X X X u
X X X up pT T
![Page 23: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/23.jpg)
Properties of the LS Estimator
We can employ an alternative method to prove consistency. An estimator is consistent if:
• It is asymptotically unbiased
• Its variance goes to zero when the sample grows to infinity.
ˆlimT
E
ˆlim ( ) 0T
Var
![Page 24: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/24.jpg)
Properties of the LS Estimator
The first condition is held by simply considering that:
ˆlim limT T
E
![Page 25: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/25.jpg)
Properties of the LS Estimator
The second condition also holds.
1
1 12 2
ˆlim ( ) lim '
' 'lim lim lim
0 0
T T
T T T
X
Var X X
X X X XT T T T
![Page 26: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/26.jpg)
Properties of the LS Estimator
We can also follow a very intuitive approach. If an estimator is consistent, we have that:
• The estimator asymptotically collapses towards a value, which is the true value of the parameter.
Thus, the differences between estimator and true value of the parameter vanishes.
• It is also true that the higher the sample size, the more information available to estimate the model.
![Page 27: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/27.jpg)
Properties of the LS Estimator
Let us consider that the observations of a dependent variable are generated by the following model:
T={10, 50, 100, 500, 1000}
The perturbation is generated by a N(0,1) distribution
2 1,2,...,t ty u t T
![Page 28: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/28.jpg)
Properties of the LS Estimator
Then, we estimate by OLS the following model
The results we have obtained are reported in next Figures.
1,2,...,t ty u t T
![Page 29: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/29.jpg)
![Page 30: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/30.jpg)
![Page 31: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/31.jpg)
![Page 32: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/32.jpg)
![Page 33: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/33.jpg)
![Page 34: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/34.jpg)
Properties of the OLS estimator. Efficiency
An estimator is efficient if it has the lowest variance, compare to other unbiased estimators.
ˆ( ) ( )Var Var
![Page 35: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/35.jpg)
Properties of the OLS estimator. Efficiency
To prove it:
• We should impose that the perturbation follows a normal distribution
• We should then obtain the Cramer-Rao bound.
2
1 1
;Var
IE X
![Page 36: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/36.jpg)
Properties of the OLS estimator. Efficiency
• We should compare the Cramer-Rao bound with the variance of the OLS estimator.
• For the OLS estimator vector, the Cramer-Raobound is 2 (X’X)-1.
• It coincides with the covariance matrix of the OLS estimator vector.
• Consequently, the OLS estimator is efficient.
![Page 37: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/37.jpg)
OLS estimator of the variance of the perturbation.
• If we assume that the perturbation vector follows a N(0, 2 I) distribution, we have that the vector y follows a N(X,2 I) distribution.
• We already know how the vector can be estimated
• We need now to get an estimator of the variance of the perturbation (2)
• We can use the OLS approach
![Page 38: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/38.jpg)
OLS estimator of the variance of the perturbation.
The OLS estimator can be defined as follows:
With being the sum of the squared residuals (SSR)
2ˆ ˆˆ ˆˆ
y X y Xu uT k T k
ˆ ˆu u
![Page 39: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/39.jpg)
OLS estimator of the variance of the perturbation.
This estimator shows the following properties:
• It is unbiased
• It is NOT linear
• It is consistent
• It is NOT efficient
![Page 40: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/40.jpg)
Maximum Likelihood estimator
The ML estimation selects the set of values of the model parameters that maximizes the
likelihood function.
• Assuming all the previous hypothesis, the joint density function of (y1, y2, …, yT) can be expressed as follows:
2 2 2 1/ ; , 2 exp2
T
f y X y X y X
![Page 41: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/41.jpg)
Maximum Likelihood estimator
Then, the likelihood function can be stated as follows:
Which is the difference with respect the joint density function?
2 2 2 1, ; / 2 exp2
T
y X y X y X
![Page 42: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/42.jpg)
Maximum Likelihood estimator
To obtain the ML estimators, we should first the log-likelihood function
Then, we should obtain the values of the parameters that maximize this function
2 2 1, ln 2 ln2 2 2T T y X y X
2
0
0
![Page 43: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/43.jpg)
Maximum Likelihood estimator
The solutions to this optimization problem are:
The ML estimator and the OLS estimator coincide.
Both have the same properties.
1ˆ ( )X X X Y
![Page 44: theme 2. estimation - unizar.espersonal.unizar.es/amontane/ade_econometrics/theme 2. estimation.… · Estimation Estimation theory is a branch of statistics that deals with providing](https://reader033.vdocuments.mx/reader033/viewer/2022050200/5f53f155e14b3e47f41c6678/html5/thumbnails/44.jpg)
Maximum Likelihood estimator
The ML estimator of the variance of the perturbation is:
It is consistent
It is not unbiased, linear, efficient
It has asymptotical good properties (it is a MLE)
2ˆ ˆy X y X
T