econ 4160, 2017. lecture 3 - universitetet i oslomodel mis-speci–cationconsequences, tests and...

Model mis-specification Consequences, tests and responses Empirical models Congruence and encompassing

ECON 4160, 2017. Lecture 3Empirical models and mis-specification testing

Ragnar Nymoen

University of Oslo

5 September 2017

1 / 21


References to Lecture 3

I HN Ch. 9, 11.1-11.2, 11.4-11.5.I Computer Class Material: Note about standardmis-specification tests, section 1 and 2 overlap with Ch. 9 andCh 11.1-11.2

I (BN2011: kap 8)

2 / 21


Theoretical model specification I

I So far, we have almost exclusively been reviewing statisticaland econometric theory.

I Given the assumptions of the statistical model, what are thestatistical properties of:

I the dependent variables (Y1, Y2, ...,Yn ), andI the estimators (for example β2, and σ2),I and test-statistics (for example t-values).

that we use to make statistical inference?I This theory is the basis for applied econometric work.I But what about testing the assumptions of the statisticalmodel? Is there any point in doing that ?

3 / 21


Controlled experiment I

I Let’s think of a research situation where the assumptions ofthe statistical model are “obviously” correct, and no testingis necessary.

I A classical laboratory experiment perhaps? We illustrate thatsituation by:

Yiresult

= g(Xi )input

+ vishock

(1)

where the interpretation is that the “input” is controlled bythe researcher, and that the “shock”has stochastic propertiesthat are known trough the experimental design.

I Typically, vi ∼ IID(0, σ2), and maybe also N(0, σ2).

4 / 21


Controlled experiment III In this case, the experimental researcher is in

I control of the Data Generating Process (DGP).I She knows that (Y1, Y2, ...,Yn ) is an Identical andIndependently distributed sequence of random variables.

I And she knows which inference theory to use:

I When g(Xi ) is a linear in parameters, it is the MaximumLikelihood motivated OLS theory that we have reviewed.

I We have hinted that it is not much more diffi cult when g(Xi )is a non-linear in parameters, although we then use NLS, andnumerical optimization.

I We can sum up the controlled lab-situation by saying that thestatistical model is the same as the true DataGenerating Process (DGP) of the variables (Xi ,Yi ).

5 / 21


Relevance of controlled experiment situation foreconometrics I

I Historically, genuine experimental data has been scarce, but isincreasing.

I A lot of modern micro econometric research use data fromnatural experiments, pseudo experiments and pseudo-naturalexperiments.

I But even there, and definitively for standard cross-sectiondata, big-data etc, it is impossible to maintain the positionthat the statistical model is the same as the DGP.

I Think about the purpose of the data collection for example:Usually, it was not the research question that you have inmind!

6 / 21


Relevance of controlled experiment situation foreconometrics II

I Hence the need for testing the assumptions of the model, inorder to verify as far as possible that the inference is reliable.

I The alternative is to invoke the Axiom of correct specification(asserting that the model is the DGP, even though no proof ofthat has been given.)

7 / 21


The situation when the model is NOT the DGP I

I We come closer to the reality of econometric modelling byreplacing (1) by:

Yiobserved

= f (Xi )explained

+ ε iremainder

(2)

I Our explanation is given by the function f (Xi ), in theregression case it is the conditional expectation function.

I For the non-experimental Yi , all variation in Yi that we do notaccount for with f (Xi ), must therefore “end up” in theremainder ε i .

8 / 21


The situation when the model is NOT the DGP III Unlike (1), where vi represents free and independent variationto Yi , ε i in (2) is an implied variable which gets its propertiesfrom the DGP and the explanation, in effect from the modelf (Xi ). Hence in econometrics, we should write:

ε i = Yi − f (Xi ) (3)

to describe that whatever we do on the right hand side of (3)by way of changing the specification of f (Xi ) or by changingthe measurement of Yi , the left-hand side is derived as aresult.

I Hence, we need to test the assumption aboutheteroskedasticity for example.

I Luckily, we can do that by using the observablesε i (i = 1, 2, ..., n).

9 / 21


Consequences of residual mis-specification I

I In models for cross-section data the main damage of

mis-specification is that the OLS estimated√Var(β2), is

incorrect.I ⇒ The standard reported p-values for t-values will be wrong(incorrect critical values in small samples)

I For example: Maybe that Type-I error probability is e.g., 15 %)even if p-value is 0.05

I Can be due to heteroskedasticity or non-normality.I It may be that inference is still valid in large samples: If forexample IID holds, even though the distribution is non-normal.

I But as HN warns, there can may many causes ofnon-normality: As omitted variables or wrong functional-form.

10 / 21


Omitted variables bias I

DIY 3.1

Consider the linear in parameters system of X ,Y and Z . Assumethat you have a sample of n mutually independent triplets(Xi ,Yi ,Zi ). Give an expression for the relationship between theplim of the OLS estimated total derivative of Y with respect to X ,and the true partial derivative of Y with respect to X .

11 / 21


Test for absence from normality IThis test is based on the two moments κ23 = ∑ε3i /σ3 (skewness)and κ24 = ∑ ε4i /σ4 − 3 (kurtosis) where ε i denote a residual fromthe estimated model.Skewness refers to how symmetric the residuals are around zero.Kurtosis refers to the “peakedness”of the distribution. For anormal distribution the kurtosis value is 3. These two moments areused to construct the test statistics

χ2skews = nκ236 χ2kurt = n

κ2424 and, jointly χ2norm = χ2skew + χ2kurt

with degrees of freedom 1, 1 and 2 under the null hypothesis ofnormality of ε i .In PcGive χ2norm is found as Normality test: Chi^2(2) belowthe estimation results in the Results window.Same as the Jarque-Bera-test in other programs.

12 / 21


Heteroskedasticity tests I

These tests are based on auxiliary regressions. A simple example:

ε2i = a0 + a1Xi + a2X2i , i = 1, 2, ...., n, (4)

where the ε i’s are the OLS residuals from the model under testing.Under the null-hypothesis of homoskedasticity:

H0: a1 = a2 = 0

which can be tested by the usual F-test on (4).When there are more than one regressor, products of regressorscan also be tested for. The program reports them as Heterotestand Hetero-X test.

13 / 21


RESET test I

The RESET test in the last test in the PcGive test battery. It isbased on the auxiliary regression

Yi = a0 + a1Xi + a2Y 2i + a3Y3i + vi , i = 1, 2, ..., n, (5)

where Yi denotes the fitted values.RESET23 test indicates that there is both a squared and a cubicterm in (5) so that the joint null hypothesis is: a2 = a3 = 0. Ifyou access the Model-Test menu, you also get the RESET testthat only includes the squares Y 2i .(A drawback with the RESET-test is that it is not invariant tore-parameterizations of dynamic regression models. )

14 / 21


Responses to residual mis-specification IBecause a test does not only have power in one direction, asignificant mis-specification does not usually imply a precisealternative to the model we have estimated.So what do we do? Two schools:

I Keep the model specification, and use robust√Var(β2); or

change estimation method from OLS to GLS (βGLS

mentioned in Lecture 2).I Can easily become a protective belt around a poor model.

I Revise the model specification to bring properties of residualsε i in accordance to the statistical assumptions, i.e., abouthomoskedasticity.

I “Re-make, Re-model”

15 / 21


Empirical models II Ch 11 in HN: “Empirical models and modelling”I The chapter discusses the implications of Data GeneratingProcess (DGP) is in practice unknown to us aseconometricians.

I First of all: Estimated econometric models are empiricalmodels which have properties that depend on the observeddata.

I We made this point above: For example: If the variance ofthe Y is clearly heteroskedastic, the model residuals willbecome manifestly heteroskedastic unless we account for thatvariation in variance in the model. If we fail in our empiricalmodelling, estimation and testing based on the assumptions ofthe IID regression model may not be reliable. Henceimportance of mis-specification testing.

16 / 21


Empirical models II

I Ch 11.3 is an interesting discussion of four differentinterpretations of linear single equation models:

1. Regression, meaning conditional expectations.Although the conditional expectation function exists underweak assumptions about the joint pdf, it can only be derivedas a linear function in a few special cases.Hence the specification of the functional form is an importantmodelling task, as stressed above.

2. Linear least squares approximation to a general functional form3. Contingent planA plan which is implemented by an economic agent afterobserving the outcome of the conditioning variable.Contingent plan models can therefore be estimated at leastconsistently by OLS

17 / 21


Empirical models III

4. Behavioural model (could have used “expectations model”)In this case, the agents’plan depends on expectations, X ei ,about the variable Xi . OLS estimation can give biasedestimates to the parameters of this type of model. It dependson how expectations are formed.Under the assumption of Rational Expectations model, OLSgives biased estimated. We therefore return to this point underthe heading Lucas-critique of OLS estimated time seriesmodels.

18 / 21


Congruence and Encompassing (ch 11.4 and 11.5) I

I From the premise that the true DGP is complex and unknownto us, at least two implications follow

I It is non-trivial to match the theoretical framework to theobservations. An empirical model that achieves that aim iscalled a congruent model. Mis-specification testing isnecessary to test for congruency.

I There can exist two or more competing models of the samevariable.

I Encompassing has been developed as a concept and researchideal to tackle that kind of situation.

I Encompassing means literary: “putting a fence around”.

19 / 21


Congruence and Encompassing (ch 11.4 and 11.5) II

I In econometrics, this entails that if there is an existing(incumbent) empirical model A of the variable Y and youbuild a new model B, then your model MB should explain theresults and properties of MA. (Can write this as MB E MA.)

I Parsimonious encompassing (explaining more by less) givesthe most value-added to you model.

I There is a handful of formal tests of encompassing. Thesimplest is to form the union model of MA and MB and thenuse the LR test (in Chi-square or F-test form ) to test the twosets of restrictions.

20 / 21


Congruence and Encompassing (ch 11.4 and 11.5) III

I Simple regression example:

MA : Yi = β1 + β2Xi + εAi

MB : Yi = γ1 + γ2Zi + εBi

then the nesting-model (M0 ) is simply

M0 : Yi = λ1 + λ2Xi + λ3Zi + ε0i

and the question about encompassing becomes a question ofvariable selection. Return to this in Lecture 7.

21 / 21

econ 4160, 2017. lecture 3 - universitetet i oslomodel mis-speci–cationconsequences, tests and...

Documents