deciding between alternative approaches in macroeconomics...much past applied econometrics research...

Deciding between Alternative Approaches

Deciding between Alternative Approaches inMacroeconomics

David F. Hendry

Institute for New Economic Thinking at theOxford Martin School, University of Oxford

Research jointly with Jennifer Castle, Jurgen Doornik,Søren Johansen, Grayham Mizon and Bent Nielsen

ESRC/OMS Conference, 2012

David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 1 / 49

Introduction

[1] All macroeconomic theories are incomplete, incorrect andchangeable;

[2] all macroeconomic time-series data are aggregated,inaccurate and rarely match theoretical concepts;

[3] all empirical macro-econometric models are non-constant, and[4] mis-specified in numerous ways;

[5] economic policy often has unexpected effects different fromprior analyses;

[6] macroeconomic forecasts regularly go awry:

so how to decide between alternative approaches?


Credibility versus verisimilitude

Main justification of empirical macro-econometric evidence isconformity with conventionally-accepted economic theory:‘internal credibility’ as against verisimilitude.

Partly justified by manifest inadequacy of short, dependent, andheterogeneous time-series data,often subject to extensive revision:if data are unreliable, better to trust the theory.

But theories have evolved greatly, and most previous analysesabandoned: almost self-contradictory to justify an empiricalmodel by an invalid theory that will soon be altered.

Why is incorrect and mutable theory more reliable than dataevidence?


Three more possible explanations

Worries concern prevalence of non-stationarity, endogeneity, potentiallack of identification, and collinearity,belief that ‘data mining’ can produce almost any desired result—but so can theory choice by matching non-existent ‘stylizedfacts’, that are neither constant nor facts.

Part is a mistaken conflation of economic-theory models of humanbehavior with the data generation process (DGP):huge gap between abstract theory and non-stationary evidencefinessed by asserting that the model is the mechanism–‘let’s take the model seriously’–no, let’s leave the talk.

Final part is false belief that data-based model selection is asubterfuge of scoundrels–rather than the key to understanding the complexities ofmacro-economies.


Reality includes unanticipated shifts

original distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

original distribution



original distribution fat-tailed distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

original distribution fat-tailed distribution



original distribution fat-tailed distribution shift in distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

original distribution fat-tailed distribution shift in distribution


Both credibility & verisimilitude matter

Cannot discuss all pertinent issues here, focus on constellation of:[1] model selection while retaining (not imposing) theory;[2] tackling multiple location shifts; and[3] testing the validity of policy models.

Forecasting addressed in Castle, Fawcett, and Hendry (2011);and nowcasting in Castle, Fawcett, and Hendry (2009).


How to proceed?

Nest ‘theory-driven’ and ‘data-driven’ approaches:retain theory insights unaffected by selection, but select overrival candidate variables, lags, functional forms, etc..

Multiple breaks also rarely included in theoriesbut can be accommodated empirically.

Multi-path search algorithm with tight critical valuescontrols false retention of irrelevant variables at low levelsyet retains all theory-based variables, irrespective of their significance.

If theory correct and complete, distributions of parameterestimators identical to directly fitting to data: if incorrect,discover better empirical model.

Can have more candidate variables, N, than sample size T .

Yet selection costless if not needed and beneficial otherwise,precisely the opposite of current beliefs.


Discovery in economics

Discoveries in economics mainly from theory.But all economic theories are:(a) incomplete; (b) incorrect; and (c) mutable.(a) Need strong ceteris paribus assumptions:inappropriate in a non-stationary, evolving world.(b) Consider an economic analysis which suggests:

y = f (x) (1)where the k variables y depend on r ‘explanatory’ variables x withm > r instruments z.Form of f (·) in (1) depends on:utility or loss functions of agents,constraints they face, & information they possess.Analyses arbitrarily assume: forms for f (·), that f (·) is constant, thatonly x matters, & that the zs are ‘exogenous’.Yet must aggregate across heterogeneous individualswhose endowments shift over time, often abruptly.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 8 / 49

Theory evolves

(c) Economic analyses have not only changed our understanding, theyhave changed the world: from the ‘invisible hand’ in Adam Smith’sTheory of Moral Sentiments (1759, p.350) onwards, theory hasprogressed dramatically–key insights into option pricing, auctions and contracts,principal-agent and game theories, trust and moral hazard,asymmetric information, institutions:major impacts on market functioning, industrial,and even political, organization.

But imagine imposing 1900’s economic theory in empiricalresearch today.

Much past applied econometrics research is forgotten:discard the economic theory that it ‘quantified’ andyou discard the associated empirical evidence.

Hence fads & fashions, ‘cycles’ and ‘schools’ in economics.


Automatic empirical model discovery

Need to tackle all complications jointly.Re-frame empirical modeling as discovery processcombined with theory evaluation.

Starting from T observations on N > r variables w,aim to find β∗ for s lagged functions g(w∗t) . . . g(w∗t−s) of a subset ofk variables w∗, jointly with 1t=ti–indicators for breaks, outliers etc.

Embeds initial economic analysis y = f(x),but in a much more general initial model.

Globally, learning must be simple to general;but locally need not be in observational disciplines.

Approach explained in Castle, Doornik, and Hendry (2011) andHendry and Johansen (2012).


Extensions outside standard information

Extensions determine how well DGP is approximated

Create three extensions automatically:(i) lag formulation to implement sequential factorization;(ii) functional form transformations for non-linearity;(iii) impulse-indicator saturation (IIS) for parameter non-constancyand data contamination.

(i) Create s lags wt . . . wt−s to formulate general linear model:

yt = β0 +

s∑i=1

λiyt−i +

r∑i=1

s∑j=0

βi,jwi,t−j + εt (2)

Focus on single equations, but systems can be handled.


Automatic non-linear extensions

(ii) Castle and Hendry (2011) propose cubics and exponentials ofprincipal components ut of the wt.

Let wt ∼ Dr [µ, Ω], where Ω = HΛH′ with H′H = Ir.

Empirically: Ω = T−1∑Tt=1(wt − w)(wt − w)′ = HΛH′

so that ut = H′(wt − w) leading to ut ∼app

Dr [0, I].

Presently implemented general cubics with exponential functions.u2i,t; u

3i,t; ui,te

−|ui,t|.

When Ω is non-diagonal, each ui,t is a linear combination of everywi,t, so u2

i,t involves squares and cross-products of every wi,t etc.

Low dimensional, with no collinearity between elements of ut yetincludes most important sources of departure from linearity, e.g.asymmetry.


Non-linear functions

zi2×zi

-2.5 0.0 2.5

2.5

5.0

7.5

10.0

12.5

15.0

17.5zi2×zi zi

3×zi

-2.5 0.0 2.5

-40

-20

0

20

40

60

zi3×zi zie

−|zi |×zi

-2.5 0.0 2.5

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

zie−|zi |×zi


Detecting multiple breaks

Numbers, timings and magnitudes of breaks in models usuallyunknown: obviously so for unknowingly omitted variables.

‘Portmanteau’ approach required to detect location shifts anywhere insample, while also selecting over many candidate variables.

To check the null of no outliers or location shifts in a model,impulse-indicator saturation (IIS)creates complete set of indicator variables:

1j=t= 1 when j = t and 0 otherwise for j = 1, . . . , T :

need to add T impulse indicators to set of candidate variables when Tobservations.Feasible ‘split-sample’ IIS algorithm in Hendry, Johansen, andSantos (2008), generalized by Johansen and Nielsen (2009), and thenimplemented in Autometrics by Doornik (2009).Castle, Doornik, and Hendry (2012) show power under alternatives.


Dynamic generalizations

Johansen and Nielsen (2009) extend IIS to both stationary andunit-root autoregressions.

When distribution is symmetric, adding T impulse-indicators to aregression with n variables, coefficient β (not selected) and secondmoment Σ:

T 1/2(β − β)D→ Nn

[0,σ2

εΣ−1Ωβ

]Same rate of convergence, to normal, with correct centering, despiteT extra indicators.Efficiency of IIS estimator β with respect to OLS β measured by Ωβdepends on cα and distribution form.

Must lose efficiency under null; small loss αT of 1 observation atT = 100 if α = 1/T = 0.01 so 99% efficient.

Potential for major gain under alternatives of breaks and/oroutliers, yet can be done jointly with all other selections.


Null ‘split-sample’ search in IIS

0 50 100

0.5

1.0Dummies included initially

Blo

ck 1

Selected model: actual and fitted

0 50 100

10.0

12.5 actualfitted

0 50 100

0.5

1.0Dummies retained

0 50 100

0.5

1.0

Blo

ck 2

0 50 100

10.0

12.5

0 50 100

0.5

1.0

0 50 100

0.5

1.0

Fina

l

0 50 100

10.0

12.5

0 50 100

0.5

1.0


‘Split-sample’ search in IIS

0 50 100

Dummies included initially

actual fitted

0 50 100

0

5

10

15Selected model: actual and fitted

Blo

ck 1

actual fitted

0 50 100

Dummies retained

0 50 100 0 50 100

0

5

10

15

Blo

ck 2

0 50 100

0 50 100 0 50 100

0

5

10

15

Fina

l

0 50 100


Specification of GUM

Formulation decisions of which r variables wt;their lag lengths (s);cubics + exponentials in ut, after orthogonalizing wt;location shifts (any number, anywhere).Leads to general unrestricted model (GUM):

yt =

r∑i=1

s∑j=0

βi,jwi,t−j +

r∑i=1

s∑j=0

κi,ju2i,t−j +

r∑i=1

s∑j=0

θi,ju3i,t−j

+

r∑i=1

s∑j=0

γi,jui,te−|ui,t| +

s∑j=1

λjyt−j +

T∑i=1

δi1i=t + εt(3)

K = 4r(s+ 1) + s potential regressors, plus T indicators:bound to have N > T–consider exogeneity later.


Retaining economic theory insights

Approach is not atheoretic.

Theory formulations should be embedded in GUM,can be retained without selection,but does not guarantee they will be significant.

Can also ensure theory-derived signs of long-run relation maintained,if not significantly rejected by the evidence.

But much observed data variability in economics is due tofeatures absent from most economic theories:which empirical models must handle.

Extension of DGP candidates, xt, in GUM allows theory formulationas special case, yet protects against contaminating influences (likeoutliers) absent from theory.

‘Extras’ can be selected at tight significance levels.


Theory exactly correct

Correct n valid conditioning variables, zt, constant parameters β:

yt = β′zt + εt (4)

where εt ∼ IN[0,σ2ε], independently of zt. Then:

β = β+

(T∑t=1

ztz′t

)−1 T∑t=1

ztεt ∼ Nn

β,σ2ε

(T∑t=1

ztz′t

)−1 (5)

Next, zt retained during model selection over second set of kirrelevant candidate variables, wt, with coefficients γ = 0 when(k+ n) << T , so GUM is:

yt = β′zt + γ′wt + νt (6)Orthogonalize zt and wt by:

wt = Γzt + ut (7)Then as γ = 0:

yt = β′zt + γ′wt + νt = β′zt + γ′ut + νt (8)

Coefficient of zt unaltered.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 20 / 49

Distributions of forced estimates

Consequently:(β − β

γ

)=

( ∑Tt=1 ztz

′t

∑Tt=1 ztu

′t∑T

t=1 utz′t

∑Tt=1 utu

′t

)−1( ∑Tt=1 ztνt∑Tt=1 utνt

)

∼ Nn+k

( 00

),σ2ε

(T∑t=1

ztz′t

)−1

0

0

(T∑t=1

utu′t

)−1

(9)

as∑Tt=1 ztu

′t = 0, so distribution of β in (9) identical to that of β in

(5): unaffected by model selection.

Only costs of selection are:(a) chance retentions of some ut from selection, controlled by verytight significance levels (α 6 min[0.01, 1/(N+ T)]; and(b) impact on estimated distribution of β through σ2

ε,offset by bias correction.


Theory only part of explanation

Two distinct forms of under-specification:a] omitting relevant functions or lags of variables in DGP;avoided by sufficiently general initial model:b] omitting relevant variables, ηt, from the DGP;induces less useful DGP–hard to avoid if ηt unknown.

In a], γ 6= 0, as zt and ut orthogonal in (8), coefficient of former isβ + γ′Γ, which is estimated if (4) is simply fitted to the data: but maybe significant with anticipated signs.

In b], when (6) nests DGP, but ηt omitted from DGP,selection can substantively improve the final model:see Castle and Hendry (2012).Win-win situation: theory kept if valid and complete;yet learn when it is not correct–empirical model discovery embedding theory evaluation.

Can automatic model selection still work when N > T?


As many candidate variables as observations

Analytic approach to understanding IIS applies for N = T IID mutuallyorthogonal candidate regressors under the null.

Add first N/2 and select at significance level α = 1/T = 1/N.Record which were significant, and drop all.Now add second block of N/2, again select at significance levelα = 1/N, and record which are significant.Finally, combine recorded variables from the two stages(if any), and select again at significance level α = 1/N.

At both sub-steps, on average αN/2 = 1/2 a variable will be retainedby chance, so on average αN = 1 from the combined stage.

Again 99% efficient under the null at eliminating irrelevantvariables–lose one degree of freedom on average.

If N > T , divide in more sub-blocks; and if relevant variables to beretained, orthogonalize them with respect to the rest.


Evaluating policy models

When breaks occur, estimated models constant only if:(a) all substantive variables are included; and

(b) no internal shifts occur,

both demanding requirements.

Tackle (a) by commencing from large initial information set.

Super exogeneity violated by omitted variables:policy model becomes non-constant when policy is changed.

Investigate in advance when previous policy changes:examine marginal process for in-sample shifts;test for co-breaking with policy model.

For (b), an indicator for location shifts would remove non-constancy,as will IIS when it reflects such an indicator.

Location shifts induce changing error variances, so|t| < cα if no indicators, yet larger with.


Testing super exogeneity

Parameter invariance essential in policy models:else mis-predict under regime shifts.

Automatic test of super exogeneity in Hendry and Santos (2010):IIS in marginal models, retain all significant outcomes andtest their relevance in conditional model

No ex ante knowledge of timing or magnitudes of breaks:need not know DGP of marginal variables

Test has correct size under null of super exogeneityfor a range of sizes of marginal-model saturation tests

Power to detect failures of super exogeneity when location shiftsin marginal models:applies equally to models with expectations like NKPC:see Castle, Doornik, Hendry, and Nymoen (2012).


Non-invariance of NKPCs

‘Hybrid’ NKPC given by:∆pt = γf

>0E [∆pt+1|It] + γb

>0∆pt−1 + π

>0st + ut (10)

∆pt, st are rate of inflation & firms’ real marginal costs, so:∆pt = γf∆pt+1 + γb∆pt−1 + πst + εt, εt ∼ D

[0,σ2

ε

](11)

where:E [∆pt+1 | It] = ∆pt+1 + νt+1 (12)

which appears to suggest that expectations are unbiased as:

E[νt+1 | It] = 0. (13)

Then ∆pt+1 instrumented by k variables zt implicitly postulating:

∆pt = κ′zt + vt (14)Assumes a constant-parameter world, so test by IIS on (14) by addingretained indicators to (11): significance refutes invariance.Also, insignificance of γf inconsistent with forward-looking formulation.


Euro-area NKPC

Euro-area hybrid NKPC with IV estimation, ∆pt+1 and stendogenous, using instruments: five lags of inflation, two lags of st,detrended output and wage inflation;sample T = 102 (1972(2) to 1998(1)):

∆pt = 0.655(0.135)

∆pt+1 + 0.280(0.117)

∆pt−1 + 0.012(0.014)

st + 0.009(0.010)

χ2S(6) = 11.88

(15)Elasticities sum to 0.94 and γf comparable to reported GMMestimates: Galı and Gertler (1999).Forecasting equation for ∆pt uses instrument set for NKPC estimationwith IIS in Autometrics: for α = 0.025, finds 11 indicators.


Findings for Euro-area NKPC with IIS

Adding gapt−1 and the 11 indicators makes NKPC congruent:χ2S(4) = 2.42: no significant tests of residual mis-specification.

Nine of the 11 ‘reduced form’ indicators retained:clear evidence for lack of invariance in feed-forward NKPC.

Coefficient of ∆pt+1 is negative andinsignificantly different from zero.

Coefficient of wage-share is sizeable,serves as an important equilibrating mechanism.

Inflation ‘persistence’ an artifact of mis-specified NKPC model

A failure to model breaks induces spurious significance offeed-forward terms proxying expectations:but a deeper problem lurks.


Problem lies in expectations formation

Write f∆pt as density of ∆pt: then (13) is really Ef∆pt[νt+1 | It] = 0.

Nothing ensures Ef∆pt+1[νt+1 | It] = 0 for unbiased expectations:

Ef∆pt+1[∆pt+1 | It] =

∫∆pt+1f∆pt+1

(∆pt+1 | It) d∆pt+1 (16)

so (16) requires a crystal ball for future f∆pt+1(∆pt+1|It).

Best an agent can do is form a sensible expectation, forecastingf∆pt+1

(·) by f∆pt+1(·). If moments of f∆pt+1

(·) alter, no good rules forf∆pt+1

(·), but f∆pt+1(·) = f∆pt(·) is not a good choice.

Agents cannot know f∆pt+1(·) if no time invariance:

nor how It enters.

Sleight of hand by not subscripting expectations by the relevantdistribution: revealed by accounting for shifts.


Law of iterated expectations fails

When variables, say (xt+1, xt), at different dates are drawn fromsame distribution fx:

Efx[Efx

[xt+1 | xt]] = Efx[xt+1] (17)

But when distributions shift:

Efxt

[Efxt+1

[xt+1 | xt]]6= Efxt+1

[xt+1] (18)

as:fxt+1

(xt+1 | xt) fxt(xt) 6= fxt+1

(xt+1 | xt) fxt+1(xt) (19)

See Hendry and Mizon (2010) for formal derivations.

Invalidates inter-temporal derivations facing unanticipated shifts:or theory requires that no location shifts occur—so is empirically irrelevant.DSGEs are intrinsically non-structural:mathematical basis fails when distributions alter.


Conclusions on empirical modelling

All essential steps feasible once target DGP defined:1. automatically create general model from investigator’s xt:extra variables, longer lags, non-linearity, & shift indicators–ensures congruent GUM andavoids huge costs from under-specified models;2. embed theory-model, orthogonalizing other variables–ensures specification is retained unaltered;3. select most parsimonious congruent encompassing model–ensures undominated representation;4. compute near-unbiased parameter estimates–ensures appropriate quantification; and5. stringently evaluate results, especially super exogeneity–ensures selected model valid, and usable.

Generalizes to N > T with expanding and contracting searches.


Conclusions on unanticipated location shifts

Many risks from location shiftsForecast failure primarily due to location shifts;

systematic mis-forecasting by all equilibrium-correctionmodels: regressions, VARs, DSGEs, EqCMs, GARCH, etc.;

difficult to predict, but can mitigate failure by robust devices;

location shifts invalidate law of iterated expectations;

conditional expectations not unbiased for next period;

‘rational expectations’ systematically biased after shifts;

yet every DGP parameter can shift without noticeable effect;

agents could not quickly learn what had changed;

super exogeneity test can exploit location shifts;

NKPC shows lack of invariance: insignificant feed-forwardterm and little inflation persistence.


References

Castle, J. L., J. A. Doornik, and D. F. Hendry (2011). Evaluating automatic model selection. Journal of Time SeriesEconometrics 3 (1), DOI: 10.2202/1941–1928.1097.

Castle, J. L., J. A. Doornik, and D. F. Hendry (2012). Model selection when there are multiple breaks. Journal ofEconometrics 169, 239–246.

Castle, J. L., J. A. Doornik, D. F. Hendry, and R. Nymoen (2012). Mis-specification testing: Non-invariance ofexpectations models of inflation. Econometric Reviews, forthcoming.

Castle, J. L., N. W. P. Fawcett, and D. F. Hendry (2011). Forecasting Breaks and During Breaks. In M. P. Clements andD. F. Hendry (Eds.), Oxford Handbook of Economic Forecasting, pp. 315–353. Oxford: Oxford University Press.

Castle, J. L. and D. F. Hendry (2011). Automatic selection of non-linear models. In L. Wang, H. Garnier, andT. Jackman (Eds.), System Identification, Environmental Modelling and Control, pp. 229–250. New York: Springer.

Castle, J. L. and D. F. Hendry (2012). Model selection in under-specified equations with breaks. Journal ofEconometrics, forthcoming.

Castle, J. L. and N. Shephard (Eds.) (2009). The Methodology and Practice of Econometrics. Oxford: OUP.Doornik, J. A. (2009). Autometrics. See Castle and Shephard (2009), pp. 88–121.Gabaix, X. (2012). Variable rare disasters: an exactly solved framework for ten puzzles in macro-finance. The

Quarterly Journal of Economics 127, 645–700.Galı, J. and M. Gertler (1999). Inflation dynamics: A structural econometric analysis. Journal of Monetary

Economics 44, 195–222.Hendry, D. F. and S. Johansen (2012). Model discovery and Trygve Haavelmo’s legacy. Econometric Theory ,

forthcoming.Hendry, D. F., S. Johansen, and C. Santos (2008). Automatic selection of indicators in a fully saturated regression.

Computational Statistics 33, 317–335. Erratum, 337–339.Hendry, D. F. and H.-M. Krolzig (2005). The properties of automatic Gets modelling. Economic Journal 115, C32–C61.Hendry, D. F. and G. E. Mizon (2010). On the mathematical basis of inter-temporal optimization. Discussion paper

497, Economics Department, Oxford.Hendry, D. F. and C. Santos (2010). An automatic test of super exogeneity. In M. W. Watson, T. Bollerslev, and

J. Russell (Eds.), Volatility and Time Series Econometrics, pp. 164–193. Oxford: Oxford University Press.Johansen, S. and B. Nielsen (2009). An analysis of the indicator saturation estimator as a robust regression

estimator. See Castle and Shephard (2009), pp. 1–36.


Retracing route

(A) Empirical model discovery(B) Multiple location shifts(C) Evaluating theory models(D) Evaluating policy models(E) Non-invariance of NKPCsConclusion


Identification does not need prior information

When system is:

yt = Ψzt + vt where vt ∼ INm [0, Ωv] (20)

with model:

Byt = Czt + εt where εt ∼ INm [0, Σε] (21)

uniqueness of B, C from solving BΨ = C is intrinsic to DGP.If B, C identified, can be found without prior knowledge as (21) is areduction of (20): see Hendry and Krolzig (2005).Confirmed by over-identified restrictions being testable byparsimonious encompassing.

If B, C not identified, (20) is least restricted identified representation;but all just-identified representations equivalent.

For given B, C over-identified in DGP, (21) is unique.But other B∗, C∗ could be equally over-identified with equal likelihood.


Super exogeneity violated by omissions

To illustrate, consider conditional DGP:

yt = β0 + β′1x1,t + β

′2x2,t + εt (22)

where processes non-constant with:

x2,t = E [x2,t] + Γ (x1,t − E [x1,t]) + vt (23)

E [x1,t] = ψ1,1+ψ1,21t>T1 and E [x2,t] = ψ2,1+ψ2,21t>T2 (24)Not known x2,t is relevant, so DGP is:

yt = β0+β′2 (E [x2,t] − ΓE [x1,t])+

(β′1 + β

′2Γ)

x1,t+β′2vt+εt (25)

Intercept non-constant in (25) if Γ 6= 0.

Assumed model mis-specified by omitting x2,t:

yt = γ0 + γ′1x1,t + et (26)

even if constant in-sample, when x1,t is changed to implement apolicy, outcome will not be as anticipated.


Policy failure

From the DGP (22):∂yt

∂x′1,t

= β1 (27)

so if x1,t is shifted to x1,t +∇x1,t, then yt shifts by β′1∇x1,t.But (25) and (26) entail that shift is (β′1 + β

′2Γ)∇x1,t,

so intercept shift of −β′2Γ∇x1,t occurs in the model.

Policy model non-constant precisely when policy is changed:failure of super exogeneity.

Investigate in advance if any previous policy changes:examine marginal process of x1,t for in-sample shifts.Ascertain occurrence and timing of location shifts in x1,t, and checkwhether they coincide with shifts in conditional model yt|x1,t.


Implementing the test

First stage is IIS in marginal,retaining dummies at significance level α1:

xt = π0 +

s∑j=1

Πjxt−j +

m∑i=1

ρi,α11t=ti + v∗2,t (28)

Second stage addsm retained indicators to conditional:

yt = µ0 + β′xt +

m∑i=1

τi,α21t=ti + εt (29)

Conduct F-test for significance of (τ1,α2 . . . τm,α2) at level α2

Test has power as significant impulse indicators capture outliers andlocation shifts not explained by regressors.


Basis of approach

Data generation process (DGP):joint density of all variables in economy

Impossible to accurately theorize about or model preciselyToo high dimensional and far too non-stationary.

Need to reduce to manageable size in ‘local DGP’ (LDGP):the DGP in space of r variables xt being modelled

Theory of reduction explains derivation of LDGP:joint density Dx(x1 . . . xT |θ).Acts as DGP, but ‘parameter’ θ may be time varying

Knowing LDGP, can generate ‘look alike data’ for xtwhich only deviate from actual data by unpredictable noise

Once xt chosen, cannot do better than know Dx(·)–so the LDGP Dx(·) is the target for model selection:need to relate theory model to that target.


Empirical econometrics

To establish ‘truth’ requires at least these 12 assumptions:1. correct, comprehensive, & immutable economic theory;2. correct, complete choice of all relevant variables & lags;3. validity & relevance of all regressors & instruments;4. precise functional forms for all variables;5. absence of hidden dependencies;6. all expectations formulations correct;7. all parameters identified, constant over time, & invariant;8. exact data measurements on every variable;9. errors are ‘independent’ & homoscedastic;10. error distributions constant over time;11. appropriate estimator at relevant sample sizes;12. valid and non-distortionary method of model selection.

If ‘truth’ is not on offer–what is?


Theory and evidence

Many features of models not derivable from theory.

Almost always must be data-based on available sample:need to discover what matters empirically.

Need empirical evidence on which:[a]: variables are actually relevant (specification),[b]: their lagged responses (dynamic reactions),[c]: functional forms of relationships (non-linearities),[d]: structural breaks & unit roots (non-stationarities),[e]: simultaneity (or exogeneity), expectations, etc.

Theory provides an object for modeling–not the target:(A) embed that object in much more general formulation;(B) search for the simplest acceptable representation;(C) evaluate the findings for congruence and encompassing.

How to accomplish? And what are its properties?


Implications for automatic methods

Seven stages for empirical model discovery and theoryevaluation in econometrics.First, theoretical derivation of x, and putative relationships.Second, automatically create more general model in w embeddingy = f(x).Third, automatic selection over orthogonalized representations,retaining theory.Fourth, end with congruent parsimonious-encompassing model.Fifth, quantify the outcome by unbiasedly estimating resultingmodel.Sixth, evaluate theory directly and any discoveries on new data, newtests and new procedures.Seventh, summarize possibly vast information set inparsimonious but undominated model.


Euro-area NKPC with IIS

Selecting by Autometrics with α = 0.05:

∆pt=−0.298(0.264)

∆pt+1+ 0.115(0.029)

st+ 0.505(0.126)

∆pt−1+ 0.086(0.021)

+ 0.0015(0.0004)

gapt−1

+1.10(0.30)

I73(1),t+ 1.09(0.38)

I73(3),t+ 0.73(0.31)

I73(4),t+ 0.85(0.34)

I74(2),t

+0.80(0.33)

I74(3),t+ 0.98(0.38)

I76(2),t+ 0.57(0.29)

I76(3),t− 0.65(0.28)

I78(4),t+ 0.69(0.29)

I83(1),t

χ2S(6) = 5.06 Far(5, 85) = 1.55 Farch(4, 96) = 1.49

Fhet(14, 80) = 1.41 χ2nd(2) = 1.04

(coefficients of dummies multiplied by 100)Fname denotes an approximate F-test:Far for kth-order serial correlation;Fhet for heteroskedasticity;Freset for functional form;Farch for kth-order ARCH; andχ2

nd(2) for normality.


Problems with forecasting

Consider stationary scalar AR(1) DGP with known exogenouszt ∼ IN[0, 1]:

xt = µ+ρxt−1+γzt+εt where εt ∼ IN[0,σ2

ε

]and |ρ| < 1 (30)

over t = 1, . . . , T where E[zt] = κ.When µ, ρ, and γ known & constant, forecast from xT for known zT+1:

xT+1|T = µ+ ρxT + γzT+1 (31)

is unbiased for xT+1 with smallest possible variance.What parameter shifts cause problems?

Location shifts alone are problematic, namely shifts inθ = (µ+ γκ)/(1 − ρ):change all parameters, forecasting with mis-specified model—no failure;change just ρ using in-sample DGP with zero intercept—massive failure.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 44 / 49

Incorrect specification,µ = κ = 10 and ρ changed twice

~xT+h |T+h−1 xT+h |T+h−1

30 35 40 45 50

92

94

96

98

100

102

104Incorrect specification+2 breaks: µ=10,µ∗ =50, ρ=0.8, ρ∗ =0.4, κ=10


Model incorrectly specified, zT+1 omitted with κ = 10. Forecastsafter breaks in ρ = 0.8 to ρ∗ = 0.4, & µ = 10 to µ∗ = 50 at T = 41then back at T = 46 so:

xT+h = µ∗ + ρ∗xT+h−1 + γzT+h + εT+h (32)Yet no forecast failure when xT+h|T+h−1 = µ+ ρxT+h−1.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 45 / 49

Correct specification,µ = 0 and changed ρ


30 35 40 45 50

20

25

30

35

40

45

50Correct specification+same breaks: µ=0, κ=10


Model correctly specified in-sample,forecasts for same break, µ = 0, and κ = 10 but zt+1 included.Forecast failure is manifest. In-sample correct specification neednot help even with a zero intercept and known future zT+h.


Many parameters shift

xt

30 35 40 45 50

20

30

40

50µ=0; γ=2; κ=5; ρ=0.8; µ=0; γ∗=1.36; κ=5; ρ∗=0.6

a

xt

xt

30 35 40 45 50

20

30

40

50µ=5; γ=1; κ=5; ρ=0.8;µ∗ =2.5; γ∗=0.86; κ=5; ρ∗=0.6

bxt

xt

30 35 40 45 50

20

30

40

50

µ=5.0, γ= 0.25, κ=20, ρ=0.8 µ*=0.5, γ*= 0.4, κ=20, ρ*=0.5

c

xt

xt

30 35 40 45 50

20

30

40

50µ=5; γ=2.5; κ=2; ρ=0.8; µ∗ =−2.95; γ∗=7; κ=2; ρ∗=0.35

d

xt

Can essentially replicate break by changing µ, γ κ and ρ in manycombinations: economic agents could not tell what had shifted tilllong afterwards. Analysis in Gabaix (2012) is infeasible.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 47 / 49

What can economic agents do?

Robust forecasts avoid systematic failure.Difference mis-specified model:

∆xT+h|T+h−1 = ρ∆xT+h−1 or:

xT+h|T+h−1 = xT+h−1 + ρ∆xT+h−1 (33)

Uses ‘wrong’ ρ for first 5 forecasts;incorrectly differenced;and omits relevant variable.

But no location shifts.


Differenced wrong model with changed ρ


30 35 40 45 50

15

20

25

30

35

40

45

50

Robust forecasting device


Robust forecasting device (33) avoids most of last 9 forecast errors.RMSFE of estimated DGP (30) is 6.6 versus 5.5 here; but 3.8 versus2.0 over last 9 forecasts, so nearly halved.Never judge a model’s verisimilitude by forecast performance.David F. Hendry (INET at OMS) Deciding between Alternative Approaches ESRC/OMS Conference, 2012 49 / 49

deciding between alternative approaches in macroeconomics...much past applied econometrics research...

Documents