analysis of numerical errors - amazon s3analysis of numerical errors adrian peralta-alva and manuel...
TRANSCRIPT
Research Division Federal Reserve Bank of St. Louis Working Paper Series
Analysis of Numerical Errors
Adrian Peralta-Alva and
Manuel S. Santos
Working Paper 2012-062A http://research.stlouisfed.org/wp/2012/2012-062.pdf
December 2012
FEDERAL RESERVE BANK OF ST. LOUIS Research Division
P.O. Box 442 St. Louis, MO 63166
______________________________________________________________________________________
The views expressed are those of the individual authors and do not necessarily reflect official positions of the Federal Reserve Bank of St. Louis, the Federal Reserve System, or the Board of Governors.
Federal Reserve Bank of St. Louis Working Papers are preliminary materials circulated to stimulate discussion and critical comment. References in publications to Federal Reserve Bank of St. Louis Working Papers (other than an acknowledgment that the writer has had access to unpublished material) should be cleared with the author or authors.
Analysis of Numerical Errors
Adrian Peralta-Alva and Manuel S. Santos
⇤
This paper provides a general framework for the quantitative analysis ofstochastic dynamic models. We review convergence properties of somenumerical algorithms and available methods to bound approximation er-rors. We then address convergence and accuracy properties of the sim-ulated moments. Our purpose is to provide an asymptotic theory forthe computation, simulation-based estimation, and testing of dynamiceconomies. The theoretical analysis is complemented with several illus-trative examples. We study both optimal and non-optimal economies.Optimal economies generate smooth laws of motion defining Markov equi-libria, and can be approximated by recursive methods with contractiveproperties. Non-optimal economies, however, lack existence of contin-uous Markov equilibria, and need to be computed by other algorithmswith weaker approximation properties.
Keywords: Stochastic Dynamic Model, Markov Equilibrium, Nu-merical Solution, Approximation Error, Accuracy, Simulation-Based Es-timation, Consistency.
JEL Codes: C63, E60.
⇤A. Peralta-Alva: Research Division, Federal Reserve Bank of Saint Louis. M. Santos: Depart-
ment of Economics, University of Miami. The usual disclaimer applies.
1
1 Introduction
Numerical methods are essential to assess the predictions of nonlinear economic mod-els. Indeed, a vast majority of models lack analytical solutions, and hence researchersmust rely on numerical algorithms—which contain approximation errors. At theheart of modern quantitative analysis is the presumption that the numerical methodmimics well the original model statistics. In practice, however, matters are not sosimple and there are many situations in which researchers are unable to control forundesirable propagating effects of numerical errors.
In static economies it is usually easy to bound the size of the error. But ininfinite-horizon models we have to realize that numerical errors may cumulate inunexpected ways. Cumulative errors can be bounded in models where equilibriamay be approximated by a contraction operator. But if the contraction property ismissing then the most that one can hope for is to establish asymptotic properties ofthe numerical solution as we refine the approximation. Numerical errors may biasstationary solutions and the simulated moments, and hence parameter estimatesfrom simulation-based estimation.
Model simulation is mechanically performed in macroeconomics and other disci-plines, but there is much to be learned about laws of large numbers that can justifythe convergence of the simulated moments, and the propagating effects of numericalerrors in these simulations.
Simulation-based estimation must also cope with changes in parameter valuesaffecting the dynamics of the system. Indeed, the estimation process encompasses acontinuum of invariant distributions indexed by a vector of parameters. Therefore,simulation-based estimation needs fast and accurate algorithms that can sample theparameter space. Asymptotic properties of these estimators such as consistency andnormality are much harder to establish than in traditional data-based estimation inwhich there is a unique stochastic distribution given by the data generating process.
This chapter is intended to survey theoretical work on convergence properties ofnumerical algorithms and the accuracy of simulations. More specifically, we shallreview the established literature with an eye towards a better understanding of thefollowing issues: (i) Convergence properties of numerical algorithms and accuracytests that can bound the size of approximation errors, (ii) Accuracy properties ofthe simulated moments from numerical algorithms and laws of large numbers thatcan justify model simulation, and (iii) Calibration and simulation-based estimation.
We study these issues along with a few illustrative examples. We focus on a large
1
class of dynamic general equilibrium models of wide application in economics andfinance. We break the analysis into optimal and non-optimal economies. Optimaleconomies satisfy the welfare theorems. Hence, equilibria can be computed by as-sociated optimization problems, and under regular conditions these equilibria admitMarkovian representations defined by continuous (or differentiable) policy functions.Non-optimal economies may lack existence of Markov equilibria — or such equilib-ria may not be continuous. One could certainly restore the Markovian property byexpanding the state space, but the non-continuity of the equilibrium still remains.These technical problems limit the application of standard algorithms which assumecontinuous or differentiable approximation rules. Differentiability properties of thesolution are instrumental to characterize the dynamics of the system and to estab-lish error bounds. We here put together several results for the computation andsimulation of upper semicontinuous correspondences. The idea is to build reliablealgorithms and laws of large numbers that can be applied to economies with marketfrictions and heterogeneous agents as commonly observed in many macroeconomicmodels.
Section 2 lays out an analytical setting conformed by several equilibrium condi-tions that include feasibility constraints and first-order conditions. This simplifiedframework is appropriate for computation. We then consider three illustrative ex-amples: A growth model with taxes, a consumption-based asset-pricing model, andan overlapping generations economy. We show how these economies can readily bemapped into our general framework of analysis.
Our main theoretical results are presented in Sections 3 and 4. Each section startswith a review of some numerical solution methods, and then goes into the analysis ofassociated computational errors and convergence of the simulated statistics. Section3 deals with models with continuous Markov equilibria. There is a vast literature onthe computation of these equilibria, and here we only deal with the bare essentials.We nevertheless provide a more comprehensive study of the associated error fromnumerical approximations. Some regularity conditions, such as differentiability orcontraction properties, may validate error bounds. Accuracy of the simulated mo-ments and consistency properties of simulation-based estimators are also discussed.
Section 4 is devoted to non-optimal economies. For this family of models, Markovequilibria on the natural state space may fail to exist, and standard computationalmethods – which iterate over continuous functions – may produce inaccurate solu-tions. We discuss a reliable algorithm based on the iteration over candidate equi-librium correspondences. The algorithm has good convergence and approximation
2
properties, and its fixed point contains a Markovian correspondence that generatesall competitive equilibria. Hence, competitive equilibria still admit a recursive rep-resentation. But this representation may only be obtained in an enlarged state space(which includes the shadow values of asset holdings), and may not be continuous.The non-continuity of the equilibrium solution precludes application of standardlaws of large numbers. This is problematic because we need an asymptotic theoryto justify the simulation, and estimation of macroeconomic models. We discuss anextended version of the law of large numbers which entails that the sample momentsfrom numerical approximations must approach those of some invariant distributionof the model as the error in the approximated equilibrium correspondence vanishes.
Section 5 presents several numerical experiments. We first study a standardbusiness cycle problem. This optimal planning problem becomes handy to assessthe accuracy of the computed solutions using the Euler equation residuals. We thenintroduce some non-optimal economies in which simple Markov equilibria fail to exist:An overlapping generations economy and an asset-pricing model with endogenousconstraints. These examples make clear that standard solution methods would resultin substantial computational errors that may drastically change the ergodic sets andcorresponding equilibrium dynamics. These examples are then computed by ourreliable algorithm introduced in Section 4. This algorithm can also be applied to someother models of interest with heterogeneous agents such as a production economywith taxes and individual rationality constraints, and a consumption-based assetpricing model with collateral requirements. There are cases in which the solution ofthis robust algorithm approaches a continuous policy function, and hence we havenumerical evidence of existence of a unique equilibrium. Uniqueness of equilibriumguarantees existence and continuity of a simple Markov equilibrium—which simplifiesthe computation and simulation of the model. Uniqueness of equilibrium is hard tocheck using standard numerical methods.
We conclude in Section 6 with further comments and suggestions.
2 Stochastic Dynamic Economies
Our objective is to study quantitative properties of stochastic sequences {st
(z
t
)}t�0
that emerge as equilibria of our model economies. These equilibrium sequencesarise from the solution of nonlinear equation systems, the intertemporal optimizationbehavior of individual agents, as well as the economy’s aggregate constraints, and theexogenously given sequence of shocks {zt}. Our framework of analysis encompasses
3
both competitive and non-competitive economies, with or without a governmentsector, and incomplete financial markets.
Time is discrete, t = 0, 1, 2 · · · , and z
t
= (z0, z1, ..., zt) is a history of shocks up toperiod t, which is governed by a time invariant Markov process. For convenience, letus decompose the economic variables of interest as s
t
(z
t
) = (x
t
(z
t
) , y
t
(z
t
)) . Vectorx
t
represents predetermined variables, such as capital stocks and portfolio holdings.Future values of these variables will be determined endogenously by current andfuture actions. Vector y
t
denotes all other current endogenous variables such asconsumption, investment, asset prices, interest rates, and so on. For convenience,sometimes vector s may include the shock z.
The dynamics of state vector x will be captured by a system of nonlinear equa-tions:
' (x
t+1, xt
, y
t
, z
t
) = 0. (1)
Function ' may incorporate technological constraints and individual budget constraints.Likewise, present and future values of vector y are linked by the non-linear system:
� (x
t
, y
t
, z
t
, E
t
[x
t+1, yt+1, zt+1]) = 0, (2)
where Et
[·] is the expectations operator conditional on information at time t. Condi-tions describing function � may correspond to individual optimality conditions (suchas Euler equations), short-sales and liquidity requirements, endogenous borrowingconstraints, individual rationality constraints, and market clearing conditions.
We now present three different examples to illustrate that standard macro modelscan readily be mapped into this framework.
2.1 A growth model with taxes
The economy is made up of a representative household and a single firm. Theexogenously given stochastic process z
t
is an index of total factor productivity. Forgiven sequences of interest rates, r
t
, wages, wt
, profits redistributed by the firm, ⇡t
,government lump-sum transfers, T
t
, and tax functions, ⌧t
, the household solves thefollowing optimization problem:
4
maxE0
1X
t=0
�
t
[log(c
t
) + �log(1� l
t
)]
s.t.
⇡
t
+ (r
t
+ (1� �)) k
t
+ w
t
l
t
� ⌧
h
t
+ T
t
� (c
t
+ k
t+1) = 0, (3)
k0 given, 0 < � < 1,
c
t
� 0, kt+1 � 0 for all zt, and t � 0.
Here, ct
denotes consumption, lt
denotes the amount of labor supplied, kt
denotesholdings of physical capital. Parameter � is the discount factor and � is the capitaldepreciation rate. Taxes
�
⌧
h
t
may be non-linear functions of income variables (suchas capital or labor income) and of the aggregate capital stock K
t
. Households takethe sequences of tax functions as given—contingent upon the history of realizationsz
t
.
For a given sequence of technology shocks {zt
} , factor prices and output taxesn
r
t
, w
t
, ⌧
f
t
o
, the representative firm seeks to maximize one-period profits by selectingthe optimal amount of capital and labor
⇡
t
= max
K
t
,L
t
z
t
⇣
1� ⌧
f
t
⌘
f (K
t
, L
t
)� r
t
K
t
� w
t
L
t
.
All tax revenues are rebated back to the representative household as lump-sumtransfers T
t
.
For a given sequence of tax functions {⌧t
} and transfers {Tt
}, a competitive
equilibrium for this economy is conformed by stochastic sequences of factor pricesand profits {r
t
, w
t
, ⇡
t
}, and sequences of consumption, capital and labor allocations{c
t
, k
t
, l
t
, K
t
, L
t
}, such that: (i) {ct
, k
t
, l
t
} solve the above optimization problem of thehousehold; and {K
t
, L
t
} maximizes one-period profits for the firm; (ii) The suppliesof capital and labor are equal to the quantities demanded: k
t
(z
t
) = K
t
(z
t
), andl
t
(z
t
) = L
t
(z
t
) for all zt and t � 0; (iii) Consumption and investment allocations arefeasible
z
t
f(k
t
, 1� l
t
) + (1� �)k
t
� (c
t
+ k
t+1) = 0, for all zt and t � 0. (4)
Getting back to our general framework, we observe that capital is the only predeter-mined endogenous variable: x
t
= k
t
, while consumption and hours worked are theendogenous variables y
t
= (c
t
, l
t
). Function ' is thus given by (4). The intertemporal
5
equilibrium conditions amount to
1
c(z
t
)
� �E
t
8
<
:
[1� � + z
t+1(1� ⌧
f
t
)f
K
t+1(zt
)� @⌧
h
t+1
@k
t+1]
c(z
t+1)
9
=
;
= 0. (5)
(1� ⌧
f
t
)f
L
t+1(zt
)� @⌧
h
t
@l
t
� �
c(z
t
)
1� l(z
t
)
= 0. (6)
Therefore, � is defined by equations systems (5-6) over constraint (3).
2.2 An asset pricing model with financial frictions
The economy is populated by a finite number of agents, i = 1, 2, · · · , I. At each nodez
t, there exist spot markets for the consumption good and a fixed set j = 1, 2, · · · , Jof securities. For convenience we assume that the supply of each security is equalto unity. Among these securities, we may include a one-period real bond whichis a promise to one unit of the consumption good at all successor nodes z
t+1|zt.Our general stylized framework above can embed several financial frictions such asincomplete markets, collateral requirements, and short-sale constraints.
Each agent i maximizes the intertemporal objective
E
" 1X
t=0
�
�
i
�
t
u
i
�
c
i
t
�
#
, (7)
where �
i 2 (0, 1), and u
i is strictly increasing, strictly concave, and continuouslydifferentiable with derivative (u
i
)
0(0) = 1. At each node z
t the agent receivese
i
(z
t
) > 0 units of the consumption good contingent on the present realization z
t
.Securities are specified by the current vector of prices, q
t
(z
t
) = (· · · , qjt
(z
t
), · · · ), andthe vectors of dividends d(zr) = (· · · , dj(zr), · · · ) promised to deliver at future infor-mation sets z
r|zt for r > t. The vector of security prices q
t
(z
t
) is non-negative, andthe vector of dividends d
t
(z
t
) is positive and depends only on the current realizationof the vector of shocks z
t
.For a given price process (q
t
(z
t
))
t�0 , each agent i can choose desired quantitiesof consumption and security holdings (c
i
t
(z
t
) , ✓
i
t+1 (zt
))
t�0 subject to the following
6
sequence of budget constraints
c
i
t
�
z
t
�
+ ✓
i
t+1(zt
) · qt
�
z
t
�
� (8)
[e
i
(z
t
) + ✓
i
t
�
z
t�1�
·�
q
t
�
z
t
�
+ d (z
t
)
�
] = 0
0 ✓
i
t+1
�
z
t
�
, ✓
i
0 given, (9)
for all zt. Note that (9) imposes non-negative holdings of all securities. Let �
i
(z
t
)
be the associated vector of multipliers to this non-negativity constraint.A competitive equilibrium for this economy is a collection of vectors (c
t
(z
t
) ,
✓
t+1 (zt
) , p
t
(z
t
), q
t
(z
t
))
t�0 such that: (i) Each agent i maximizes the objective (7)subject to constraints(8)-(9), and (ii) Markets clear:
J
X
j
d
j
(z
t
) +
I
X
i
e
i
t
�
z
t
�
�I
X
i
c
i
t
�
z
t
�
= 0 , (10)
I
X
i
✓
ji
t+1
�
z
t
�
� 1 = 0, (11)
for j = 1, · · · , J, all zt.It is not hard to see that this model can be mapped into our analytical framework.
Again, the vector of exogenous shocks {zt
} defines sequences of endowments anddividends, {e
t
(z
t
), d
t
(z
t
)} . Without loss of generality, we have assumed that the spaceof asset holdings is given by ⇥ =
n
✓ 2 R
JI
+ :
P
I
i=1 ✓ji
= 1 for all jo
. Asset holdings✓ are the predetermined variables corresponding to vector x
t
, whereas consumptionc and asset prices q are the current endogenous variables corresponding to vector y
t
.Function ' is simply given by the vector of individual budget constraints (8-9).
Function � is defined by the first-order conditions for intertemporal utility maxi-mization over the equilibrium conditions for the aggregate good and financial mar-kets (10-11). Observe that all constraints hold with equality as we introduce theassociated vectors of multipliers for the non-negativity constraints.
2.3 An overlapping generations economy
We study a version of the economy analyzed by Kubler and Polemarchakis (2004).The economy is subject to an exogenously given sequence of shocks {z
t
} , with z
t
2 Z
for all t = 1, · · · . At each date, I new individuals appear in the economy and staypresent for N + 1 periods. Thus, agents are distinguished by their individual typei 2 I, and the specific date-event in which they initiate their life span z
⌧
= (z0, ..., z⌧ ).
7
There are L goods, and each individual receives a positive stochastic endowmente
i,z
⌧
(z
⌧+a
) 2 RL
+ at every node z
⌧+a while present in the economy. Endowmentsare assumed to be Markovian—defined by the type of the agent, i, age, a, and thecurrent realization of the shock, z
t
. Preferences over stochastic consumption streamsc are represented by an expected utility function
U
i,z
⌧
= E
z
⌧
N
X
a=0
u
i,a,z
(c
i,z
⌧
(z
⌧+a
)). (12)
Again, we impose a Markovian structure on preferences—assumed to depend on i,a, and the current realized value z.
At each date-event z
t agents can trade one-period bonds that pay one unit ofnumeraire good 1 regardless of the state of the world next period. These bonds arealways in zero net supply, and q
b
(z
t
) is the price of a bond that trades at date-eventz
t
. An infinitely-lived Lucas tree may also be available from time zero. The treeproduces a random stream of dividends d(z
t
) of consumption good 1. Then, qs(zt)is the market value of the tree, and ✓
b,i,z
⌧
, ✓
s,i,z
⌧ the holdings of bonds and shares ofthe tree for agent (i, z
⌧
) . Shares cannot be sold short.Each individual consumer (i, z⌧ ) faces the following budget constraints for periods
⌧ t ⌧ +N + 1,
p(z
t
)c
i,z
⌧
(z
t
) + q
s
(z
t
)[✓
s,i,z
⌧
(z
t+1)� ✓
s,i,z
⌧
(z
t
)] + q
b
(z
t
)✓
b,i,z
⌧
(z
t+1)�
[p(z
t
)e
i,z
⌧
(z
t
) + ✓
b,i,z
⌧
(z
t
) + ✓
s,i,z
⌧
(z
t
)d(z
t
)] = 0, (13)
0 ✓
s,i,z
⌧
(z
t+1) (14)
0 ✓
b,i,z
⌧
(z
⌧+N+2) (15)
Note that (14) insures that share holdings must be non-negative, whereas (15) insuresthat debts must be honored in the terminal period.As before, a competitive equilibrium for this economy is conformed by sequencesof prices, (qb(zt),qs(zt), p(zt)), consumption allocations, (c(zt+a
)) and asset holdings(✓
b,i,z
⌧
, ✓
s,i,z
⌧
) for all agents over their corresponding ages, such that: (i) Each agentmaximizes her expected utility subject to individual budget constraints, (ii) Thegoods markets clear: Consumption allocations add up to the aggregate endowment
8
at all possible date-events, and (iii) Financial markets clear: Bond holdings add upto zero and share holdings add up to one.
Of course, our purpose now is to clarify how to map this model into the analyticalframework developed before. The vector of exogenous shocks is {zt} , which definesthe endowment and dividend processes
�
d(z
t
), e
i,z
⌧
(z
t
)
. As predetermined variableswe have x(z
t
) =
�
✓
b,i,z
⌧
(z
t
), ✓
s,i,z
⌧
(z
t
)
; that is, the portfolio holdings for all agents(i, z
⌧
) alive at every date-event z
t. And as current endogenous variables we havey(z
t
) =
�
c
i,z
⌧
(z
t
), q
b
(z
t
), q
s
(z
t
)
the consumption allocations and the prices of boththe bond and the Lucas tree for every date-event z
t.For the sake of the presentation, let’s consider a version of the model with one
consumption good and two agents that live for two periods. Function ' is simplygiven by the vector of individual budget constraints (13). Function � is defined by:(i) The individual optimality conditions for bonds
u
i,a,z
c
�
c
i,z
⌧
(z
t
)
�
q
b
(z
t
)� E
⇥
u
i,a,z
c
�
c
i,z
⌧
(z
t+1)
�⇤
= 0; (16)
(ii) If the Lucas tree is available, the Euler equation
u
i,a,z
c
�
c
i,z
⌧
(z
t
)
�
q
s
(z
t
)� E
⇥
u
i,a,z
c
�
c
i,z
⌧
(z
t+1)
� �
q
s
(z
t+1) + d(z
t+1)
�⇤
� �
i,z
⌧
(z
t+1) = 0,
(17)where � is the multiplier on the short-sales constraint (14), and (iii) Market clearingconditions. It is clear that many other constraints may be brought up into theanalysis, such as a collateral restriction along the lines of Kubler and Schmedders(2003) that set up a limit for negative holdings of the bond based on the value of theholdings of the tree.
3 Numerical Solution of Simple Markov Equilibria
For the above models, fairly general conditions guarantee the existence of stochasticequilibrium sequences. But even if the economy has a Markovian structure (i.e., thestochastic process driving the exogenous shocks and conditions (1) and (2) over theconstraints are Markovian), equilibrium sequences may depend on the full history ofshocks. Equilibria with this type of path dependence are not amenable to numericalor statistical methods.
Hence, most quantitative research has focused on models where one can findcontinuous functions g
x
, g
y such that the sequences
9
(x
t+1, yt+1) = (g
x
(x
t
, z
t
), g
y
(x
t
, z
t
)) (18)
generate a competitive equilibrium. Following Krueger and Kubler (2008), a com-petitive equilibrium that can be generated by equilibrium functions of the form (18)will be called a simple Markov equilibrium.
For frictionless economies the second welfare theorem applies—equilibrium allo-cations can be characterized as solutions to a planner’s problem. Using dynamicprogramming arguments, well established conditions on primitives insure existenceof a simple Markov equilibrium [cf., Stokey, Lucas and Prescott (1989)]. Mattersare more complicated in models with real distortions such as taxes, or with financialfrictions such as incomplete markets and collateral constraints. Section 4 details theissues involved. In this section we will review some results on accuracy of numericalmethods for simple Markov equilibria. First, we discuss some of the algorithms avail-able to approximate equilibrium function g. Then, we study methods to determinethe accuracy of such approximations. Finally, we discuss how the approximationerror in the policy function may propagate over the simulated moments affecting theestimation of parameter values.
3.1 Numerical approximations of equilibrium functions
We can think of two major families of algorithms approximating simple Markovequilibria. The first group approximates directly the equilibrium functions (1-2) usingthe Euler equations and constraints [A variation of this method is Marcet’s originalparameterized expectations algorithm that indirectly gets the equilibrium functionvia an approximation of the expectations function below, e.g., see Christiano andFisher (2000)]. These numerical algorithms may use local approximation techniques(perturbation methods), or global approximation techniques (projection methods).Projection methods require finding the fixed point of an equations system whichmay be highly non-linear. Hence, projection methods offer no guarantee of globalconvergence and uniqueness of the solution.
Another family of algorithms is based on dynamic programming (DP). The DPalgorithms are reliable and have desirable convergence properties. However, theircomputational complexity increases quite rapidly with the dimension of the statespace, especially because maximizations must be performed at each iteration. Inaddition, DP methods cannot be extended to models with distortions where thewelfare theorems do not apply. For instance, in the above examples in Section 2,
10
for most formulations, the growth model with taxes, the asset pricing model withvarious added frictions, and the overlapping generations economy cannot be solveddirectly by DP methods. In all these economies, an equilibrium solution cannot becharacterized by a social planning problem.
3.1.1 Methods based on the Euler equations
Simple Markov equilibria are characterized by continuous functions gx(x, z), gy(x, z)that satisfy
' (g
x
(x, z), x, g
y
(x, z), z
t
) = 0 (19)
�
�
x, g
y
(x, z)
t
, z, E
z
0|z [gx
(x, z), g
y
(g
x
(x, z), z
0), z
0]
�
= 0, (20)
for all (x, z). Of course, in the absence of an analytical solution the system must besolved by numerical approximations.
As mentioned above, two basic approaches are typically used to obtain approxi-mate functions bgx, bgy. Perturbation methods—pioneered by Judd and Guu (1997)—take a Taylor approximation around a point with known solution or quite close to theexact solution. This point typically corresponds to the deterministic steady state ofthe model, that is, an equilibrium where z
t
= z
⇤, x
t
= x
⇤, y
t
= y
⇤ for all t. Projectionmethods—developed by Judd (1992)—aim instead at more global approximations.First, a finite-dimensional space of functions is chosen that can approximate arbi-trarily well continuous mappings. Common finite-dimensional spaces include finiteelements (tent maps, splines, polynomials defined in small neighborhoods), or globalbases such as polynomials or other functions defined over the whole domain X. Letbg
x
⇠
x
n
, bg
y
⇠
y
n
be elements of this finite-dimensional space evaluated at the nodal points⇠
x
n
, ⇠
y
n
defining these functions. Nodal values bgx⇠
x
n
, bg
y
⇠
y
n
are obtained as solutions of non-linear systems conformed by equations (19-20) evaluated at some pre-determinedpoints of the state space X. It is assumed that this non-linear system has a welldefined solution—albeit in most cases existence of the solution is hard to show. Rulesfor the optimal placement of such pre-determined points exist for some functionalbasis; e.g., Chebyshev polynomials that have some regular orthogonality propertiescould be evaluated at the Chebyshev nodes in the hope of minimizing oscillations.
11
3.1.2 Dynamic programming
For economies satisfying the conditions of the second welfare theorem, equilibria canbe computed by an optimization problem over a social welfare function subject toaggregate feasibility constraints. Then, one can find prices that support the plan-ner’s allocation as a competitive equilibrium with transfer payments. A competitiveequilibrium is attained when these transfers are equal to zero. Therefore, we need tosearch for the appropriate individual weights in the social welfare function in orderto make these transfers equal to zero.
Matters are simplified by the principle of optimality: The planner’s intertemporaloptimization problem can be summarized by a value function V satisfying Bellman’sfunctional equation
V (x, z) = sup
x
0,y
F (x, x
0, y, z) + �E
z
0|zV (x
0, z
0) (21)
s.t. (x
0, y
0) 2 �(x, z).
Here, 0 < � < 1 is the intertemporal discount factor, F is the one-period returnfunction, and �(x, z) is a correspondence that captures the feasibility constraints ofthe economy. Note that our vectors x and y now refer to allocations only, whilein previous decentralized models these vectors may include prices, taxes, or othervariables outside the planning problem.
Value function V is therefore a fixed point of Bellman’s equation (21). Undermild regularity conditions [cf., Stokey, Lucas and Prescott (1989)] it is easy to showthat this fixed point can be approximated by the following operator. Let V be thespace of bounded functions. Then, operator T : V !V is defined as
TW (x, z) = sup
x
0,y
F (x, x
0, y, z) + �E
z
0|zW (x
0, z
0) (22)
s.t. (x
0, y
0) 2 �(x, z).
Operator T is actually a contraction with modulus �. It follows that V is a uniquesolution of the functional equation (21), and can be found as the limit of the sequencerecursively defined by V
n+1 = TV
n
for an arbitrarily given initial function V0. Thisiterating procedure is called the method of successive approximations, and operatorT is called the DP operator.
By the contraction property of the DP operator, it is possible to construct reliable
12
numerical algorithms discretizing (22). For instance, Santos and Vigo-Aguiar (1998)establish error bounds for a numerical DP algorithm preserving the contraction prop-erty. The analysis starts with a set of piecewise-linear functions defined over statespace X on a discrete set of nodal points with grid size h. Then, a discretized ver-sion T
h of operator T is obtained by solving the optimization problem (22) at eachnodal point—with piecewise-linear interpolation over all other functional values forV (x
0, z
0). For piecewise-linear interpolation, operator T
h is also a contraction map-ping. Hence, given any grid size h, and any initial value function V0 the sequence offunctions V h
n+1 = T
h
V
h
n
under repeated application of operator T h are guaranteed toconverge to a unique solution V
⇤,h. Moreover, the contraction property of operatorT
h can help bound the distance between such limit V
⇤,h, and the N -th applicationof this operator, V h
N+1. Finally, it is important to remark that this approximationscheme will converge to the true solution of the model as the grid size h goes tozero; that is, V ⇤,h will be sufficiently close to the original value function V for somesmall h—as a matter of fact convergence is of order h2. Of course, once a numericalvalue function V
h has been secured it is easy to obtain good approximations for ourequilibrium functions bgx, bgy from operator T
h
V
h
N+1.
What slows down the DP algorithm is the maximization process at each itera-tion. Hence, functional interpolation—as opposed to discrete functions just definedover a set of nodal points—facilitates the use of some fast maximization routines.Splines and high-order polynomials may also be operative but these approximationsmay damage the concavity of the computed functions; moreover, for some interpo-lations there is no guarantee that the discretized operator is a contraction. Thereare other procedures to speed up the maximization process. Santos and Vigo-Aguiar(1998) use a multigrid method which can be efficiently implemented by an analysisof the approximation errors. Another popular method is policy iteration—contraryto popular belief this latter algorithm turns out to be quite slow for very fine grids[Santos and Rust (2004)].
3.2 Accuracy
As already pointed out, the quantitative analysis of non-linear models primarily relieson numerical approximations bg. Then, care must be exercised so that numericalequilibrium function bg is close enough to the actual decision rule g; more precisely,we need to insure that ||g� bg|| ", where ||.|| is a norm relevant for the problem athand, and " is a tolerance estimate.
13
We now present various results for bounding the error in numerical approxima-tions. Error bounds for optimal decision rules are available for some computationalalgorithms such as the above DP algorithm. It should be noted that these errorbounds are not good enough for most quantitative exercises in which the object ofinterest is the time series properties of the simulated moments. Error bounds foroptimal decision rules quantify the period-by-period bias introduced by a numeri-cal approximation. This error, however, may grow in long simulations. A simpleexample below illustrates this point where the error of the simulated statistics getslarge even when the error of the decision rule can be made arbitrarily small. Hence,the last part of this section considers some of the regularity conditions required fordesirable asymptotic properties of the statistics from numerical simulations.
3.2.1 Accuracy of equilibrium functions
Suppose that we come up with a pair of numerical approximations bgx, bgy. Is there away of assessing the magnitude of the approximation error without actual knowledgeof the solution of the model: g
x
, g
y?To develop intuition on key ideas behind existing accuracy tests, let us define the
Euler equation residuals for functions bgx, bgy as
' (bg
x
(x, z), x, bg
y
(x, z), z
t
) = EE
'
(bg
x
, bg
y
) (23)
�
�
x, bg
y
(x, z)
t
, z, E
z
0|z [bgx
(x, z), bg
y
(bg
x
(x, z), z
0), z
0]
�
= EE
�(bg
x
, bg
y
). (24)
Note that an exact solution of the model will have Euler equation residuals equalto zero at all possible values of the state (x, z). Hence, “small” Euler equationresiduals should indicate that the approximation error is also “small”. The relevantquestion, of course, is what we mean by “small”. Furthermore, we are circumventingother technical issues since first-order conditions may not be enough to characterizeoptimal solutions.
Den Haan and Marcet (1994) appeal to statistical techniques and propose testingfor orthogonality of the Euler equation residuals over current and past information asa measure of accuracy. Since orthogonal Euler equation residuals may occur in spiteof large deviations from the optimal policy, Judd (1992) suggests to evaluate thesize of the Euler equation residuals over the whole state space as a test for accuracy.Moreover, for strongly concave infinite-horizon optimization problems Santos (2000)demonstrates that the approximation error of the policy function is of the same order
14
of magnitude as the size of the Euler equation residuals, and the constants involvedin these error bounds can be related to model primitives.
These theoretical error bounds are based on worse-case scenarios and hence theyare usually not optimal for applied work. In some cases, researchers may want toassess numerically the approximation errors in the hope of getting more operativeestimates [cf. Santos (2000)]. Besides, for some algorithms it is possible to deriveerror bounds from their approximation procedures. This is the case of the DP al-gorithm [Santos and Vigo-Aguiar (1998)] and in some models with quadratic-linearapproximations [Schmitt-Grohe and Uribe (2004)].
The logic underlying numerical estimation of error bounds from the Euler equa-tion residuals goes as follows [Santos (2000)]. We start with a model under a fixedset of parameter values. Then, Euler equation residuals are computed for severalnumerical equilibrium functions. We need sufficient variability in these approxima-tions in order to obtain good and robust estimates. This variability is obtained byconsidering various approximation spaces or by changing the grid size. Let bg
acc
bethe approximation with the lowest Euler equation residuals, which would be our bestcandidate for the true policy function. Then, for each available numerical approxi-mation bg we compute the approximation constant
M
NUM
bg =
k bg � bg
acc
kk EE(bg)k . (25)
Here, k ·k is the max norm in the space of functions. From the available theory [cf.Santos (2000)], the approximation error of the policy function is of the same orderof magnitude as that of the Euler equation residuals. Then, the values of MNUM
bg
should have bounded variability (unless the approximation bg is of the same orderof magnitude as bg
acc
). Indeed in many cases M
NUM
bg hovers around certain values.Hence, any upper bound M
NUM for these values would be a conservative estimatefor this approximation constant. It follows that the resulting assessed value, MNUM ,can be used to estimate an error bound for our candidate solution:
kg � bg
acc
k M
NUM kEE(bg
acc
)k . (26)
Note that in this last equation we contemplate the error between our best policyfunction bg
acc
and the true policy function g.Therefore, worst-case error bounds are directly obtained from constants given by
the theoretical analysis. These bounds are usually very conservative. The numerical
15
estimation of these bounds is presented here as a heuristic procedure to calculate theactual value of the bounding constant. From the available theory we know that theerror of the equilibrium function is of the same order of magnitude as the size of theEuler equation residuals. That is, the following error bound holds:
kg � bgk M
NUM kEE(bg)k . (27)
We thus obtain an estimate M
NUM for constant M from various comparisons ofapproximated equilibrium functions.
3.2.2 Accuracy of the simulated moments
Researchers usually focus on long-run properties of equilibrium time series. Thecommon belief is that equilibrium orbits will stabilize and converge to a station-ary distribution. Stationary distributions are simply the stochastic counterparts ofsteady states in deterministic models. Computation of the moments of an invariantdistribution for a non-linear model is usually a rather complicated task—even for an-alytical equilibrium functions. Hence, laws of large numbers are invoked to computethe moments of an invariant distribution from the sample moments.
The above one-period approximation error (27) is just a first step to controlthe cumulative error of numerical simulations. Following Santos and Peralta-Alva(2005), our goal now is to present some regularity conditions so that the error fromthe simulated statistics converges to zero as the approximated equilibrium functionapproaches the exact equilibrium function. The following example illustrates thatcertain convergence properties may not always hold.
Example: The state space S is a discrete set with three possible states, s1, s2, s3.Transition probability P is defined by the following Markov matrix
⇧ =
2
6
4
1 0 0
0 1/2 1/2
0 1/2 1/2
3
7
5
.
Each row i specifies the probability of moving from state s
i
to any state in S, sothat an element ⇡
ij
corresponds to the value P (s
i
, {sj
}), for i, j = 1, 2, 3. Note that⇧
n
= ⇧ for all n � 1. Hence, p = (1, 0, 0), and p = (0, 1/2, 1/2) are invariantprobabilities under ⇧, and {s1} and {s2, s3} are the ergodic sets. All other invariantdistributions are convex combinations of these two probabilities.
16
Let us now perturb ⇧ slightly so that the new stochastic matrix is the following
b
⇧ =
2
6
4
1� 2↵ ↵ ↵
0 1/2 1/2
0 1/2 1/2
3
7
5
for 0 < ↵ < 1/2.
As n ! 1, the sequence of stochastic matrices {b⇧n} converges to
2
6
4
0 1/2 1/2
0 1/2 1/2
0 1/2 1/2
3
7
5
.
Hence, p = (0, 1/2, 1/2) is the only possible long-run distribution for the system.Moreover, {s1} is a transient state, and {s2, s3} is the only ergodic set. Consequently,a small perturbation on a transition probability P may lead to a pronounced changein its invariant distributions. Indeed, small errors may propagate over time and alterthe existing ergodic sets.
Santos and Peralta-Alva (2005) show that certain continuity properties of thepolicy function suffice to establish some generalized laws of large numbers for nu-merical simulations. To provide a formal statement of their results, we need to laydown some standard concepts and terminology.
For ease of presentation, we restrict attention to exogenous stochastic shocks ofthe form
z
t+1 = (zt, "t+1)
where " is an iid shock. The distribution of this shock " is denoted by probabilitymeasure Q on a measurable space (E,E). Then, as it is standard in the literature [cf.Stokey, Lucas and Prescott (1989)] we define a new probability space comprising allinfinite sequences ! = ("1, "2, · · · ). Let ⌦ = E
1 be the countably infinite cartesianproduct of copies of E. Let F be the �-field in E
1 generated by the collection of allcylinders A1 ⇥ A2 ⇥ · · · ⇥ A
n
⇥ E ⇥ E ⇥ E ⇥ · · · where A
i
2 E for i = 1, · · · , n. Aprobability measure � can be constructed over the finite-dimensional sets as
�{! : "1 2 A1, "2 2 A2, · · · , "n 2 A
n
} =
n
Y
i=1
Q(A
i
).
Measure � has a unique extension on F. Hence, the triple (⌦,F,�) denotes a prob-ability space. Now, for every initial value s0 and sequence of shocks ! = {"
t
}, let
17
{xt
(s0,!), yt(s0,!)} be the sample paths generated by the policy functions gx, gy, sothat s
t+1(s0,!) = (x
t+1(s0,!), yt+1(s0,!)) = (g
x
(x
t
(s0,!), "t+1), gy
(x
t
(s0,!), "t+1))
for all t � 1.
Let bs
jt
(s0,!) be the sample path generated from an approximate policy func-tion bg
j
. Averaging over these sample paths we get sequences of simulated statistics{ 1T
P
T
t=1 f(bsjt(s0,!))} as defined by some function f . Let E(f) =
´f(s)µ
⇤(ds) be
the expected value under an invariant distribution µ
⇤ of the original equilibriumfunction g. Santos and Peralta-Alva (2005) establish the following result:
Theorem 1 Assume that the sequence of approximated equilibrium functions {bgj
}converges in the sup norm to equilibrium function g. Assume that g is a continuousmapping over a compact domain, and contains a unique invariant distribution µ
⇤.Then, for every ⌘ > 0 there are constants J and T
j
(!) such that for all j � J andT � T
j
(!),
| 1T
T
X
t=1
f(s
jt
(bs0,!))� E(f)| < ⌘
for all s0 and �-almost all !.
Therefore, for a sufficiently good numerical approximation bgj
and for a sufficientlylarge T the series { 1
T
P
T
t=1 f(bsjt(s0,!))} is close (almost surely) to the expected valueE(f) =
´f(s)µ
⇤(ds) of the invariant distribution µ
⇤ of the original equilibriumfunction g.
Note that this theorem does not require uniqueness of the invariant distributionfor each numerical policy function. This requirement would be rather restrictive:Numerical approximations may contain multiple steady states. For instance, con-sider a polynomial approximation of the policy function. As is well understood, thefluctuating behavior of polynomials may give raise to several ergodic sets. But ac-cording to the theorem, these multiple distributions from these approximations willeventually be close to the unique invariant distribution of the model. Moreover, if themodel has multiple invariant distributions, then there is an extension of Theorem 1in which the simulated statics of computed policy functions bg
j
become close to thoseof some invariant distribution of the model for j large enough [see op. cit.].
The existence of a invariant distribution is guaranteed under the so called Fellerproperty [cf. Stokey, Lucas and Prescott (1989)]. The Feller property is satisfied ifequilibrium function g is a continuous mapping on a compact domain or if the domainis made up of a finite number of points. (These latter stochastic processes are calledMarkov chains.) There are several extensions of these results to non-continuous
18
mappings and non-compact domains [cf. Futia (1982), Hopenhayn and Prescott(1992), and Stenflo (2001)]. These papers also establish conditions for uniquenessof the invariant distribution under mixing or contractive conditions. The followingcontraction property is taken from Stenflo (2001):
CONDITION C: There exists a constant 0 < � < 1 such that´kg(s, ")� g(s
0, ")kQ(d")
� ks� s
0k for all pairs s, s
0.
Condition C may arise naturally in growth models [Schenk-Hoppe and Schmalfuss(2001)], in learning models [Ellison and Fudenberg (1993)], and in certain types ofstochastic games [Sanghvi and Sobel (1976)].
Using Condition C, the following bounds for the approximation error of the sim-ulated moments are established in Santos and Peralta-Alva (2005). A real-valuedfunction f on S is called Lipschitz with constant L > 0 if |f(s)� f(s
0)| L ks� s
0kfor all pairs s and s
0.
Theorem 2 Let f be a Lipschitz function with constant L. Let d(g, bg) � for some� > 0. Assume that g satisfies Condition C. Then, for every ⌘ > 0 there exists afunction b
T (!) such that for all T � b
T (!),
| 1T
T
X
t=1
f(bs
t
(s0,!))� E(f)| L�
1� �
+ ⌘ (28)
for all s0 and �-almost all !.
Again, this is another application of the contraction property, which becomesinstrumental to establish error bounds.
3.3 Calibration, estimation and testing
As in other applied sciences, economic theories build upon the analysis of highlystylized models. The estimation and testing of these models can be quite challenging,and the literature is still in a process of early development in which various technicalproblems need to be overcome. Indeed, it should be stressed that there are importantclasses of models for which we still lack a good sense of the type of conditions underwhich simulation-based methods may yield estimators that achieve consistency andasymptotic normality. Moreover, computation of these estimators may turn out tobe a quite complex task.
19
A basic tenet of simulation-based estimation is that parameters are often specifiedas the by-product of some simplifying assumptions with no close empirical counter-parts. These parameter values will affect the equilibrium dynamics which can behighly non-linear; besides, statistical inference usually requires certain regularityconditions. Hence, as a first step in the process of estimation it seems reasonableto characterize the invariant probability measures or steady-state solutions, whichcommonly determine the long-run behavior of a model. But because of lack of infor-mation about the domain and form of these invariant probabilities, the model mustbe simulated to compute the moments and other useful statistics of these distribu-tions.
Therefore, the process of estimation may entail the simulation of a parameterizedfamily of models. Relatively fast algorithms are thus needed in order to sample theparameter space. Classical properties of these estimators such as consistency andasymptotic normality will depend on various conditions of the equilibrium functions.The study of these asymptotic properties requires methods of analysis of probabilitytheory in its interface with dynamical systems.
Our purpose here is to discuss some available methods for model estimationand testing. To make further progress in this discussion, let us rewrite (18) in thefollowing form
x
t+1 = g(✓, x
t
, z
t
, "
t+1) (29)
z
t+1 = (✓2, zt, "t+1), (30)
where ✓ = (✓1, ✓2) is a vector of parameters, and t = 0, 1, 2, · · · . Functions g and may represent the exact solution of a dynamic model or some numerical approxima-tion. One should realize that the assumptions underlying these functions may be ofa different economic significance, since g governs the law of motion of the vector ofendogenous variables x, and represents the evolution of the exogenous process z.Observe that the vector of parameters ✓2 characterizing the evolution of the exoge-nous state variables z may influence the law of motion of the endogenous variables x,but this endogenous process may also be influenced by some additional parameters✓1 which may stem from utility and production functions.
For a given notion of distance the estimation problem may be defined as follows:Find a parameter vector ✓
0= (✓
01, ✓
02) such that a selected set of model predictions
are closest to those of the data generating process. An estimator is thus a rule that
20
yields a sequence of candidate solutions b✓t
from finite samples of model simulationsand data. It is generally agreed that a reasonable estimator should possess thefollowing consistency property: As sampling errors vanish the sequence of estimatedvalues b
✓
t
should converge to the optimal solution ✓
0. Further, we would like theestimator to satisfy asymptotic normality so that it is possible to derive approximateconfidence intervals and address questions of efficiency.
Data-based estimators are usually quite effective, since they may involve lowcomputational cost. For instance, standard non-linear least squares (e.g., Jennrich,1969) and other generalized estimators (cf., Newey and McFadden, 1994) may beapplied whenever functions g and have analytical representations. Similarly, fromfunctions g and one can compute the likelihood function that posits a probabilitylaw for the process (x
t
, z
t
) with explicit dependence on the parameter vector ✓. Ingeneral, data-based estimation methods can be applied for closed-form representa-tions of the dynamic process of state variables and vector of parameters. This isparticularly restrictive for the law of motion of the endogenous state variables: Onlyunder rather especial circumstances one obtains a closed-form representation for thesolution of a non-linear dynamic model g.
Since a change in ✓ may feed into the dynamics of the system in rather com-plex ways, traditional (data-based) estimators may be of limited applicability fornon-linear dynamic models. Indeed, these estimators do not take into account theeffects of parameter changes in the equilibrium dynamics, and hence they can onlybe applied to full-fledged, structural dynamic models under fairly specific conditions.In traditional estimation there is only a unique distribution generated by the dataprocess, and such distribution is not influenced by the vector of parameters. For asimulation-based estimator, however, the following major analytical difficulty arises:Each vector of parameters is manifested in a different dynamical system. Hence,proofs of consistency of the estimator would have to cope with a continuous familyof invariant distributions defined over the parameter space.
An alternative route to the estimation of non-linear dynamic models is via the Eu-ler equations (e.g., see Hansen and Singleton, 1982) where the vector of parameters isdetermined by a set of orthogonality conditions conforming the first-order conditionsor Euler equations of the optimization problem. A main advantage of this approachis that one does not need to model the shock process or to know the functional de-pendence of the law of motion of the state variables on the vector of parameters,since the objective is to find the best fit for the Euler equations over available datasamples, within the admissible region of parameter values. The estimation of the
21
Euler equations can then be carried out by standard non-linear least squares or bysome other generalized estimator [Hansen (1982)]. However, model estimation via theEuler equations under traditional statistical methods is not always feasible. Thesemethods are only valid for convex optimization problems with interior solutions inwhich technically the decision variables outnumber the parameters; moreover, theobjective and feasibility constraints of the optimization problem must satisfy certainstrict separability conditions along with the process of exogenous shocks. Sometimesthe model may feature some latent variables or some private information which isnot observed by the econometrician (e.g., shocks to preferences); lack of knowledgeabout these components of the model may preclude the specification of the Eulerequations [e.g. Duffie and Singleton (1993)]. An even more fundamental limitationis that the estimation is confined to orthogonality conditions generated by the Eu-ler equations, whereas it may be of more economic relevance to estimate or test amodel along some other dimensions such as those including certain moments of theinvariant distributions or the process of convergence to such stationary solutions.
3.3.1 Calibration
Faced with these complex analytical problems, the economics literature has comeup with many simplifying approaches for model estimation. Starting with the realbusiness cycle literature [e.g., Cooley and Prescott (1995)], parameter values areoften determined from independent evidence or from other parts of the theory notrelated to the basic facts selected for testing. This is loosely referred as modelcalibration. Christiano and Eichembaum (1992) is a good example of this approach.They consider a business cycle model, and pin down parameter values from varioussteady-state conditions. Hence, the model is evaluated according to business cyclepredictions, and it is calibrated to replicate empirical properties of balanced growthpaths. Actually, Christiano and Eichembaum (1992) are able to provide standarderrors for their estimates, and hence their analysis goes beyond most calibrationexercises.
3.3.2 Simulation-based estimation
The aforementioned limitations of traditional estimation methods for non-linear sys-tems along with advances in computing have fostered the more recent use of es-timation and testing based upon simulations of the model. Estimation by modelsimulation offers more flexibility to evaluate the behavior of the model by computing
22
statistics of its invariant distributions that can be compared with their data coun-terparts. But this greater flexibility inherent in simulation-based estimators entailsa major computational cost: Extensive model simulations may be needed to samplethe entire parameter space. Relatively little is known about the family of models inwhich simulation-based estimators would have good asymptotic properties such asconsistency and normality. These properties would seem a minimal requirement fora rigorous application of estimation methods under the rather complex and delicatetechniques of numerical simulation in which approximation errors may propagate inunexpected ways.
To fix ideas, we will focus on a simulated moments estimator (SME) put forwardby Lee and Ingram (1991). This estimation method allows the researcher to assessthe behavior of the model along various dimensions. Indeed, the conditions charac-terizing the estimation process may involve some moments of the models invariantdistributions or some other features of the dynamics on which the desired vector ofparameters must be selected.
Several elements conform the SME. First, one specifies a target function or func-tion of interest which typically would characterize a selected set of moments of theinvariant distribution of the model and those of the data generating process. Second,a notion of distance is defined between the selected statistics of the model and itsdata counterparts. The minimum distance between these statistics is attained atsome vector of parameters ✓
0= (✓
01, ✓
02). Then, the estimation method yields a se-
quence of candidate solutions b✓t
over increasing finite samples of models simulationsand data so as to approximate the true value ✓
0.(a) The target function (or function of interest) f : S ! Rp is assumed to be
continuous. This function may represent p moments of an invariant distribution µ
✓
under ✓ defined as E✓
(f) =
´f(s)µ
✓
(ds) for s = (x, z). The expected value of f overthe invariant distribution of the data generating process will be denoted by E(f
dg
).(b) The distance function G : Rp ! Rp is assumed to be continuous. The
minimum distance is attained at a vector of parameter values
✓
0= arg inf G(E
✓
(f), E(f
dg
)) (31)
A typical specification of the distance function G is the following quadratic form:
G(E
✓
(f), E(f
dg
)) = (E
✓
(f), E(f
dg
)) ·W · (E✓
(f), E(f
dg
)), (32)
where W is a positive definite p⇥p matrix. Under certain standard assumptions (cf.,
23
Santos and Peralta-Alva 2005, Theorem 3.2) one can show there exists an optimalsolution ✓
0. Moreover, for the analysis below there is no restriction of generality toconsider that ✓0 is unique.
(c) An estimation rule characterized by a sequence of distance functions {GT
}N�1
and choices for the horizon {⌧T
}T�1 of the model’s simulations. This rule yields a
sequence of estimated values {b✓T
}T�1 from associated optimization problems with
finite samples of model’s simulations and data. The estimated value b
✓
T
(s0,!, s̃) isobtained as
b
✓
T
(s0,!, s̃) = arg inf
✓2⇥G
T
(
1
⌧
T
(!, s̃)⌃
⌧
T
(!,̃s)t=1 f(s
t
(s0,!, ✓)),1
T
⌃
T
t=1f(s̃t),!, s̃). (33)
We assume that the sequence of continuous functions {GT
(·, ·,!, s̃)}T�1 converges
uniformly to function G(·, ·) for ˜
�-almost all (!, s̃), and the sequence of functions{⌧
T
(!, s̃)}T�1 goes to 1 for ˜�-almost all (!, s̃). Note that both functions G
T
(·, ·,!, s̃)and ⌧
N
(!, s̃) are allowed to depend on ! and s̃, and ˜
� is a measure defined over !
and s̃. These functions will usually depend on all information available up to timeT. The rule ⌧
T
reflects that the length of model’s simulations may be different fromthat of data samples.
It should be stressed that problem (31) is defined over population characteristicsof the model and of the data generating process, whereas problem (33) is definedover statistics of finite simulations and data.Definition: The SME is a sequence of measurable functions {b✓
T
(s0,!, s̃)}T�1 suchthat each function b
✓
T
satisfies (33) for all s0 and ˜
�-almost all (!, s̃).By the measurable selection theorem [Crauel (2002)] there exists a sequence of
measurable functions {b✓T
}T�1. See Duffie and Singleton (1993) and Santos (2010)
for asymptotic properties of this estimator. Sometimes vector ✓2 could be estimatedindependently, and hence we could then try to get an SME estimate of ✓1. Thismixed procedure can still recover consistency and it may save on computational cost.Consistency of the estimator can also be established for numerical approximations:The SME would converge to the true value ✓
0 as the approximation error goes tozero.
Another route to estimation is via the likelihood function. The existence ofsuch functions imposes certain regularity conditions on the dynamics of the modelwhich are sometimes hard to check. Fernandez-Villaverde and Rubio-Ramirez (2007)propose computation of the likelihood function by a particle filter. Numerical errorsof the computed solution will also affect the likelihood function and the estimated
24
parameter values [see Fernandez-Villaverde, Rubio-Ramirez and Santos (2006)].
4 Recursive Methods for Non-optimal Economies
We now get into the more complex issue of numerical simulation of non-optimaleconomies. In general, these models cannot be computed by associated global opti-mization problems—ruling out the application of numerical DP algorithms as wellas the derivation of error bounds for strongly concave optimization problems. Thisleaves the field open for algorithms based on approximating the Euler equationssuch as perturbation and projection methods. These approximation methods, how-ever, search for smooth equilibrium functions; as already pointed out, existence ofcontinuous Markov equilibria cannot be insured under regularity assumptions. Theexistence problem is a technical issue which is mostly ignored in the applied liter-ature. See Hellwig (1983) and Kydland and Prescott (1980) for early discussionson non-existence of simple Markov equilibrium, and Abreu, Pierce and Stacchetti(1990) for a related approach to repeated games.
As it is clear from these early contributions, simple Markov equilibrium may onlyfail to exist in the presence of multiple equilibria. Then, to insure uniqueness ofequilibrium the literature has considered a stronger related condition: Monotonicityof equilibrium. This monotonicity condition means that if the values of our pre-determined state variables are increased today, then the resulting equilibrium pathmust always reflect higher values for these variables in the future. Monotonicity ishard to verify in models with heterogeneous agents with constraints that occasionallybind, or in models with incomplete financial markets, or with distorting taxes andexternalities.
Indeed, most well known cases of monotone dynamics have been confined toone-dimensional models. For instance, Coleman (1991), Greenwood and Huffman(1995) and Datta, Mirman and Reffett (2002) consider versions of the one-sectorneoclassical growth model and establish existence of a simple Markov equilibrium byan Euler iteration method. This iterative method guarantees uniform convergence,but it does not display the contraction property as the DP algorithm. It is unclearhow this approach may be extended to other models, and several examples havebeen found of non-existence of continuous simple Markov equilibria [cf. Kubler andSchmedders (2002), Kubler and Polemarchakis (2004) and Santos (2002)].
Therefore, for non-optimal economies a recursive representation of equilibria mayonly be possible when conditioning over an expanded set of state variables. Follow-
25
ing Duffie et al. (1994), the existence of a Markov equilibrium in a generalized spaceof variables is proved in Kubler and Schmedders (2003) for an asset pricing modelwith collateral constraints. Feng et al. (2012) extend these existence results to othereconomies, and define a Markov equilibrium as a solution over an expanded state ofvariables that includes the shadow values of investment. The addition of the shadowvalues of investment as state variables facilitates computation of the numerical so-lution. This formulation was originally proposed by Kydland and Prescott (1980),and later used in Marcet and Marimon (1998) for recursive contracts, and in Phe-lan and Stacchetti (2001) for a competitive economy with a representative agent.The main insight of Feng et al. (2012) is to develop a reliable and computable al-gorithm for the numerical simulation of competitive economies with heterogeneousagents and market frictions including endogenous borrowing constraints, and studyits approximation properties. Before advancing to the study of the theoretical issuesinvolved, we begin with a few examples to illustrate some of the pitfalls found in thecomputation of non-optimal economies.
4.1 Problems in the simulation of non-optimal economies
The following examples make clear that a continuous Markov equilibrium on theminimal state space may fail to exist. Hence, application of standard numerical al-gorithms may actually result in serious quantitative biases. Therefore, other familiesof algorithms are needed for the numerical approximation of non-optimal economies.
4.1.1 A growth model with taxes
Consider the following parameterization for the growth model with taxes of Section2:
f(K,L) = K
1/3, � = 0.95, � = 1,� = 0.
Assume that income taxes are only imposed on households capital income. Morespecifically, this form of taxation is determined by the following piecewise linearschedule:
⌧
h
(K) =
8
>
<
>
:
0.10 if K 0.160002
0.05� 10(K � 0.165002) if 0.160002 K 0.170002
0 if K � 0.170002.
26
Santos (2002, Prop. 3.4) shows that a continuous Markov equilibrium fails to exist.For this specification of the model, there are three steady states: The middle steadystate is unstable and has two complex eigenvalues while the other two steady statesare saddle-path stable; see Figure 1. Standard algorithms approximating the Eulerequation would solve for a continuous policy function of the form
k
t+1 = g(k
t
, ⇠),
where g belongs to a finite dimensional space of continuous functions as defined by avector of parameters ⇠. We obtain an estimate for ⇠ by forming a discrete system ofEuler equations over as many grid points k
i as the dimensionality of the parameterspace:
u
0(k
i
, g(k
i
, ⇠)) = �u
0(g(k
i
, ⇠), g(g(k
i
, ⇠), ⇠)) ·⇥
f
0(g(k
i
, ⇠))(1� ⌧(g(k
i
, ⇠)))
⇤
.
We assume that g(k
i
, ⇠) belongs to the class of piecewise linear functions, and em-ploy a uniform grid of 5000 points over the domain k 2 [0.14..0.19]. The resultingapproximation, together with a highly accurate solution (in this case the shootingalgorithm can be implemented) are illustrated in Figure 1.
This approximation of the Euler equation over piecewise continuous functionsconverged up to computer precision in only 3 iterations. This fast convergence is ac-tually deceptive because as pointed out above no continuous policy function does ex-ist. Indeed, the dynamic behavior implied by the continuous function approximationis quite different from the true one. As a matter of fact, the numerical approximationdisplays four more steady states, and changes substantially the basins of attractionof the original steady states (see Figure 1).
A further test of the fixed-point solution of this algorithm based on the Eulerequation residuals produced mixed results (see Figure 2). First, the average Eulerequation residual over the domain of feasible capitals is fairly small, i.e. it is equal to0.0073. Second, the maximum Euler equation residual is slightly more pronouncedin a small area near the unstable steady state. But even in that area, the erroris not extremely large: In three tiny intervals the Euler equation residuals are justaround 0.06. Therefore, from these computational tests a researcher may be led toconclude that the putative continuous solution should mimic well the true equilibriumdynamics.
27
4.1.2 An overlapping generations economy
Consider the following specification for the overlapping generations economy pre-sented in Section 2. There are two perishable commodities, and two types of agentsthat live for two periods. There is no Lucas tree. In the first period of life ofeach agent, endowments are stochastic and depend only on the current state z
t
,while in the second period they are deterministic. In particular, e
1,zt
1 (z
t
) = 10.4,e
2,zt
1 (z
t
) = 2.6 if zt
= z1, and e
1,zt
1 (z
t
) = 8.6313, e2,zt
1 (z
t
) = 4.3687 if zt
= z2, whilee
1,zt(z
t+1) = (12, 1) and e
2,zt(z
t+1) = (1, 12) .
The utility function of an agent of type 1 is given by
� 1024
(c
11 (z
t
))
4 + E
z
t+1|zt
� 1024
(c
11 (z
t+1))
4 � 1
(c
12 (z
t+1))
�
,
while that of agent of type 2 is given by
� 1
(c
21 (z
t
))
4 + E
z
t+1|zt
� 1
(c
21 (z
t+1))
4 � 1024
(c
22 (z
t+1))
�
.
For this model, it is easy to show that a competitive equilibrium exists. Practi-tioners are, however, interested competitive equilibria that have a recursive structureon the space of shocks and wealth distributions. Specifically, standard computationalmethods search for a Markovian equilibrium on the natural state space. Hence, let usconsider that there exists a continuous function f such that equilibrium allocationscan be characterized by:
(✓
b,1,zt(z
t+1), q
b
(z
t
), p(z
t
), (c
i,z
⌧
j
)
i=1,2,j=1,2,⌧=t,t+1) = f
⇣
✓
b,1,zt(z
t
), z
t
⌘
.
Kubler and Polemarchakis (2004) show that such a representation does not exist forthis economy. Specifically, the unique equilibrium of this economy is described by:
1. ✓
b,1(z
t
) = 0 at all zt.
2. Given node z
t�1 with z
t�1 = z1, we have that for all successors of z
t�1,
namely z
t
= z
t�1(z1) and z
t
= z
t�1(z2) :
⇣
c
1,zt�1
1 (z
t
) , c
1,zt�1
2 (z
t
)
⌘
= (10.4, 2.6),⇣
c
2,zt�1
1 (z
t
) , c
2,zt�1
2 (z
t
)
⌘
= (2.6, 10.4), and p = 1.
28
3. Given node z
t�1 with z
t�1 = z2, we have that for all successors of z
t�1,
namely z
t
= z
t�1(z1) and z
t
= z
t�1(z2) :
⇣
c
1,zt�1
1 (z
t
) , c
1,zt�1
2 (z
t
)
⌘
= (8.4, 1.4),⇣
c
1,zt�1
1 (z
t
) , c
1,zt�1
2 (z
t
)
⌘
= (4.6, 11.6) , and p = 7.9.
Observe that knowledge of the current shock and wealth distribution are not enoughto characterize consumption of the old.
As in our previous example, and in spite of knowing that a recursive equilibriumon the natural state space does not exist, we applied the projection method to obtaina numerical approximation to function f. We employed a grid of 100 equally spacedpoints under piecewise-linear interpolation, and assumed ✓ 2 [�0.2, 0.2]. Based onthis approach, we ended up with an approximated policy function with Euler equationresiduals of order 10
�5 (on average). We again find that the time series propertiesof the approximated policy may be substantially different from equilibrium. As afirst illustration of this consider Figures 3 and 4, which summarize portfolio holdingsand the relative price of good two, respectively. In equilibrium bond holdings shouldequal zero, while the approximate policy yields positive values. Similarly, the relativeprice of good 2 should equal either 1 or 7.9, depending on the shock, while it takesa continuum of values ranging from 6.5 to 9 in the approximate policy. To furtherillustrate the differences between approximate and exact solutions, Table 1 reportssimulated sample for the exact and approximate solutions over the same sequence ofshocks in a sample path of 10,000 periods.
meantrue(✓) meanf (✓) meantrue(p) meanf (p)0.0 0.2 4.4 7.3
Table 1: Simulated moments – f refers to the approximate policy f .
In summary, for non-optimal economies standard solution methods may introducesubstantial biases into our quantitative predictions.
4.2 Numerical solution of non-optimal economies
Feng et. al. (2012) develop a numerical algorithm for approximating equilibriumsolutions of non-optimal economies. A recursive representation for equilibria is es-tablished on a state space conformed by the standard variables, (x, z), and the vectorof shadow values of the marginal return to investment for all assets and all agents, m.
This algorithm is guaranteed to converge and has desirable asymptotic properties.
29
4.2.1 The theoretical algorithm
A fundamental element of this approach is operator B. An iterative procedure basedon this operator converges to the equilibrium correspondence V
⇤(x, z) . This equi-
librium correspondence is defined as the set of possible equilibrium values for m,
given (x, z) . As illustrated presently, once the equilibrium correspondence has beensecured, we can provide a recursive representation of equilibria on the enlarged state(x, z,m).
Let z be any initial node, and z+ be the set of immediate successor states. Pickany correspondence V : X ⇥ Z ! M, where M is the set of possible shadow valuesof investment. Then, for each (x, z) , we define operator B (V ) (x, z) as the set ofall values m with the property that there are current endogenous variables y, andvectors x+(z+) and m+ (z+) 2 V (x+, z+) for each of the successors of z, denoted byz+, that satisfy the temporary equilibrium conditions
�(x, y, z, E [m+ (z+)]) = 0
' (x+, x, y, z) = 0.
The following result is proved in Feng et al. (2012):
Theorem 3 (convergence) Let V0 be a compact-valued correspondence such thatV0 � V
⇤. Let V
n
= B (V
n�1) , n � 1. Then, Vn
! V
⇤ as n ! 1. Moreover, V
⇤
is the largest fixed point of operator B; that is, if V = B(V ), then V ⇢ V
⇤.
Theorem 3 provides the theoretical foundations for computing equilibria for non-optimal economies. Specifically, this result states that operator B can be appliedto any initial guess (correspondence) of possible values V0 (x, z) � V
⇤(x, z) and
iterate until a desirable level of convergence to V
⇤ is attained. From operator B :
graph(V
⇤) ! graph(V
⇤) we can select a measurable policy function y = g
y
(x, z,m),and a transition function m+ (z+) = g
m
(x, z,m; z+), for all z+ 2 Z. These functionsmay not be continuous but the state space has been adequately chosen so thatthey yield a Markovian characterization of a dynamic equilibrium in the enlargedstate space (x, z,m). An important advantage of this approach is that if multipleequilibria exist then all of them can be computed. If the equilibrium is always unique,then B : graph(V
⇤) ! graph(V
⇤) defines a continuous law of motion or Markovian
equilibrium over state variables (x, z).
30
4.2.2 Numerical implementation
We first partition the state space into a finite set of simplices {Xj} with non-emptyinterior and maximum diameter h. Over this partition define a family of step corre-spondences (defined as correspondences that take constant set values over each X
j).To obtain a computer representation of a step correspondence, the image must bediscretized. We can employ an outer approximation in which each set-value is definedby N elements. Using these two discretizations we obtain a computable approxima-tion of operator B, which we denote by B
h,N . By a suitable selection of an initialcondition V0 and of these outer approximations, the sequence {V h,N
n+1 } defined recur-sively as V
h,N
n+1 = B
h,N
V
h,N
n
converges to a limit point V
⇤,h,N, which must contain
the equilibrium correspondence V
⇤. Again, if the equilibrium is always unique then
these approximate solutions would converge uniformly to the continuous Markovianequilibrium law of motion. The following result is proved in Feng et al. (2012):
Theorem 4 (accuracy) For given h, N, and initial condition V0 ◆ V
⇤, consider therecursive sequence {V h,N
n+1 } defined as V
h,N
n+1 = B
h,N
V
h,N
n
. Then, (i) V
h,N
n
◆ V
⇤ forall n; (ii) V
h,N
n
! V
⇤,h,N uniformly as n ! 1; and (iii) V ⇤,h,N ! V
⇤ as h ! 0
and N ! 1.
4.3 Simulated statistics
To assess model predictions, analysts usually calculate moments of the simulatedpaths from a numerical approximation. The idea is that the simulated momentsshould approach those obtained from the original model. As discussed in Section 3, ifthe optimal policy is a continuous function, or if certain monotonicity conditions hold,it is possible to establish desirable convergence properties of the simulated moments.For non-optimal economies, continuity or monotonicity of Markov equilibria do notcome out so naturally. In those models the equilibrium law of motion is described byan expectations correspondence conformed by feasibility and short-run equilibriumconditions. Hence, for an initial vector of state variables there could be multiplecontinuation equilibrium paths, and coordination over these multiple equilibria maybe required.
More precisely, for non-optimal models the dynamics may by characterized by
s
n+1 2 ⌥(sn, "n+1), n = 0, 1, 2, · · · ,
where ⌥ : S ⇥ E ! S is an upper semicontinuous correspondence (instead of a
31
continuous function as in the previous section) over a compact domain. By themeasurable selection theorem [e.g., Crauel (2002) and Hildenbrand (1974)] thereexists a sequence of measurable mappings {b⌥
j
}, b⌥j
: S⇥E ! S, such that ⌥(s, ") =cl{b⌥
j
(s, ")} for all (s, ") and all j (cl denoting closure). Let us pick a measurableselection b
⌥ 2 ⌥. Then, we can define a transition probability Pb⌥(s, A) by
Pb⌥(s, A) = ⌫({"|b⌥(s, ") 2 A}). (2.2)
Note that Pb⌥(s, ·) is a probability measure for each s 2 S, and Pb⌥(·, A) is a measur-able function for each A in S.
Finally, given an initial probability µ0 on S, the evolution of future probabili-ties, {µ
n
}, can be specified by the following operator T
⇤b⌥ that takes the space of
probabilities on S into itself
µ
n+1(A) = (T
⇤b⌥µn
)(A) =
ˆPb⌥(s, A)µn
(ds),
for all A in S and n � 0. An invariant probability measure or invariant distributionµ
⇤ is a fixed point of operator T
⇤b⌥, i.e., µ⇤
= T
⇤b⌥µ
⇤. Measure µ
⇤ is called ergodic ifµ
⇤(A) = 0 or µ
⇤(A) = 1 for every invariant set A under transition probability Pb⌥.
To guarantee existence of an ergodic measure some researchers have resorted toa discretization of the state space [Ericson and Pakes (1995)]. Discrete state spacesare quite convenient to compute the set of invariant measures, but these spacesbecome awkward for the characterization of optimal solutions and the calibrationand estimation of the model. If the state takes a continuum of values then there aretwo basic ways to establish existence of an invariant measure [e.g., Crauel (2002)]:(i) Via the Markov-Kakutani fixed-point theorem: An upper semicontinuous convex-valued correspondence in a compact set has a fixed point; and (ii) Via a Krylov-Bogolyubov type argument: The invariant measure is constructed by an iterativeprocess as limit of a sequence of empirical probability measures or time means. Blume(1982) and Duffie et al. (1994) follow (i), and are required to randomize over theexisting equilibria to build a convex-valued correspondence. Randomizing over theequilibrium correspondence may result in an undesirable expansion of the equilibriumset.
Recent work by Peralta-Alva and Santos (2012) follows (ii) and dispenses withrandomizations. They also validate a generalized law of large numbers that guaran-tees convergence of the simulated moments to the population moments of some sta-
32
tionary equilibrium. These results apply naturally to approximate solutions. Hence,the simulated moments from a numerical solution approach asymptotically someinvariant distribution of the numerical approximation. Finally, combining these ar-guments with some convergence results, they establish some accuracy properties forthe simulated moments as the approximation error goes to zero. We summarize theseresults as follows:
(i) Existence of an invariant distribution for the original model : Transitioncorrespondence P⌥(s, ·) has an invariant probability µ
⇤; this invariant distribution is
constructed as a limit of a sequence of empirical measures using a Krylov-Bogolyubovtype argument. This iterative process is extended to stochastic dynamical systemsdescribed by correspondences, and it works when the space of measures is compactand the equilibrium correspondence is upper semicontinuous.
(ii) Simulation of the computed equilibrium laws of motion y = g
y,h,N
n
(x, z,m),and m+ (z+) = g
m,h,N
n
(x, z,m; z+). There are tight upper USM and lower LSM
bounds such that with probability one the corresponding moments from simulatedpaths (x
t
(z
t
), y
t
(z
t
))
1t=0 of these approximate functions stay within the prescribed
bounds. More precisely, let s = (x, y,m) and f : S ! R+ be a function of in-terest. Let
⇣
P
T
t=0 f(st)
⌘
/T represent a simulated moment or some other statistic.
Then, with probability one, every limit point of⇣
P
T
t=0 f(st)
⌘
/T must be within thecorresponding bounds LSM and USM .
(iii) Accuracy of the simulated moments: For every ✏ > 0 we can consider asufficiently good discretized operator Bh,N and equilibrium correspondence V h,N
n
suchthat for every simulated path (s
t
, z
t
)
1t=0 there are equilibrium invariant distributions
µ
⇤, µ
0⇤ satisfying´f(s)dµ
⇤ � ✏ ⇣
P
T
t=0 f(st)
⌘
/T ´f(s)dµ
0⇤+ ✏ almost surely.
Of course, the model has a unique invariant distribution µ
⇤ then µ
0⇤= µ
⇤ and theabove expression reads as
´f(s)dµ
⇤ � ✏ ⇣
P
T
t=0 f(st)
⌘
/T ´f(s)dµ
⇤+ ✏.
The primitive elements in (i� iii) are the Markovian equilibrium functions whichare obtained from the original equilibrium correspondences without performing ar-bitrary randomizations.
5 Numerical Experiments
In this section we consider some further examples to illustrate the workings of somealgorithms and the accuracy of numerical approximations. There is a vast literaturedevoted to the construction of algorithms computing simple Markov equilibria. Wewill show how the approximation error can be estimated from the Euler equation
33
residuals. We also consider certain specifications for our model economies with mul-tiple Markov equilibria — or where a Markov equilibrium is not known to exist.In these latter cases the application of algorithms searching for continuous policyfunctions may lead to rather unsatisfactory results.
5.1 Accuracy for models with simple Markov equilibria
We now consider a specification for the growth model of Section 2 with no taxation.We allow for a CES Bernoulli utility function:
E0
1X
t=0
�
t
(c
✓
t
(1� l
t
)
1�✓
)
1��
1� �
.
The production technology will be assumed Cobb-Douglas so that total output isthus given by e
z
t
K
↵
t
L
1�↵
t
. In our computations the shock process is set so as toapproximate an underlying continuum law of motion z
t
= ⇢z
t�1 + ✏
t
, with ✏
t
⇠N(0, �
2✏
).
Aruoba et al. (2006) provide a thorough examination of the properties of alterna-tive approximation schemes for the solution of this model. We follow their approachand study the accuracy of approximations employing their basic parameterizations.Let us start with their benchmark case:
� = 0.9896, ✓ = 0.357,↵ = 0.4, � = 0.0196, ⇢ = 0.95, � = 2.0, �
✏
= 0.007.
Once we have secured the best possible numerical approximation g
acc
, we canprovide estimates for the approximation error as described in Section 3. The keyelement of this approach requires values for
M
NUM
bg =
||bg � g
acc
||||EE(bg)|| ,
where bg is any other coarser numerical approximation, and EE(bg) is the maximumEuler residual under policy bg.
We follow Aruoba et al. (2006) and derive numerical approximations for themodel under various approximations, including the policy with the smallest Eulerequation residuals (g
acc
), under the DP approximation, and other faster methods (toobtain alternative bg) such as linear approximations, perturbations (of orders 2 and5) and projections. We take the highest value for MNUM
bg over all approximations as
34
our estimate M
NUM for the constant required in the error estimates of Section 3:
||g � g
acc
|| = M
NUM ||EE(g
acc
)||.
Our accuracy estimates for the baseline specification of the model and for somealternative parameterizations are summarized in Table 2 below. All errors are es-timated for an interval of the deterministic steady state comprising ±30% of thesteady-state value.
Parameterization M
NUM ||EE(g
acc
)||Baseline 52.3 3.32⇥ 10
�7
� = 50, �
✏
= 0.035 40.5 4.42⇥ 10
�6
� = 0.95 20.1 2.89⇥ 10
�7
Table 2: Accuracy estimates. Parameterizations are only indicated for deviationsfrom baseline values.
Hence, Aruoba et al. (2006) provide Euler equation residuals of the order of 10�7.
Our exercise shows that these residuals translate into approximation errors for thepolicy function of the order of 10�5
, since the constants M involved in these errorestimates are always below 100.
5.2 Simulation of non-optimal economies
5.2.1 An overlapping generations model
We now rewrite the OLG economy of Section 2 along the lines of the classical mon-etary models of Benhabib and Day (1982) and Grandmont (1985). This version ofthe model is useful for illustrative purposes because it can be solved with arbitraryaccuracy. Hence, we can compare the true solution of the model with alternativenumerical approximations. The model is deterministic. There are two agents thatlive for two periods (except for the initially old agent, who only lives for one period).Each individual receives an endowment e1 of the perishable good when young ande2 when old. There is a single asset, money, that pays zero dividends at each givenperiod. The initial old agent is endowed with the existing money supply M. Let P
t
be the price level at time t. An agent born in period t solves:
maxu (c1t) + �v (c2t+1)
35
subject to
c1t +M
t
P
t
= e1,
c2t+1 = e2 +M
t
P
t+1.
Equilibria can be characterized by the following first-order condition:
1
P
t
u
0✓
e1 �M
P
t
◆
=
1
P
t+1�v
0✓
e2 +M
P
t+1
◆
.
Let b
t
= M/P
t
be real money balances at t. Then,
b
t
u
0(e1 � b
t
) = b
t+1�v0(e2 + b
t+1) .
It follows that all competitive equilibria can be generated by an offer curve in the(b
t
, b
t+1) space. A simple recursive equilibrium would be described by a functionb
t+1 = g (b
t
) . We focus on the following parameterization:
u (c) = c
0.45, v (c) = �1
7
c
�7, � = 0.8, M = 1, e1 = 2, e2 = 2
6/7 � 2
1/7.
In this case, the offer curve is backward bending (see Figure 5). Hence, the equi-librium correspondence is multi-valued. Therefore, standard methods – based onthe computation of a continuous equilibrium function b
t+1 = g (b
t
) – may portray apartial view of the equilibrium dynamics. There is a unique stationary solution atabout b⇤ = 0.4181, which is the point of crossing of the offer curve with the 45-degreeline.Comparison with other computational algorithms
A common practice in OLG models is to search for an equilibrium guess functionb
0= bg(b), and then iterate over the temporary equilibrium conditions. We applied
this procedure to our model. Depending on the initial guess, we find that either theupper or the lower arm of the offer curve would emerge as a fixed point. This strongdependence on initial conditions is a rather undesirable feature of this computationalmethod. In particular, if we only consider the lower arm of the actual equilibrium cor-respondence then all competitive equilibria converge to autarchy. Indeed, the uniqueabsorbing steady state associated with the lower arm of the equilibrium correspon-dence involves zero monetary holdings. Hence, even in the deterministic version,we need a global approximation of the equilibrium correspondence to analyze the
36
various predictions of the model. As shown in Figure 6, the approximate equilibriumcorrespondence has a cyclical equilibrium in which real money holdings oscillate be-tween 0.8529 and 0.0953. It is also known that the model has a three-period cycle.But if we iterate over the upper arm of the offer curve, we find that money holdingsconverge monotonically to M̄
p
= 0.4181 (as illustrated by the dashed lines of Figure6). As a matter of fact, the upper arm is monotonic, and can at most have cycles ofperiod two, whereas the model generates equilibrium cycles of various periodicities.
In conclusion, for OLG economies, standard computational methods based oniteration of continuous functions do not guarantee convergence to an equilibriumsolution, and may miss some important properties of the equilibrium dynamics. Inthese economies it seems pertinent to compute the set of all sequential competitiveequilibria. It is certainly an easy task to compute this simple model by the algorithmof Section 4 of Feng et al. (2012). We presently illustrate the workings of this reliablealgorithm in a stochastic economy with two types of agents.
5.2.2 Asset pricing models with market frictions
An important family of macroeconomic models incorporates financial frictions in theform of sequentially incomplete markets, borrowing constraints, transactions costs,cash-in-advance constraints, and margin and collateral requirements. Fairly generalconditions rule out the existence of financial bubbles in these economies; hence,equilibrium asset prices are determined by the expected value of future dividends[Santos and Woodford (1997)]. There is, however, no reliable algorithm for thenumerical approximation and simulation of these economies. Here, we illustratethe workings of our algorithm in the economy of Kehoe and Levine (2001). Theseauthors provide a characterization of steady-state equilibria for an economy withidiosyncratic risk under exogenous and endogenous borrowing constraints.
The basic economic environment stems from the asset pricing model of Section2. There are two possible values for the endowment, high, e
h
, or low, e
l
. Thereis no aggregate risk: One household gets the high endowment whilst the other onegets the low endowment at every date. There is only one asset, a Lucas tree witha constant dividend, d. Households maximize expected utility (7) subject to thesequence of budget constraints (8). We now consider an important departure fromthe basic model of Section 2: endogenous credit limits. More specifically, allocations
37
(and the implied borrowing) must satisfy the participation constraint
E
z
t
1X
⌧=t
�
⌧
u
i
�
c
i
⌧
�
� V
i,aut
(z
t
), for all i and z
t
. (34)
Here, V i,aut
(z
t
) denotes the expected discounted value of making consumption equalto the endowment from period t onwards. This is the payoff of defaulting on creditobligations. The algorithm of Section 4 can be readily modified to accommodate thistype of constraints. It simply requires iterating simultaneously on pairs of candidateshadow values of investment and values for participation (the lifetime utility of neverdefaulting). This operator is monotone (in the set inclusion sense) and thus theapproximation results of Section 4 still hold [see Feng, et al. (2012)].The equilibrium correspondence
Note that market clearing for shares requires ✓
1= 1 � ✓
2. Hence, in the sequelwe let ✓ be the share holdings of household 1, and e
s
be the endowment of household1, for s = l, h. Then, the equilibrium correspondence V
⇤(✓, e
s
) is a map from thespace of possible values for share holdings and endowments for agent 1 into the setof possible equilibrium shadow values of investment for each agent (m
1,m
2).
The FOCs of the household’s problem areqDu
i
(e
i
+ ✓
i
(d+ q)� ✓
i · q) = �
i
�
i
⇡[e
i
+|ei]mi
+.
Asset holdings and prices are state contingent and thus both ✓, q are vectors in R2.
Observe that �
i � 1 is a ratio of multipliers corresponding to the participation con-straints. That is, �i
=
1+µ
i+µ
i
+
1+µ
i
, where µ
i � 0 is a multiplier associated with today’sparticipation constraint, and µ
i
+ � 0 is a multiplier associated with tomorrow’s par-ticipation constraint at state ei+|ei. Therefore, �i
> 1 only if tomorrow’s participationconstraint is binding.Computational algorithm
We start with a correspondence V0 such that V0(✓, es) ◆ V
⇤(✓, e
s
) for all (✓, es
)
with s = l, h. It is easy to come up with the initial candidate V0, since the lowendowment e
l
is a lower bound for consumption, and the marginal utility of con-sumption can be used to bound asset prices as discounted values of dividends. It isalso straightforward to derive bounds for the value of participation P0.
Iterations of operator B result in new candidate values for the shadow values ofinvestment, and new candidate values for participation. Specifically, given (✓, e
s
),(m
1,m
2) 2 V
n
(✓, e
s
), and (p
1, p
2) 2 P
n
(✓, e
s
,m
1,m
2) we have that (m
1,m
2) 2
V
n+1(✓, es), and (p
1, p
2) 2 P
n+1(✓, es,m1,m
2) iff we can find portfolio holdings for
38
next period, ✓+, a bond price q, multipliers (�1,�
2), continuation shadow values of in-
vestment (m1+,m
2+) 2 V
n
(✓+, es+), and continuation utilities (p1+, p2+) 2 P
n
(✓+, es+ ,m1+,m
2+)
such that the individual’s intertemporal optimality conditions are satisfied, and areconsistent with the definition of promised utilities and with participation constraints
p
i
= u(c
i
) + �Ep
i
+
p
i � V
i,aut
(e
s
).
Our algorithm can then be used to generate a sequence of approximations to theequilibrium correspondence via the recursion (V
n+1, Pn+1) = B(V
n
, P
n
).Table 3 reports sample statistics for equilibrium time series. In this table, q refers
to the price of a state uncontingent share.
Model mean(q) std(q) mean(c1) stdev(c1)Endogenous constraint 1.07 0.00 17.00 4.52
Table 3: Simulated moments – mean and standard deviation (stdev).
Perfect risk sharing would require constant consumption across states. The en-dogenous participation constraint prevents perfect risk sharing and so consumptiondisplays some volatility. Since the unique equilibrium is a symmetric stochasticsteady state and the agent with the good shock (who is unconstrained) determinesthe price of the asset, the price of a state uncontingent share is constant. As is wellunderstood, however, the volatility of the pricing kernel of this economy is higherthan that of a complete markets economy but we do not report state contingentprices.
6 Concluding Remarks
In this paper we present a systematic approach for the numerical simulation of dy-namic economic models. There is a fairly complete theory for the simulation ofoptimal economies, and a variety of algorithms are available for the computation ofthese economies. The dynamic programming (DP) algorithm guarantees convergenceto the true solution, and the approximation error can be bounded. There are otheralgorithms for which a numerical solution is not known to exist — or convergenceto the numerical solution cannot be guaranteed. These algorithms are usually much
39
faster than the DP algorithm, and easier to implement. We have presented an ac-curacy test based on the Euler equation residuals which is particularly relevant fornon-reliable algorithms. This test can estimate the accuracy of the computed solu-tion from a plot of the residuals without further reference to the particular algorithmcomputing the solution.
Of course, in dynamic models the one-period error estimated by the Euler equa-tion residuals may cumulate over time. We then develop some approximation prop-erties for the simulated moments and the consistency of the simulation-based estima-tors. Error bounds and asymptotic normality of these estimators may require furtherdifferentiability properties of the invariant distributions of the original model.
For non-optimal economies, a continuous Markov equilibrium may not exist.Hence, algorithms searching for a continuous policy function are usually not ade-quate. Indeed, we discussed some examples is which standard algorithms producedmisleading results. We analyzed a reliable algorithm based on the computation ofcorrespondences rather than functions. We also studied some convergence propertiesof the numerical solutions. Still, for non-optimal economies there are many open is-sues such as bounding approximation errors and the estimation of parameter valuesby simulation-based estimators.
We have focused on the theoretical foundations of numerical simulation ratherthan on a thorough description of the type of economic models to which this theorycan be applied. There are certain models that clearly fall outside the scope of ourapplications – even though the theoretical results presented here may still offer someuseful insights. For instance, see Algan et al. (2010) for algorithms related to thecomputation of models with a continuum of agents of the type of Krusell and Smith(1998), and Ericson and Pakes (1995) for the computation of a model of an industry.
REFERENCES
Abreu, D., D. Pierce and E. Stacchetti (1990): “Toward a theory of repeatedgames with discounting,” Econometrica, 58, 1041-1063.
Algan, Y., O. Allais, W. J. Den Haan, and P. Rendahl (2010): “Solvingand simulating models with heterogeneous agents and aggregate uncertainty,”manuscript.
Aruoba, S. B., J. Fernandez-Villaverde and J. Rubio-Ramirez (2006):“Comparing solution methods for dynamic equilibrium economies,” Journal of
40
Economic Dynamics and Control, 30, 2477-2508.
Benhabib, J. and R. H. Day (1982): “A characterization of erratic dynamicsin the overlapping generations model,” Journal of Economic Dynamics andControl, 4, 37-55.
Blume, L. E. (1982): “New techniques for the study of stochastic equilibrium pro-cesses,” Journal of Mathematical Economics, 9, 61-70.
Cooley, T. F. and E. C. Prescott (1995): “Economic growth and businesscycles.” Frontiers of Business Cycle Research. Ed., T. F. Cooley, NJ: PrincetonUniversity Press.
Christiano, L. J. and M. Eichembaum (1992): “Current business cycle theoriesand aggregate labor-market fluctuations,” American Economic Review, 82, 430-450.
Christiano, L. J. and J. D. M. Fisher (2000): “Algorithms for solving dynamicmodels with occasionally binding constraints,” Journal of Economic Dynamicsand Control, 24, 1179-1235.
Coleman, W. J. (1991): “Equilibrium in a production economy with an incometax,” Econometrica, 59, 1091-1104.
Crauel, H. (2002), Random Probability Measures on Polish Spaces. StochasticMonographs 11, London: Taylor and Francis.
Datta, M., L. J. Mirman and K. L. Reffett (2002): “Existence and uniquenessof equilibrium in distorted dynamic economies with capital and labor,” Journalof Economic Theory, 103, 377-410.
Den, Haan, W. J. and A. Marcet (1993): “Accuracy in simulations,” Reviewof Economic Studies, 61, 3-17.
Duffie, D., J. Geanakoplos, A. Mas-Colell and A. McLennan (1994):“Stationary Markov equilibria,” Econometrica, 62, 745-781.
Duffie, D. and K. J. Singleton (1993): “Simulated moments estimation ofMarkov models of asset prices,” Econometrica, 61, 929-952.
Ellison, G. and D. Fudenberg (1993): “Rules of thumb for social learning,”Journal of Political Economy, 101, 612-643.
41
Ericson, R. and A. Pakes (1995): “Markov-perfect industry dynamics: A frame-work for empirical work,” Review of Economic Studies, 82, 53-82.
Feng, Z., J. Miao, A. Peralta-Alva and M. S. Santos (2012): “Numericalsimulation of nonoptimal dynamic equilibrium models,” Working Paper, Fed-eral Reserve Bank of Saint Louis.
Fernandez-Villaverde, J. and J. F. Rubio-Ramirez (2007): “Estimatingmacroeconomic models: A likelihood approach," Review of Economic Studies,74, 1059-1087.
Fernandez-Villaverde, J., J. F. Rubio-Ramirez and M. S. Santos (2006):“Convergence properties of the likelihood of computed dynamic models,” Econo-metrica, 74, 93-119.
Futia C. (1982): “Invariant distributions and the limiting behavior of Markovianeconomic models,” Econometrica, 50, 377-408.
Grandmont, J. M. (1985): “On endogenous competitive business cycles,” Econo-metrica, 53, 995–1045.
Greenwood, J., and G. Huffman (1995): “On the existence of nonoptimal equi-libria in dynamic stochastic economies,” Journal of Economic Theory, 65, 611-623.
Hansen, L. P. (1982): “Large sample properties of the generalized method ofmoments,” Econometrica, 50, 1029-1054.
Hansen, L. P. and K. J. Singleton (1982): “Generalized instrumental variablesestimation of nonlinear rational expectations models,” Econometrica, 50, 1269-1286.
Hellwig, M. (1983): “A note on the implementation of rational expectations equi-libria,” Economics Letters, 11, 1-8.
Hildenbrand, W. (1974): Core and Equilibria of a Large Economy. Princeton,NJ: Princeton University Press.
Hopenhayn, H. and E. C. Prescott (1992): “Stochastic monotonicity and sta-tionary distributions for dynamic economies,” Econometrica, 60, 1387-1406.
42
Jennrich, R. I. (1969): “Asymptotic properties of non-linear least squares estima-tors,” Annals of Mathematical Statistics, 40, 633-643.
Judd, K. L., (1992): “Projection methods for solving aggregate growth models,”Journal of Economic Theory, 58, 410-452.
Judd, K. L. and S.-M. Guu (1992): “Asymptotic methods for aggregate growthmodels,” Journal of Economic Dynamics and Control, 21, 1025-1042.
Kehoe, T. J., and D. K. Levine (2001): “Liquidity constrained markets versusdebt constrained markets,” Econometrica, 69, 575-598.
Krueger, D. and F. Kubler (2008): “Markov equilibria in macroeconomics.” TheNew Palgrave Dictionary of Economics. Second Edition. Eds., S. N. Durlaufand L. E. Blume. Macmillan.
Krusell, P. and A. Smith (1998): “Income and wealth heterogeneity in themacroeconomy,” Journal of Political Economy, 106, 867-896.
Kubler, F. and H. M. Polemarchakis (2004): “Stationary Markov equilibriafor overlapping generations,” Economic Theory, 24, 623-643.
Kubler, F. and K. Schmedders (2002): “Recursive equilibria in economies withincomplete markets,” Macroeconomic Dynamics, 6, 284-306.
Kubler, F. and K. Schmedders (2003): “Stationary equilibria in asset-pricingmodels with incomplete markets and collateral,” Econometrica, 71, 1767-1795.
Kydland, F. E. and E. C. Prescott (1980): “Dynamic optimal taxation, ra-tional expectations and optimal control,” Journal of Economic Dynamics andControl, 2, 79-91.
Lee, B.-S. and B. F. Ingram (1991): “Simulation estimation of time-series mod-els,” Journal of Econometrics, 47, 197-205.
Marcet, A., and R. Marimon (1998): “Recursive contracts,” Economics WorkingPapers No. 337, Universitat Pompeu Fabra.
Newey, W. K. and D. McFadden (1994): "Large sample estimation and hy-pothesis testing." Handbook of Econometrics, volume 4. Eds., D. McFaddenand R. Engle. Amsterdam: Elsevier, North Holland.
43
Peralta-Alva, A. and M. S. Santos (2012): “Ergodic invariant measures fornon-optimal economies,” manuscript.
Phelan, C. and E. Stacchetti (2001): “Sequential equilibria in a Ramsey taxmodel,” Econometrica, 69, 1491-1518.
Sanghvi, A. P. and M. J. Sobel (1976): “Bayesian games as stochastic pro-cesses,” International Journal of Game Theory, 5, 1-22.
Santos, M. S. (2000): “Accuracy of numerical solutions using the Euler equationresiduals,” Econometrica, 68, 1337-1402.
Santos, M. S. (2002): “On non-existence of Markov equilibria for competitive-market economies,” Journal of Economic Theory, 105, 73-98.
Santos, M. S. (2010): “Consistency properties of a simulation-based estimator fordynamic processes,” Annals of Applied Probability, 20, 196-213.
Santos, M. S. and A. Peralta-Alva (2005): “Accuracy of simulations for stochas-tic dynamic models,” Econometrica, 73, 1939-1976.
Santos, M. S. and J. Rust (2004): “Convergence properties of policy iteration,”SIAM Journal on Control and Optimization, 42, 2094-2115.
Santos, M. S., and J. Vigo-Aguiar (1998): “Analysis of a numerical dynamicprogramming algorithm applied to economic models, ” Econometrica, 66, 409-426.
Santos, M. S., and M. Woodford (1997): “Rational asset pricing bubbles,”Econometrica, 65, 19-58.
Schenk-Hoppe, K. R. and B. Schmalfuss (2001): “Random fixed points in astochastic Solow growth model,” Journal of Mathematical Economics, 36, 19-30.
Schmitt-Grohe, S. and M. Uribe (2004): “Solving dynamic general equilibriummodels using a second-order approximation to the policy function,” Journal ofEconomic Dynamics and Control, 28, 755-775
Stenflo, O. (2001): “Ergodic theorems for Markov chains represented by iteratedfunction systems,” Bulletin of the Polish Academy of Sciences: Mathematics,49, 27-43.
44
Stokey, N. L., R. E. Lucas, and E. C. Prescott (1989): Recursive Methodsin Economic Dynamics. Cambridge, MA: Harvard University Press.
45
Figure 1: Exact and numerical solution.
Figure 2: Euler equation residuals of the numerical solution.
1
Figure 3: Numerical policy function of bond holdings.
Figure 4: Numerical approximation of the relative price.
2
Figure 5: Offer curve.
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Real Money Holding, t
Rea
l Mon
ey H
oldi
ng, t
+1
Offer curveSimulation, our solutionSimulation, continuous policy function
Figure 6: Equilibrium dynamics.
3