lecture notes in dynamic optimization - jorge barro - … notes in dynamic optimization jorge a....
Post on 01-Apr-2018
218 Views
Preview:
TRANSCRIPT
Lecture Notes in Dynamic Optimization
Jorge A. BarroDepartment of EconomicsLouisiana State University
December 5, 2012
1
1 Introduction
While a large number of questions in economics can be answered by considering only the static
decisions of individuals, firms, and other economic agents, this set is by no means exhaustive. We
know that in the real world, a trip to the produce market involves a decision between how many
apples and oranges to buy today and how many apples you may want to buy tomorrow. More
generally, consumption decisions of individuals are a tradeoff between a bundle of goods today
and a bundle of goods at some time in the future. These intertemporal consumption decisions are
linked by a savings decision. Firms make intertemporal decisions as well. Consider, for example, the
decision of a CEO choosing the optimal dividend paid to stockholders. If these stockholders are paid
an excessively high dividend, the company could forego an opportunity to invest in capital or hire
workers that could increase the company’s future profits. Suppose now the firm is deciding whether
to hire new employees. The process consists of taking the time, effort, and financial resources
to advertise the opening, interview potential employees, and train the incumbent. Given these
large costs associated with adjusting the labor force, most firms would benefit from accounting for
employees’ productivity over some time horizon. You should think of several examples of these
dynamic decisions in the context of your own field.
The notes that follow are partly my own and partly from a number of resources, including Dy-
namic Economics by Jerome Adda and Russell Cooper (2003),1 Recursive Methods in Economic
Dynamics by Nancy Stokey, Robert Lucas, and Edward Prescott (1989),2 Recursive Macroeco-
nomic Theory by Thomas Sargent and Lars Ljungqvist (2004),3 and of course A First Course in
Optimization Theory by Rangarajan Sundaram.4
1Easiest.2Quite challenging.3A little bit harder than Adda and Cooper, but tons of applications.4A bit more difficult than Sargent and Ljungqvist. It’s concise, but still more than we wish to know at this point.
2
2 Dynamic Optimization
We’ll focus on the problem of maximizing a function f : D → R, where the dimension of D
possibly represents different time periods. For example, suppose you were considering the problem
of maximizing your utility over your lifetime. Just to keep things simple, assume you live for T
periods with certainty. Further, suppose that c ∈ RT is a vector of consumption bundles throughout
your lifetime. In other words, c1 represents the amount that you consume in the first period of
your life, c2 represents the amount that you consume in the second period of your life, and so on.
Finally, let B ⊆ RT+ be some convex set representing a financially feasible budget set of allocations
throughout your life. Then if U : RT → R represents your lifetime utility function, your utility
maximization problem can be stated as:
maxcU(c) subject to c ∈ B (1)
We know that if the function u satisfies certain properties, such as strict concavity, then any
solution to this problem will be unique. We could approach this problem in the traditional way.
That is, we could simply set up a Lagrangean function and take the usual first-order conditions - a
method we will call the method of direct attack. This is potentially a simple problem if T is small.
However, human beings tend to live long lives.5 Moreover, firms, such as the chemical company
DuPont have been operating in excess of two centuries. For this reason, we study an alternative
approach - the method of dynamic programming.
3 The Objective Function
This introduction to dynamic programming is example-driven in the sense that we consider the
utility-maximization problem. We can (and will) consider optimization problems with objective
functions that are more general payoff or return functions. For example, in an application within
Industrial Organization, the objective function will represent profits to a firm.
5Life is long purely in the mathematical optimization sense. Indeed, life is short, and we should enjoy everymoment of it. This point further motivates the use of more efficient optimization methods.
3
Our specification of the utility function relies on two important observations. First of all, income
volatility exceeds consumption volatility. This implies that individuals prefer to save their income
in a way that smooths consumption over time.6 Second, individuals tend to discount future utility.
We accept these as preferences that are consistent with observed behavior, and specify our utility
function as follows:
U(c1, . . . , cT ) =
T∑t=1
βt−1u(ct) (2)
where u(·) is a strictly increasing and strictly concave function, and β ∈ [0, 1) is the personal
discount factor. Also assume that limc→0 u′(c) = ∞ so that marginal utility becomes infinite as
consumption approaches zero. This will simplify the problem in ways that we will discuss later.
While the discounting feature of this utility function is immediately obvious, the consumption-
smoothing feature of the utility function perhaps is not. In order to understand the latter, suppose
that u(c) were linear. Then an individual would only be interested in maximizing the sum of lifetime
consumption, and any intertemporal consumption reallocation would not make the individual better
or worse off. By contrast, now assume the individual had strictly concave utility. Since the marginal
utility of consumption low when consumption in one period is high, the individual would likely be
better off by decreasing consumption in that period and increasing consumption in a period where
consumption was relatively low. This process of “transferring” consumption from one period to
another is called intertemporal substitution.
4 Simple Two-Period Optimization
In this section, we will consider the problem of maximizing utility over two periods, subject to
intertemporal budget constraints. Assume that we are endowed with a lump-sum of wealth a1 in
period 1.7 We can spend some and save the remainder a2 for period 2. For each unit, a, that we
save in a period, we get Ra in the following period, where R ≥ 1 is the gross interest rate. We state
6This motivated the permanent income theory of consumption developed by Irving Fisher and Milton Friedman.7Economists often reference this as the “cake-eating” problem.
4
the problem as follows:
maxc1,c2,a2,a3
u(c1) + βu(c2) (3)
subject to c1 + a2 = a1, and c2 + a3 = Ra2
While it may seem sub-optimal to save a positive amount in the final period of life (and indeed it is
sub-optimal), nothing currently states that an individual can’t borrow in the final period of life and
die with large amounts of debt. Perhaps we should add the following non-negativity constraint to
the optimization problem to ensure that the individual doesn’t run a Ponzi-scheme: a3 ≥ 0. Notice
that we are doing more work than is necessary. We could simplify this problem by eliminating
a2 from the optimization problem and combining the two per-period budget constraints to get a
lifetime budget constraint:
c1 +1
R(c2 + a3) = a1 (4)
Then we would get first-order conditions (differentiating the Lagrangean function with respect to
c1, c2, and a3, respectively):
u′(c1) = λ (5)
Rβu′(c2) = λ (6)
λ = φR (7)
and complementary slackness conditions:
a3 ≥ 0, φ ≥ 0, and φa3 = 0. (8)
where λ is the multiplier on the equality constraint and φ is the multiplier on the inequality
constraint. Let {c∗1, c∗2, a∗2, a∗3} denote the optimal solution to the optimization problem. Equations
(5) and (6) give the following intertemporal relationship:
u′(c∗1) = Rβu′(c∗2). (9)
5
Equation (9) is called the Euler equation, and it plays a fundamental role in macroeconomics and
finance.8 We’ll have more to say about this equation in the generalized T-period problem. For now,
we focus on the last remaining issue: the multiplier on the non-negativity constraint. Recall that
the multiplier on the equality constraint is interpreted as the marginal utility from increasing initial
wealth. Since utility is always increasing in the consumption goods, and demand is increasing in
wealth, it must be the case that marginal utility from increasing wealth is positive. Then from (7),
we know that λ = φR and λ > 0 implies that a∗3 = 0 from the complementary slackness conditions.
Also, since marginal utility is infinite near ct = 0 for t = 1, 2, we know that optimal consumption
will be positive in each period. This means that we can ignore any non-negativity constraints on c1
and c2. Finally, we know that the optimal solutions {c∗1(a1, R), c∗2(a1, R), a∗2(a1, R), a∗3(a1, R)} will
each depend on initial wealth a1, the interest rate R, and nothing else! These optimal solutions, or
policy functions as we will call them, can be substituted back into the objective function to get the
indirect utility function: U (c∗1(a1, R), c∗2(a1, R)). From here on, this indirect utility function will be
called the value function, denoted V (a1, R), and it can be written as the solution to the following
problem:
V (a1, R) = maxc1,c2,a2,a3
u(c1) + βu(c2) (10)
subject to c1 + a2 = a1, and c2 + a3 = Ra1
Exercise 1: Solve for the policy functions and value function of the previous exercise using
u(c) = log(c).
5 The Three-period Problem
Suppose now, that instead of having to make the decision at t = 1, that we were to add a
period t = 0 and solve the 3-period utility maximization problem. After doing all that hard work,
it would be nice if we did not have to start from scratch. Think about it...where did the a1 in the
value function come from? In the real world, we know that our current stock of wealth generally
8We can estimate the parameters of this function by assuming this equation holds true in expectation. This mo-tivated the seminal work of Hansen and Singleton (1982) in their generalized method of moments (GMM) estimationof the structural parameters.
6
depends on the savings decisions that we made in the past. So wouldn’t it make sense that a1 in
the 2-period problem would just be the result of our savings decision (a1) if we had considered the
problem that started at t = 0? As it turns out, the answer is YES! Fortunately for us, we already
solved the two-period problem for any value a1. We know that because we substituted the optimal
solution into the utility function for the two-period problem, then the value function V (a1, R) will
tell us how changes in initial wealth changes the utility after accounting for the optimal decision of
the individual. In other words, we don’t need to think about all the decisions that the individual
will make in the future; it suffices to know that the individual will act optimally in each subsequent
period. This is what is known as the principle of optimality. Then instead of solving the entire
problem again, we will simply solve the following problem for any initial wealth level a0:
V0(a0) = maxa1
u(c0) + βV1(Ra1) (11)
subject to c0 = a0 − a1,
where V0 and V1 are the value functions from the 3-period and 2-period problems, respectively, a0
is the new initial stock of wealth, and the parameter R has been dropped from the value function,
since we are not concerned with that right now. Also notice that we are only maximizing over
values of a1 since we could just substitute c0 = a0 − a1 into the objective function and solve the
unconstrained problem.
Exercise 2: Using the value function from the two-period problem in Exercise 1, solve for the
value function of the 3-period problem. Then solve the entire 3-period problem from scratch using
the method of direct attack, and show that this method leads to the same value function as our
“new” method.
6 The T-period and Infinite Problem
Hopefully, you can immediately see the pattern that develops from this method. In fact, we
could have solved the one-period problem at t=2, then solved the two-period problem (solved at
8
t=1) using the value function from the one period problem, then finally the three period problem
(at t=0) using the value function from the two-period problem. This concept of backwards induction
is at the heart of dynamic programming. We can solve any T-period problem through this recursive
process. In fact, given our assumptions, we can show that by choosing a very high value of T, we
can approximate the infinitely-lived individual problem. As it turns out, our process satisfies the
conditions for a contraction mapping, and the assumptions that we made earlier satisfy sufficient
conditions for convergence of our value function through this iterative process.
6.1 The T-period Problem
For any initial endowment of wealth, a0, the finite horizon problem can be stated as follows:
V0(a0) = max{at+1}Tt=0
T∑t=0
βtu(ct) (12)
subject to ct + at+1 = Rat for t = 0, . . . , T and aT+1 ≥ 0, (13)
where the first constraint is the intertemporal budget constraint and the second constraint is the
no-Ponzi-game condition. Notice that in any period t = 0, . . . , T the problem could be restated in
terms of the value functions in each period as follows:
Vt(at) = maxat+1
{u(ct) + βVt+1(at+1)} (14)
subject to ct + at+1 = Rat for t = 0, . . . , T and aT+1 ≥ 0. (15)
When we write the equation out like we did in (12), we call this the sequential statement of the
dynamic optimization problem. By contrast, when we translate it into dynamic programming
language, like we did in (14), we call this the recursive formulation of the problem. Further, (14)
is a functional equation, and generally called a Bellman equation after Richard Bellman - a pioneer
in dynamic optimization.
9
6.2 The Infinite-Horizon Problem
You might be wondering why we care about infinitely-lived problem if people don’t actually live
forever. First of all, we tend to find that the infinitely-lived problem can accurately approximate
the finitely-lived individual’s problem and simplify quantitative analysis - especially when we don’t
care about so-called life-cycle properties of decisions. The infinite-horizon problem also makes sense
when if you think about the potential role of altruism in the savings decisions. If we care about
our children and our savings decisions are at least partially motivated by an altruistic bequest
motive, then our decisions will be accurately measured by this dynastic model. This discussion has
focused on an individual’s decisions for motivating the infinite-horizon model, but as we claimed,
we could certainly think about the decision of a firm over a much longer time horizon. We take this
motivation and generalize the infinite-horizon problem in the context of the finite-horizon problem
in the following section.
7 Mathematical Foundations of Dynamic Optimization
This section takes the logic from the previous sections and generalizes it so that we can consider
the infinite-horizon dynamic optimization problem more formally. This infinite-horizon problem
consists of a state vector, st ∈ Ds ⊆ Rm, an action vector or control vector ct ∈ Dc ⊆ Rn, a
transition function g : Ds × Dc → Ds, and a one-period reward or momentary payoff function
ρ : Ds ×Dc → R. Lots to digest here.
First consider the state vector, st, which we interpret as the agent’s “environment,” which is
outside of the control of the agent at the beginning of period t. In the savings problem, this is
the amount of assets available to the individual at the beginning of period t. The state vector
is an element of the state space Ds, which determines all possible values of st. The state vector
contains all the information available at time t needed to make an optimal decision. Examples of
state spaces include an individual’s possible asset levels, education level, health level (and health
insurance status), family composition, and employment status, or firm’s labor force, capital stock,
and productivity.
10
Next consider the control vector, ct. The set of possible actions that the economic agent can
make is characterized in the control space Dc. Notice that this set of feasible actions generally
depends on the state vector, st, so we could write Dc(st). For example, the amount that you
consume today could be limited by the funds in your bank account. More generally, the set of
controls chosen by an individual in a period could include savings, consumption, labor supply,
medical expenditures, educational investment, job search intensity, or even accept or reject a job
offer.9
Now consider the transition function, st+1 = g(st, ct), which determines our state tomorrow as a
function of our current state and the actions we take today. In the savings problem, this was simply
the budget constraint. The amount that funds that wake up with tomorrow are just the difference
between the funds we wake up with today minus the amount that we consume.10 Transition
functions generally come in the form of resource constraints. Perhaps the more important feature
of the transition function is that it maps the power set Ds×Dc back into the state space Ds. This
is an important feature that, along with other properties, will qualify this dynamic optimization
problem as a contraction mapping, or a mapping from a set back onto itself.
Finally, consider the payoff function, ρ. We initially defined this function to map values of st
and ct into R. However, we could solve the transition function for ct, redefine the function so that
ct = g(st, st+1), and substitute this into the payoff function to get: ρ(st, st+1) ≡ ρ(st, ct). Similar
to the restriction that ct ∈ Dc(st), we restrict st+1 ∈ Γ(st) and define write the Bellman equation
as follows:
V (st) = maxst+1∈Γ(st)
ρ(st, st+1) + βV (st+1) ∀ st ∈ Ds. (16)
Notice that this optimization problem has the property of stationarity, i.e., it is time-independent.
This is, of course, still a dynamic optimization problem, but the specific time of the decision is
irrelevant. This means that we can get omit of all the time subscripts and instead use a prime to
9In this case, the set of possible offers would be the state space, and “accept” or “reject” would be the set ofchoices. This idea was originally proposed by McCall (1970) and serves as the foundation of the Nobel-prize-winninglabor market model of Christopher Pissarides and Dale Mortensen.
10Assuming no outside income.
11
denote next period values as follows:
V (s) = maxs′∈Γ(s)
ρ(s, s′) + βV (s′) ∀ s ∈ Ds. (17)
Now we are tasked with determining the value function, V (s) and the associated policy function
s∗(s) that solves (17). We have the following motivating theorem from Adda and Cooper p. 27:
Theorem 1. Suppose ρ(s, s′) is real valued (which we already assumed), continuous, and bounded,
β ∈ (0, 1), and the constraint set Γ(s) is non-empty, compact, and continuous. Then there exists a
unique value function V (s) that solves (17).
Understanding this involves introducing a new functional operator T onto any function W :
R+ → R that satisfies the following condition:
(TW )(s) = maxs′∈Γ(s)
ρ(s, s′) + βW (s′) ∀ s ∈ Ds. (18)
Actually, we have already used this “T” operator once already - when we solved the 3-period problem
using the value function from the 2-period problem! Let V1 be the value function from the 1-period
problem, V2 = TV1 be the value function from the 2-period problem, . . ., Vn+1 = TVn be the value
function from the (n+1)-period problem. Suppose we use the T operator to construct the sequence of
functions {Vn}. Then it can be shown that for any initial function V1,11 limn→∞ ‖Vn−V ∗‖sup = 0,
where V ∗ is the unique value function, which we know exists from Theorem 1. This result is
exactly the motivation behind value function iteration as a computational process.
8 Computational Algorithm: The Savings Problem
8.1 Preliminary Activity
This section presents a brief computational algorithm for solving the infinitely-horizon savings
problem. Notice from the previous section that value function iteration leads to uniform conver-
11In practice, if we don’t have a good initial guess of V1, we usually choose V1(s) = 0 for all s ∈ Ds
12
gence of a sequence of functions. However, we can not show uniform convergence of one function
onto another on a computer. Instead, we must solve for the function at discrete points, generally
determined from the discrete points of the state space. In this case the state space is the assets
brought into the period. Accordingly, we can discretize the state space by choosing an ordered grid
of assets, {a1, a2, . . . , ana}, where na is the number of grid points. Then we create two zero-vectors
of size na, which will be updated with Vn+1 and Vn, respectively, in each iteration.
8.2 Value Function Iteration
We begin by specifying some very small tolerance level that will determine the approximation of
the convergence of the sequence {Vn}. Then each iteration (of the “T” operator) uses the following
steps:
1. For each i ∈ 1, 2, . . . , na, solve Vn+1(ai) = maxa′ u(rai − a′) + βVn(a′).
2. Save both the value function and optimal asset choice in a vector.
3. If ‖Vn+1 − Vn‖ is less than the pre-specified tolerance, then the program has approximately
converged
4. If ‖Vn+1−Vn‖ is not less than the tolerance, update the value function vector so that Vn+1 =
Vn and repeat from Step 1.
Notice in Step 1 that the optimization routine must solve over Vn(a′), which is only specified
at a finite amount (na) points. No big deal - we can interpolate Vn using a few different methods.
Linear interpolation, for example, will simply connect the each point Vn(ai) with a straight line.
Spline interpolation, by contrast, will fit a polynomial over the points, leading to more accurate
approximations. Unfortunately, spline interpolation is also more time-consuming. As a rule of
thumb, I usually use linear interpolation until I feel comfortable with the quality of the code, then
apply spline interpolation.
13
9 A Variety of Dynamic Programming Examples
This section provides several examples of well-known dynamic programming problems. While
we have not discussed stochastic processes in the preceding notes, we can consider them in the
context of these examples. After all, many of the intertemporal economic decisions that we make
are affected by the uncertainty of future outcomes. The models in this section are partial equilibrium
in the sense that we only consider the optimal responses of economic agents for given market values,
such as prices. By contrast, general equilibrium theory completes the analysis by determining prices
endogenously. For now, we take market values as given and proceed.
9.1 Human Capital Accumulation
This subsection presents a simple model of Ben-Porath (1970) human capital accumulation.
This model has survived the test of time and serves as the foundation for models, such as Heckman,
Locher, and Taber (1998a, 1998b, 1998c) that explain wage inequality and study the effects of
labor income taxes on educational decisions. We return to that model later. Guvenen and Kuruscu
(2006) also used the model to explain trends in the wage distribution over the last thirty years.
Kuruscu (2006) later used the model to quantify the effectiveness of on-the-job training. The idea
behind the model is that low labor earnings early in life reflect a larger percentage of time training
and learning (i.e. building a stock of human capital) while higher earnings later in life reflect low
amounts of time spent developing human capital and more time spent using the human capital in
more productive ways.
Let ht be the stock of human capital that an individual has at age t. Let it be the amount
of time that the individual spends investing in human capital, and let w be the rental rate of
this human capital. Then, the individual’s labor earnings are wht(1 − it), and the problem of the
individual is to allocate time between investing in human capital (such as training or learning) and
more productive activities as follows:
V (ht) = maxit
u(ct) + βV (ht+1) (19)
14
s.t. ct = wht(1− it) (20)
and ht+1 = A (htit)α
+ (1− δ)ht, (21)
where A and α are parameters of the human capital production function, and δ is the human
capital depreciation rate. To notice the intertemporal tradeoff, think about how changes in it affect
each of the constraints. The first constraint (current period payoff), Equation (20) shows how
consumption in period t depends on the human capital stock and rises when the individual spends
less time training. The second constraint (future payoff) shows how future human capital rises
when the individual rises when the individual invests more time in training.
9.2 Unemployment and Incomplete Markets
This subsection introduces a popular model of idiosyncratic unemployment spells and partial
insurance, as in Huggett (1993), Aiyagari (1994), and Krusell and Smith (1998). Incomplete markets
exist when individuals can not fully insure against all future outcomes. The most prevalent example
of incomplete markets is unemployment. Because of moral hazard resulting from disincentives to
job search, government or private entities will generally not pay individuals the full amount of
their lost wages in the event of job loss. This leaves individuals to self-insure. We can measure
this level of self-insurance by comparing a complete market model to the following model in which
individuals can only buy a single asset that pays out the same amount regardless of the individual’s
unemployment status.
We also introduce random outcomes captured by what we call a Markov process. Two things
we need to know about Markov processes: they capture autocorrelation (outcomes that are cor-
related over time) and they can significantly simplify our problem. A Markov process will tell us
the probability of certain outcomes tomorrow, given a certain state today. For example, if we are
employed today, there is a good chance that we will be employed tomorrow. With a certain prob-
ability, however, we might randomly transition into unemployment status. We could then consider
probabilities of outcomes given that we are in the unemployed state. This set of probabilities can
be represented in a Markov matrix like the one shown in Figure 9.2.
15
Let pee denote the probability of remaining employed tomorrow given that we are employed
today (top left entry in the Markov matrix in Figure 9.2), and let puu denote the probability of
remaining unemployed tomorrow given that we are unemployed today (bottom right entry in the
Markov matrix in Figure 9.2). Given this very simple stochastic process, we can include employment
status as an element of the state space, and consider the optimal savings problem of the employed
individual as follows:
V e(at) = maxat+1
u(ct) + β [peeVe(kt+1) + (1− pee)V u(kt+1)] (22)
s.t. ct = rat − at+1 + w (23)
and for the unemployed individual as follows:
V u(at) = maxat+1
u(ct) + β [(1− puu)V e(at+1) + puuVu(at+1)] (24)
s.t. ct = rat − at+1 + b, (25)
where w is the wage of an employed individual, and b is the unemployment benefit given to an
unemployed individual. It can be shown that if w > b, then the individual will save to compensate
for this incomplete insurance. In fact, at the beginning of recessions when the probability of
unemployment increases, we tend to notice spikes in individuals’ saving rates. This can be seen in
Figure 9.2, which is the historical U.S. savings rate obtained from the Federal Reserve Bank of St.
Louis data set (FRED, as it is commonly referred).
9.3 Health Shocks and Medical Expenditures
In this subsection, we present a model of a health shock and medical expenditures that is largely
credited to Michael Grossman (1972). It should be noted that the parameters and specification of
this model remain largely untested (and perhaps a good idea for future research!). Let ht denote
the stock of health of an individual at time t, and suppose that it depreciates at an accelerated rate
17
(δt) over an individual’s lifetime. Also assume that an individual’s probability of survival depends
on the stock of health through some probability function s(ht+1), which satisfies s′(·) > 0 and
s′′(·) < 0. Further, suppose that individual’s have preferences over consumption and the quality
of their health. Finally, assume that individuals have random health shocks ε, and they purchase
medical expenditures mt that increase their stock of health. Then we can write the individual’s
optimization problem as follows:
V (at, ht, εt) = maxmt,at+1
u(ct, ht) + s(ht+1)βE [V (at+1, ht+1, εt+1)] (26)
s.t. ct = w + rat − at+1 −mt (27)
and ht+1 = (1− δt)ht + f(mt)− εt, (28)
where E in the Bellman equation is the expectation operator, w is labor income, and f(·) is a
function that determines the effectiveness of medical purchases onto health level. The original
Grossman model also included time allocation as a determinant of health quality. This is an
important consideration that can account for non-medical inputs (exercise, for example) into the
health production function.
9.4 Wage Offers and Discrete Choice Modelling
This subsection considers the simple “accept” or “reject” decision of an individual that receives
random wage offers, which is the model presented by McCall (1970). Assume that in each period,
the individual receives a random wage offer w, which arrives from some wage distribution. If she
accepts the offer, she remains employed and receives wage w forever. In this case, the payoff is
u(w)1−β . To see this, consider the geometric series u(w)
∞∑t=0
. Rejecting the offer results in receiving one
more period of the unemployment benefit b plus the discounted expected future value of the next
wage offer, w′, resulting in the following discrete-choice model:
V (w) = max
{u(w)
1− β, u(b) + βEV (w′)
}(29)
19
You might be wondering, what is the threshold wage such that an individual will accept anything
at or above the amount, and reject anything below that amount. In fact, this reservation wage is
the wage w∗ that solves the following equation:
u(w∗)
1− β= u(b) + βEV (w′), (30)
which depends on the distribution of the wage offers.
9.5 Lucas Tree Asset Pricing and Portfolio Allocation Model
Now we consider fundamental problem in financial economics - the pricing of an asset. Robert
Lucas (1978) compared this simple model to the ownership of a fruit-bearing tree. Each period,
the tree (company) produces a random quantity of fruit (dividend), dt. The individual enters each
period with some part (stock), st, of the tree and must choose the amount st+1 to purchase for next
period. Shares of the tree can be purchased or sold at price pt. Also, suppose there is a risk-free
asset at which earns return r between periods. Then the individual must allocate their portfolio
at+1 and st+1 to maximize expected utility as follows:
V (at, st) = maxat+1,st+1
u(ct) + βE [V (at+1, st+1)] (31)
s.t. ct = w + pt(st − st+1) + dtst + rat − at+1 (32)
This constraint can be modified and reinterpreted in a number of ways to address a variety of
questions in financial economics. The key here is that in equilibrium, the net supply of the risk-free
asset (or conversely, debt) in the economy must be zero. In contrast, shares of the tree (company)
must be in positive net supply, and the price times the outstanding shares of the tree determine
the value of the tree.
20
10 Conclusion
This concludes the introduction to dynamic optimization. Hopefully, you’ve learned how to
solve a dynamic optimization problem using computational techniques. By now, you should be
well-equipped to solve and simulate a variety of problems in economics using these numerical ap-
proximation methods. Always remember that there is no substitute for patience and optimism when
you’re doing computational work. Also, remember to check out economist Tony Smith’s guidelines
and recommendations for disciplined computational analysis.
11 References
References
[1] Adda, Jerome, and Russell Cooper, 2003. Dynamic Economics: Quantitative Methods and
Applications. Cambridge: MIT Press.
[2] Aiyagari, S. Rao, 1994. Uninsured Idiosyncratic Risk and Aggregate Saving. Quarterly Journal
of Economics 109 (3), 659-684
[3] Ben-Porath, Yoram, 1967. The Production of Human Capital and the Life Cycle of Earnings.
Journal of Political Economy 75(4) 352-365.
[4] Grossman, Michael, 1972. On the Concept of Health Capital and the Demand for Health.
Journal of Political Economy 80(2) 223-255.
[5] Guvenen, Fatih and Burhanettin Kuruscu, 2012. Understanding the Evolution of the US Wage
Distribution: A Theoretical Analysis. Journal of the European Economic Association 10(3)
482-517.
[6] Heckman, James J., Lance J. Lochner, and Christopher R. Taber, 1998a. Explaining Rising
Wage Inequality: Explorations with a Dynamic General Equilibrium Model of Labor Earnings
with Heterogeneous Agents. Review of Economic Dynamics 1(1) 1-58.
21
[7] Heckman, James J., Lance J. Lochner, and Christopher R. Taber, 1998b. General Equilibrium
Treatment Effects: A Study of Tuition Policy. American Economic Review. 88(2) 381-386.
[8] Heckman, James J., Lance J. Lochner, and Christopher R. Taber, 1998c. Tax Policy and
Human Capital Formation. American Economic Review. 88(2) 293-297.
[9] Huggett, Mark, 1993. The Risk-Free Rate in Heterogeneous-agent Incomplete-insurance
Economies. Journal of Economic Dynamics and Control. 17(5-6) 953-969.
[10] Krusell, Per and Anthony Smith, 1998. Income and Wealth Heterogeneity in the Macroecon-
omy. Journal of Political Economy. 106(5) 867-896.
[11] Kuruscu, Burhanettin, 2006. Training and Lifetime Income. American Economic Review 93(6)
832-846.
[12] Ljungqvist, Lars and Thomas Sargent, 2000. Recursive Macroeconomic Theory. Cambridge:
MIT Press.
[13] Lucas, Robert E. Jr., 1972. Asset Prices in an Exchange Economy. Econometrica 46(6) 1429-
1445.
[14] Lucas, Robert E. Jr., and Nancy Stokey, with Edard Prescott, 1989. Recursive Methods in
Economic Dynamics. Cambridge: Harvard University Press.
[15] McCall, J., 1970. Economics of Information and Job Search. Quarterly Journal of Economics
84(1) 113-126.
[16] Sundaram, Rangarajan, 1996. A First Course in Optimization Theory. Cambridge: Cambridge
University Press.
22
top related