multivariate calculus and constrained 1
Post on 07-Apr-2018
222 Views
Preview:
TRANSCRIPT
-
8/6/2019 Multivariate Calculus and Constrained 1
1/9
Pre-sessional Mathematics and Statistics EC961
Multivariate Calculus and Constrained Optimisation
Pre-requisites: For one variable function: simple derivatives (of log, exponential andpower functions etc.) and the chain rule, geometric interpretation of first and second
order derivatives of a function, concave and convex functions, stationary points,
Taylor series expansion. Partial derivatives.
1. Convex set and multivariate function
Definition: A set S is convex if for any two points x , 'x in S the point')1( xx + is also in S when 10 >= babxaxxxS is convex.
Definition: Let nSf : , then f is a function ofn variables.
Here f maps a point in S to a single value in . In this course, we usually deal
with functions defined on convex set S .
Definition: The graph of f consists of the set of points)),,,(,,,,( 2121 nn xxxfxxx in 1+n for ),,,( 21 nxxx in S .
Example 2: Let S be the one given in example 1, and 222121 ),( xxxxf += . The
graph is as below
O
b
a
X
X
')1( XX +
1x
2x
O
),( 21 xxf
1x
2x
1
-
8/6/2019 Multivariate Calculus and Constrained 1
2/9
Definition: Let f be a function of two variables and c be a constant. The set of all),,,( 21 nxxx such that cxxxf n =),,,( 21 is called a level curve of f (with
value c ).
If f is a two variable function, the level curve is on a two-dimensional plane. (If f
is a single variable function, what is its level curve?) For functions of threevariables, one would have a level surface.
Note that indifference curves are just level curves for utility functions.
Example 3: The level curve (heavy curve) of the function in example 2 with a value0>c is drawn below:
Definition: An upper level set of a function f with given c is a set )(cU suchthat
{ }cxfSxxcU = )(and:)( .A lower level set, )(cL , is defined as { }cxfSxxcL = )(and:)( .
So the upper level set is a collection of all level curves above value c , and the lowerlevel set is a collection of all level curves below value c .
Example 4: the upper and lower level sets for function in example 3 with respect toc are drawn below:
Definition: If f is a real-valued function, its gradient, denoted f, is defined as
O
),( 21 xxf
1x
2x
c
O
),( 21 xxf
1x
2x
c
Upper level set
O
),( 21 xxf
1x
2x
c
Lower level set
2
-
8/6/2019 Multivariate Calculus and Constrained 1
3/9
=
n
n
xf
xf
xf
xxxf
/
/
/
),,,(2
1
21
.
Example 5: For the function in example 2 (let 2=S ), the gradient of the function is
=
=2
1
2
121
2
2
/
/),(
x
x
xf
xfxxf .
Definition: A collection of second order derivatives arranged in the following way is
called the Hessian matrix
=
222
21
2
222
22
212
12
1222
12
21
///
///
///
),,,(
nnn
n
n
n
xfxxfxxf
xxfxfxxf
xxfxxfxf
xxxH
Example 6: For the function in example 2, its Hessian matrix is
=
=
20
02
//
//),(
22
221
212
221
2
21xfxxf
xxfxfxxH
(Youngs Theorem: If a two variable function is 2C , i.e. twice continuously
differentiable, then 212
122 // xxfxxf = .
In this course, we only consider these functions. In this case, the Hessian matrix is
symmetric, i.e. HHT = .)
Theorem 1 (Taylor's Formula): Let x be a given point in n where the functionf is defined, and h is a small vector, i.e. h close to zero. If f is 2C , then
)()(2
1)()()(
3hOhxHhxfhxfhxf TT +++=+ ,
where )( 3hO means of the same order as 3h .
If we know the property of a function at a given point x , then Taylors formula
allows us to approximate the value of that function in the neighbourhood of x (say,hx + ) as long as the function is relatively smooth ( 2C ).
2. Stationary points and their classifications
2.1 Stationary points
Definition: A stationary point of a function is a place where 0=f , that is a place
where the function's value does not change.
Example:
3
-
8/6/2019 Multivariate Calculus and Constrained 1
4/9
x *
f ( x )
x *
f ( x )
The definition of a stationary point also tells us how to find one: find the set of points
where 0=f , that is to solve n equations in n unknowns.
Example 7a: The stationary point for the function in example 5 is
==
=
=
0
0
0
0
2
2),(
2
1
2
121
x
x
x
xxxf .
Example 7b: Find stationary points of the function )/(),( 22 yxxyyxf += for)0,0(),( yx .
==
=
++=
)2(0)(
)1(0)(
0
0
)/()(
)/()(),(
22
22
22222
22222
yxx
xyy
yxyxx
yxxyyyxf
For )0,0(),( yx , (1) is satisfied iff yx = or yx = . These are also the solutionsto (2). So there are infinite stationary points given by ),( xx or ),( xx , 0x .
2.2. Classifying Stationary Points
Definition: A local maxima is a stationary point where for a small neighbourhood
about the stationary point the function is no greater than the stationary point. A local
minima is a stationary point where the function attains a minimum in a small
neighbourhood of the point.
E.g.
local maximum local minimum
Theorem 2: Let f be a 2C function. A stationary point ),,,(* **2*1 naaaa = is a
local maximum for f iff the Hessian matrix at *a , *)(aH , is negative semi-
definite (or negative definite); a stationary point *a is a local minimum for f iff*)(aH is positive semi-definite (or positive definite).
4
-
8/6/2019 Multivariate Calculus and Constrained 1
5/9
(Note that if *)(aH is indefinite, the stationary point *a is neither a local
maximum nor a local minimum.)
Proof: Here, we only look at the condition for local maximum, the condition for local
minimum can be obtained similarly. To show that *a is a local maximum, one has
to show that values of function in the neighbourhood of *a , )*( haf + where h
is small, are smaller or equal to the value of the function at *a , *)(af .
Since f is a 2C function, we can use the Taylors formula to expand )*( haf + at
point *a to obtain
)(*)(2
1*)(*)()*(
3
hOhaHhafhafhaf
TT
+++=+Because *a is a stationary point, 0*)( = af . The above equation is simplified
to
)0as(*)(2
1
)(*)(2
1*)()*(
3
=
+=+
hhaHh
hOhaHhafhaf
T
T
So *)()*( afhaf + iff 0*)( haHhT , i.e. *)(aH is negative semi-definite.QED
Example 8: Find and classify stationary points of 4/4/),( 44 yxyxyxf += .
The stationary points require
=+
==
+
=)2(0
)1(00),(
3
3
3
3
yx
yx
yx
yxyxf
From (2), we have3yx = (3)
Substitution of (3) into (1) yields
0)1( 56 == yyyy (4)
Equation (4) has two real solutions 0*1 =y and 1*2 =y . Substituting 0
*1 =y into (3),
one obtains 0*1 =x ; and substituting 1*2 =y into (3) to obtain 1*2 =x .
The two stationary points are therefore )0,0(),( *1*1 =yx and )1,1(),(
*2
*2 =yx .
The Hessian matrix for 4/4/),( 44 yxyxyxf += is
=2
2
31
13),(
y
xyxH .
Evaluating the Hessian at )0,0(),( *1*1 =yx gives
=
01
10)0,0(H
which is indefinite. So )0,0(),( *1*1 =yx is neither a local maximum nor a local
minimum.
5
-
8/6/2019 Multivariate Calculus and Constrained 1
6/9
Evaluating the Hessian at )1,1(),( *2*2 =yx gives
=
31
13)1,1(H
which is positive definite. So )1,1(),( *2*2 =yx is a local minimum.
3. Convexity and Concavity
3.1 Concave and convex functions
Concave and convex functions with single variable normally have the following
shapes:
concave function convex function
Notice from the figure above that the set of points below a concave function is convex
and the set of points above a convex function is convex. This will give us the
definitions.
Definition: A function )(xf is concave iff for any x and 'x in S , and for10 + .
A function )(xf is convex iff for any x and 'x in S , and for 10
-
8/6/2019 Multivariate Calculus and Constrained 1
7/9
af(.) f(.)
f(a)
f(a)+x ff(a)+z f
z+a
f(z+a)
The tangent to the function f at the point a has an equation)()( afxafy
T+= . The distance between the tangent to f at a point az+ is
indicated by the heavy line. The length of the heavy line is simply
)()()( azfafzafT
++ . Now we will use Taylor's formula for the value)( azf +
zaHz
zaHzafzafafzafazfafzaf
T
TTTT
)(2
1
])(21)()([)()()()()(
++++=++
Ifz is small the distance is given by the expression 2/)( zaHzT . If0)( zaHzT the tangent always lies above the function. If 0)( zaHzT then the
tangent always lies below the function. Notice that concave functions have the
property that the tangent always lies above the function forany starting point a ,whilst convex functions have the property that the tangent always lies below the
functionfor any starting point a . So we have the following theorem.
Theorem 3: A twice continuously differentiable function f is(1) concave if the Hessian )(xH is negative semi-definite at all points x ,(2) strictly concave if the Hessian )(xH is negative definite at all points x ,(3) convex if the Hessian )(xH is positive semi-definite at all points x ,(4) strictly convex if the Hessian )(xH is positive definite at all points x .
In economics, we sometimes use concave function to represent utility, and we focus
explicitly on its indifference curves. In what follows, we provide a theorem outlining
properties of level curves associated with concave and convex functions.
Theorem 4: Letd
be any real number that a functionf
can attain.(1) If f is concave, then its upper level set )(dU is a convex set.
(2) If f is convex, then its lower level set )(dL is a convex set.
Proof: Here, we only prove the first part of the theorem, the second part can be done
similarly. Notice that the upper level set is defined as { }dxfxdU = )(:)( , i.e., allx at which the value of function is at least d . Lets choose two arbitrary points, y and 'y , in )(dU , i.e. dyf )( and dyf )'( . We have to show that the convexcombination ')1( yyz += (for 10
-
8/6/2019 Multivariate Calculus and Constrained 1
8/9
or dzf )( and z is in )(dU . So )(dU is a convex set.QED
3.2 Quasi-concave and quasi-convex functions
A standard consumer problem --- maximising utility subject to budget constraints---would have a solution if the utility function has the property that its upper level set
is convex. Such utility function may arise from a concave function as Theorem 4
suggests. It may also arise from a function which is not concave. Pictures below
provide one example:
The 3-D surface depicted on the left panel shows a concave function. The upper level
set it generates is convex. The function shown on the right panel generates the same
upper level set, but its 3-D shape is very different from that on the left. It is obvious
that the function on the right panel is notconcave. A function which generates upper
level sets the same as some concave function is called quasi-concave, and a function
which generates lower level sets the same as some convex function is called quasi-convex. Formal definition is given below.
Definition: Let d be any real number that a function f can attain.
(1) If the upper level set )(dU is a convex set, then f is quasi-concave.
(2) If the lower level set )(dL is a convex set, then f is quasi-convex.
Theorem 4 and the above definition immediately imply that a concave function must
be a quasi-concave and a convex function must be a quasi-convex. (The reverse of
this statement is usually nottrue. The pictures above are an example.) From thedefinition above it is clear that if a function f is quasi-concave then f must be
quasi-convex.
The definition introduced above can certainly be used to identify whether a function is
quasi-concave or quasi-convex. But it may sometimes involve knowing the properties
of complex level curves. A more straightforward method is to use the bordered
Hessian.
Definition: Let ),,,( 21 nxxxx = , the bordered Hessian determinants )(xBr aregiven by
8
-
8/6/2019 Multivariate Calculus and Constrained 1
9/9
=
)()()(
)()()(
)()(0
det)(
''''1
'
''1
''11
'1
''1
xfxfxf
xfxfxf
xfxf
xB
rrrr
r
r
r
, .,,2,1 nr =
Theorem 5: Let f be a twice continuously differentiable function defined on a
convex S in n .
(1) If f is quasi-concave, then 0)()1( xBrr for all x in S and for all
.,,2,1 nr =
(2) If 0)()1( > xBrr for all x in S and for all nr ,,2,1 = , then f is strictly
quasi-concave.
For quasi-convexity, use Theorem 5 to f .
Example 9: Let 0>x and 0>y . Show that the function xyyxf =),( is notconcave but quasi-concave.
The Hessian of this function is
=
01
10),( yxH
which is indefinite. So the function is neither concave nor convex.
The largest bordered Hessian is given by
01
10
0
x
y
xy
with 0),()1( 211 >= yyxB and 02),()1( 2
2 >= xyyxB . So it is strictly quasi-
concave.
9
top related