multivariate calculus and constrained 1

8/6/2019 Multivariate Calculus and Constrained 1

1/9

Pre-sessional Mathematics and Statistics EC961

Multivariate Calculus and Constrained Optimisation

Pre-requisites: For one variable function: simple derivatives (of log, exponential andpower functions etc.) and the chain rule, geometric interpretation of first and second

order derivatives of a function, concave and convex functions, stationary points,

Taylor series expansion. Partial derivatives.

1. Convex set and multivariate function

Definition: A set S is convex if for any two points x , 'x in S the point')1( xx + is also in S when 10 >= babxaxxxS is convex.

Definition: Let nSf : , then f is a function ofn variables.

Here f maps a point in S to a single value in . In this course, we usually deal

with functions defined on convex set S .

Definition: The graph of f consists of the set of points)),,,(,,,,( 2121 nn xxxfxxx in 1+n for ),,,( 21 nxxx in S .

Example 2: Let S be the one given in example 1, and 222121 ),( xxxxf += . The

graph is as below

O

b

a

X

X

')1( XX +

1x

2x

O

),( 21 xxf

1x

2x

1


2/9

Definition: Let f be a function of two variables and c be a constant. The set of all),,,( 21 nxxx such that cxxxf n =),,,( 21 is called a level curve of f (with

value c ).

If f is a two variable function, the level curve is on a two-dimensional plane. (If f

is a single variable function, what is its level curve?) For functions of threevariables, one would have a level surface.

Note that indifference curves are just level curves for utility functions.

Example 3: The level curve (heavy curve) of the function in example 2 with a value0>c is drawn below:

Definition: An upper level set of a function f with given c is a set )(cU suchthat

{ }cxfSxxcU = )(and:)( .A lower level set, )(cL , is defined as { }cxfSxxcL = )(and:)( .

So the upper level set is a collection of all level curves above value c , and the lowerlevel set is a collection of all level curves below value c .

Example 4: the upper and lower level sets for function in example 3 with respect toc are drawn below:

Definition: If f is a real-valued function, its gradient, denoted f, is defined as

O

),( 21 xxf

1x

2x

c

O

),( 21 xxf

1x

2x

c

Upper level set

O

),( 21 xxf

1x

2x

c

Lower level set

2


3/9

=

n

n

xf

xf

xf

xxxf

/

/

/

),,,(2

1

21

.

Example 5: For the function in example 2 (let 2=S ), the gradient of the function is

=

=2

1

2

121

2

2

/

/),(

x

x

xf

xfxxf .

Definition: A collection of second order derivatives arranged in the following way is

called the Hessian matrix

=

222

21

2

222

22

212

12

1222

12

21

///

///

///

),,,(

nnn

n

n

n

xfxxfxxf

xxfxfxxf

xxfxxfxf

xxxH

Example 6: For the function in example 2, its Hessian matrix is

=

=

20

02

//

//),(

22

221

212

221

2

21xfxxf

xxfxfxxH

(Youngs Theorem: If a two variable function is 2C , i.e. twice continuously

differentiable, then 212

122 // xxfxxf = .

In this course, we only consider these functions. In this case, the Hessian matrix is

symmetric, i.e. HHT = .)

Theorem 1 (Taylor's Formula): Let x be a given point in n where the functionf is defined, and h is a small vector, i.e. h close to zero. If f is 2C , then

)()(2

1)()()(

3hOhxHhxfhxfhxf TT +++=+ ,

where )( 3hO means of the same order as 3h .

If we know the property of a function at a given point x , then Taylors formula

allows us to approximate the value of that function in the neighbourhood of x (say,hx + ) as long as the function is relatively smooth ( 2C ).

2. Stationary points and their classifications

2.1 Stationary points

Definition: A stationary point of a function is a place where 0=f , that is a place

where the function's value does not change.

Example:

3


4/9

x *

f ( x )

x *

f ( x )

The definition of a stationary point also tells us how to find one: find the set of points

where 0=f , that is to solve n equations in n unknowns.

Example 7a: The stationary point for the function in example 5 is

==

=

=

0

0

0

0

2

2),(

2

1

2

121

x

x

x

xxxf .

Example 7b: Find stationary points of the function )/(),( 22 yxxyyxf += for)0,0(),( yx .

==

=

++=

)2(0)(

)1(0)(

0

0

)/()(

)/()(),(

22

22

22222

22222

yxx

xyy

yxyxx

yxxyyyxf

For )0,0(),( yx , (1) is satisfied iff yx = or yx = . These are also the solutionsto (2). So there are infinite stationary points given by ),( xx or ),( xx , 0x .

2.2. Classifying Stationary Points

Definition: A local maxima is a stationary point where for a small neighbourhood

about the stationary point the function is no greater than the stationary point. A local

minima is a stationary point where the function attains a minimum in a small

neighbourhood of the point.

E.g.

local maximum local minimum

Theorem 2: Let f be a 2C function. A stationary point ),,,(* **2*1 naaaa = is a

local maximum for f iff the Hessian matrix at *a , *)(aH , is negative semi-

definite (or negative definite); a stationary point *a is a local minimum for f iff*)(aH is positive semi-definite (or positive definite).

4


5/9

(Note that if *)(aH is indefinite, the stationary point *a is neither a local

maximum nor a local minimum.)

Proof: Here, we only look at the condition for local maximum, the condition for local

minimum can be obtained similarly. To show that *a is a local maximum, one has

to show that values of function in the neighbourhood of *a , )*( haf + where h

is small, are smaller or equal to the value of the function at *a , *)(af .

Since f is a 2C function, we can use the Taylors formula to expand )*( haf + at

point *a to obtain

)(*)(2

1*)(*)()*(

3

hOhaHhafhafhaf

TT

+++=+Because *a is a stationary point, 0*)( = af . The above equation is simplified

to

)0as(*)(2

1

)(*)(2

1*)()*(

3

=

+=+

hhaHh

hOhaHhafhaf

T

T

So *)()*( afhaf + iff 0*)( haHhT , i.e. *)(aH is negative semi-definite.QED

Example 8: Find and classify stationary points of 4/4/),( 44 yxyxyxf += .

The stationary points require

=+

==

+

=)2(0

)1(00),(

3

3

3

3

yx

yx

yx

yxyxf

From (2), we have3yx = (3)

Substitution of (3) into (1) yields

0)1( 56 == yyyy (4)

Equation (4) has two real solutions 0*1 =y and 1*2 =y . Substituting 0

*1 =y into (3),

one obtains 0*1 =x ; and substituting 1*2 =y into (3) to obtain 1*2 =x .

The two stationary points are therefore )0,0(),( *1*1 =yx and )1,1(),(

*2

*2 =yx .

The Hessian matrix for 4/4/),( 44 yxyxyxf += is

=2

2

31

13),(

y

xyxH .

Evaluating the Hessian at )0,0(),( *1*1 =yx gives

=

01

10)0,0(H

which is indefinite. So )0,0(),( *1*1 =yx is neither a local maximum nor a local

minimum.

5


6/9

Evaluating the Hessian at )1,1(),( *2*2 =yx gives

=

31

13)1,1(H

which is positive definite. So )1,1(),( *2*2 =yx is a local minimum.

3. Convexity and Concavity

3.1 Concave and convex functions

Concave and convex functions with single variable normally have the following

shapes:

concave function convex function

Notice from the figure above that the set of points below a concave function is convex

and the set of points above a convex function is convex. This will give us the

definitions.

Definition: A function )(xf is concave iff for any x and 'x in S , and for10 + .

A function )(xf is convex iff for any x and 'x in S , and for 10


7/9

af(.) f(.)

f(a)

f(a)+x ff(a)+z f

z+a

f(z+a)

The tangent to the function f at the point a has an equation)()( afxafy

T+= . The distance between the tangent to f at a point az+ is

indicated by the heavy line. The length of the heavy line is simply

)()()( azfafzafT

++ . Now we will use Taylor's formula for the value)( azf +

zaHz

zaHzafzafafzafazfafzaf

T

TTTT

)(2

1

])(21)()([)()()()()(

++++=++

Ifz is small the distance is given by the expression 2/)( zaHzT . If0)( zaHzT the tangent always lies above the function. If 0)( zaHzT then the

tangent always lies below the function. Notice that concave functions have the

property that the tangent always lies above the function forany starting point a ,whilst convex functions have the property that the tangent always lies below the

functionfor any starting point a . So we have the following theorem.

Theorem 3: A twice continuously differentiable function f is(1) concave if the Hessian )(xH is negative semi-definite at all points x ,(2) strictly concave if the Hessian )(xH is negative definite at all points x ,(3) convex if the Hessian )(xH is positive semi-definite at all points x ,(4) strictly convex if the Hessian )(xH is positive definite at all points x .

In economics, we sometimes use concave function to represent utility, and we focus

explicitly on its indifference curves. In what follows, we provide a theorem outlining

properties of level curves associated with concave and convex functions.

Theorem 4: Letd

be any real number that a functionf

can attain.(1) If f is concave, then its upper level set )(dU is a convex set.

(2) If f is convex, then its lower level set )(dL is a convex set.

Proof: Here, we only prove the first part of the theorem, the second part can be done

similarly. Notice that the upper level set is defined as { }dxfxdU = )(:)( , i.e., allx at which the value of function is at least d . Lets choose two arbitrary points, y and 'y , in )(dU , i.e. dyf )( and dyf )'( . We have to show that the convexcombination ')1( yyz += (for 10


8/9

or dzf )( and z is in )(dU . So )(dU is a convex set.QED

3.2 Quasi-concave and quasi-convex functions

A standard consumer problem --- maximising utility subject to budget constraints---would have a solution if the utility function has the property that its upper level set

is convex. Such utility function may arise from a concave function as Theorem 4

suggests. It may also arise from a function which is not concave. Pictures below

provide one example:

The 3-D surface depicted on the left panel shows a concave function. The upper level

set it generates is convex. The function shown on the right panel generates the same

upper level set, but its 3-D shape is very different from that on the left. It is obvious

that the function on the right panel is notconcave. A function which generates upper

level sets the same as some concave function is called quasi-concave, and a function

which generates lower level sets the same as some convex function is called quasi-convex. Formal definition is given below.

Definition: Let d be any real number that a function f can attain.

(1) If the upper level set )(dU is a convex set, then f is quasi-concave.

(2) If the lower level set )(dL is a convex set, then f is quasi-convex.

Theorem 4 and the above definition immediately imply that a concave function must

be a quasi-concave and a convex function must be a quasi-convex. (The reverse of

this statement is usually nottrue. The pictures above are an example.) From thedefinition above it is clear that if a function f is quasi-concave then f must be

quasi-convex.

The definition introduced above can certainly be used to identify whether a function is

quasi-concave or quasi-convex. But it may sometimes involve knowing the properties

of complex level curves. A more straightforward method is to use the bordered

Hessian.

Definition: Let ),,,( 21 nxxxx = , the bordered Hessian determinants )(xBr aregiven by

8


9/9

=

)()()(

)()()(

)()(0

det)(

''''1

'

''1

''11

'1

''1

xfxfxf

xfxfxf

xfxf

xB

rrrr

r

r

r

, .,,2,1 nr =

Theorem 5: Let f be a twice continuously differentiable function defined on a

convex S in n .

(1) If f is quasi-concave, then 0)()1( xBrr for all x in S and for all

.,,2,1 nr =

(2) If 0)()1( > xBrr for all x in S and for all nr ,,2,1 = , then f is strictly

quasi-concave.

For quasi-convexity, use Theorem 5 to f .

Example 9: Let 0>x and 0>y . Show that the function xyyxf =),( is notconcave but quasi-concave.

The Hessian of this function is

=

01

10),( yxH

which is indefinite. So the function is neither concave nor convex.

The largest bordered Hessian is given by

01

10

0

x

y

xy

with 0),()1( 211 >= yyxB and 02),()1( 2

2 >= xyyxB . So it is strictly quasi-

concave.

9

multivariate calculus and constrained 1

Documents