midterm ii review session slides 119683075631104 2

8/13/2019 Midterm II Review Session Slides 119683075631104 2

1/108

Review for Midterm II

Math 20

December 4, 2007

Announcements

Midterm I 12/6, Hall A 78:30pm

ML Office Hours Wednesday 13 (SC 323)

Old exams and solutions on website
http://find/


2/108

Outline

Rank and other Linear AlgebraLinear dependenceRank

EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem

Functions of several variables

Graphing/Contour PlotsPartial Derivatives

Differentiation

The Chain RuleImplicit Differentiation

OptimizationUnconstrained Optimization

Constrained Optimization
http://find/


3/108

Rank and other Linear AlgebraLearning Objectives

Determine whether a set of vectors is linearly independent Find the rank of a matrix
http://find/http://goback/


4/108

Linear Independence

DefinitionLet{a1, a2, . . . , an} be a set of vectors in Rm. We say they arelinearly dependent if there exist constants c1, c2, . . . , ck R, notall zero, such that

c1a1+c2a2+ +ckan =0.

If the equation only holds when all c1 =c2 = =cn = 0, thenthe vectors are said to be linearly independent.
http://find/


5/108

Deciding linear dependence

We showed

a1, . . . , an LD c1a1+ +cnan=0 has a nonzero soln

a1 . . . an A

c1...

cn

c

=0 has a nonzero soln

system has some free variables rref(A) has a column with no leading entry to it
http://find/


6/108

Example

Determine if the vectors

101 ,

3

22 ,

0

21

are linearly dependent.
http://find/


7/108

Example


101 ,

3

22 ,

0

21


Solution

1 3 0

0 2 2

1 2 1

1

+

1 3 00 2 2

0 1 1

1 3 0

0 1 10 0 0

3

+

1 0 3

0 1 10 0 0

So the vectors are linearly dependent.
http://find/


8/108

Example


101 ,

3

22 ,

0

21


Solution

1 3 0

0 2 2

1 2 1

1

+

1 3 00 2 20

1 1

1 3 0

0 1 10 0 0

3

+

1 0 3

0 1 10 0 0

So the vectors are linearly dependent.
http://find/


9/108

Deciding linear independence

So

a1, . . . , an LI

every column of rref(A) has a leading entry to it

A

InO


10/108

Example


1

011

,3

220

,0

211



11/108

Solution

1 3 0

0 2 21 2 1

1 0 1

1 0 3

0 1 10 0 0

1 0 1

1

+

1 0 3

0 1 10 0 0

0 0 2

1 0 0

0 1 0

0 0 1

0 0 0

So the vectors are linearly independent.
http://find/


12/108

Rank

Definition

The rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A.
http://find/


13/108

Rank

Definition

The rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A. IfA is a zero matrix, wesayr(A) = 0.
http://find/


14/108

Example

Since

rref1 3 0

0 2 21 2 1 = 1 0 3

0 1 10 0 0 this matrix has rank 2.
http://goforward/http://find/http://goback/


15/108

Example

Since

rref

1 3 00

2 2

1 2 11 0 1

=

1 0 00 1 0

0 0 10 0 0

this matrix has rank 3.
http://find/


16/108

Another way to compute rank

Theorem

Book Theorem 14.1 The rank ofA is the size of the largestnonvanishing minor ofA.
http://find/


17/108

Rank and consistency

FactLetA be an m

n matrix, b an n

1vector, andAb the matrixA

augmented byb.Then the system of linear equationsAx= b has a solution (isconsistent) if and only if r(A) =r(Ab).
http://find/


18/108

Rank and redundancy

FactLetA be an m n matrix, b an n 1vector, andAb the matrixAaugmented byb. Suppose that r(A) =r(Ab) =k


19/108

Rank and freedom

FactLetA be an m n matrix, b an n 1vector, andAb the matrixAaugmented byb. Suppose that r(A) =r(Ab) =k


20/108

Outline





Differentiation


OptimizationUnconstrained OptimizationConstrained Optimization

Ei b i
http://find/


21/108

EigenbusinessLearning Objectives

Determine if a vector is an eigenvalue of a matrix

Determine if a scalar is an eigenvalue of a matrix

Find all the eigenvalues of a matrix Find all the eigenvectors of a matrix for a given eigenvalue

Diagonalize a matrix

Know when a matrix is diagonalizable

Ei b i
http://find/


22/108

Eigenbusiness

DefinitionLet A be an n n matrix. The number is called an eigenvalueofA if there exists a nonzero vector x

R

n such that

Ax= x. (1)

Every nonzero vector satisfying (1) is called an eigenvectorofAassociated with the eigenvalue .

E a le
http://find/


23/108

ExampleMidterm II, Fall 2006, Problem 4

Let A=4 2

1 1

Problem

Is21 an eigenvector for A?

Example
http://find/


24/108

ExampleMidterm II, Fall 2006, Problem 4

Let A=4 2

1 1

Problem

Is21 an eigenvector for A?

SolutionUse the definition of eigenvector:

4 2

1 1 2

1 = 6

3 = 32

1So the vector is an eigenvector corresponding to the eigenvalue3.


25/108

Let A= 4 21 1

ProblemIs0 an eigenvalue forA?
http://find/


26/108

Let A= 4 21 1

ProblemIs0 an eigenvalue forA?

Solution

The number0 is an eigenvalue forA if and only if the determinantofA 0I=A is zero. But

det A= 4 1 1 (2) = 6.

So its not.

Methods
http://find/


27/108

Methods

To find the eigenvalues of a matrix A, find the determinant ofA I. This will be a polynomial in (called thecharacteristic polynomial ofA, and its roots are theeigenvalues.

To find the eigenvector(s) of a matrix corresponding to aneigenvalue , do Gaussian Elimination on A I.

Diagonalization Procedure
http://find/


28/108

Diagonalization Procedure

Find the eigenvalues and eigenvectors.

Arrange the eigenvectors in a matrix P and the corresponding

eigenvalues in a diagonal matrix D. If you have enough eigenvectors so that the matrix P is

square and invertible, the original matrix is diagonalizable andequal to PDP1.

Example
http://find/


29/108

Example

ProblemLetA=

2 32 1

. Diagonalize.

Example
http://find/


30/108

Example

ProblemLetA=

2 32 1

. Diagonalize.

Solution

To find the eigenvalues, find the characteristic polynomial and itsroots:

|A I| =2 3

2 1

= (2 )(1 ) 6

=2

3 4 = ( + 1)( 4)So the eigenvalues are1 and4.
http://find/


31/108

To find an eigenvector corresponding to the eigenvalue1,

A+I=

3 32 2

1 10 0

So

11

is an eigenvector.
http://find/


32/108

To find an eigenvector corresponding to the eigenvalue 4,

A 4I=2 3

2 3

1 3/20 0

So

32

is an eigenvector.


33/108

LetP=

1 31 2

so

P1 =1

5 2 31 1

Then

A=

1 31 2

1 00 4

1

5

2 31 1

The Spectral Theorem
http://find/


34/108

p

Theorem (Baby Spectral Theorem)SupposeAnn has n distinct real eigenvalues. Then A isdiagonalizable.

The Spectral Theorem
http://find/


35/108

p

Theorem (Baby Spectral Theorem)SupposeAnn has n distinct real eigenvalues. Then A isdiagonalizable.

Theorem (Spectral Theorem for Symmetric Matrices)

SupposeAnn is symmetric, that is, A =A. Then A is

diagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means thatP1 =P.Thus a symmetric matrix can be diagonalized as

A= PDP,

Outline
http://find/


36/108





Differentiation



http://find/


37/108

Learning Objectives

identify functions, graphs, and contour plots

find partial derivatives of functions of several variables

Types of functions
http://find/


38/108

linear

polynomial

rational Cobb-Douglas

etc.

Examples
http://find/


39/108

ProblemIn each of the following, find the domain and range of the function.Is it linear? polynomial? rational? algebraic? Cobb-Douglas?

(a) f(x, y) =y x(b) f(x, y) = y x(c) f(x, y) = 4x2 + 9y2

(d) f(x, y) =x2 y2(e) f(x, y) =xy

(f) f(x, y) =y/x2

(g) f(x, y) = 116 x2 y2

(h) f(x, y) =

9 x2 y2

(i) f(x, y) = ln(x2 +y2)

(j) f(x, y) =e(x2+y2)

(k) f(x, y) = arcsin(y

x)

(l) f(x, y) = arctanyx

Graphing/Contour Plots
http://find/


40/108

A function of two variables can be visualized by

its graph: the surface (x, y, f(x, y) in R3

a contour plot: a collection of level curves
http://find/


41/108

Example

Graph and contour plot off(x, y) =y

x


42/108

Example

Graph and contour plot off(x, y) =y

x

2

1

0

1

2 2

1

0

1

2

4

2

0

2

4

2 1 0 1 2

2

1

0

1

2
http://find/


43/108

Example

Graph and contour plot off(x, y) =

y

x
http://find/


44/108

Example


y

x

2

1

0

1

22

1

0

1

2

0.0

0.5

1.0

1.5

2.0

2 1 0 1 2

2

1

0

1

2
http://find/


45/108

Example

Graph and contour plot off(x, y) = 4x2 + 9y2
http://find/


46/108

Example

Graph and contour plot off(x, y) = 4x2 + 9y2

2

1

0

1

2 2

1

0

1

2

0

20

40

2 1 0 1 2

2

1

0

1

2
http://find/


47/108

Example

Graph and contour plot off(x, y) =x2

y2
http://find/


48/108

Example

Graph and contour plot off(x, y) =x2

y2

2

1

0

1

2 2

1

0

1

2

4

2

0

2

4

2 1 0 1 2

2

1

0

1

2
http://find/


49/108

Example

Graph and contour plot off(x, y) =xy
http://find/


50/108

Example

Graph and contour plot off(x, y) =xy

2

1

0

1

2 2

1

0

1

2

4

2

0

2

4

2 1 0 1 2

2

1

0

1

2
http://find/


51/108

Example

Graph and contour plot off(x, y) = yx2
http://find/


52/108

Example

Graph and contour plot off(x, y) = yx2

2

1

0

1

2 2

1

0

1

2

5

0

5

2 1 0 1 2

2

1

0

1

2

E ample


53/108

Example

Graph and contour plot off(x, y) = 1

16 x2

y2
http://find/


54/108


55/108

Example


9 x2 y2

E l
http://find/


56/108

Example


9 x2 y2

2

0

2

2

0

2

0

1

2

3

3 2 1 0 1 2 3

3

2

1

0

1

2

3

E l
http://find/


57/108

Example

Graph and contour plot off(x, y) = ln(x2 +y2)

E l
http://find/


58/108

Example

Graph and contour plot off(x, y) = ln(x2 +y2)

2

0

2

2

0

2

1

0

1

2

3 2 1 0 1 2 3

3

2

1

0

1

2

3

E l
http://find/


59/108

Example

Graph and contour plot off(x, y) =e(x2+y2)

Example
http://find/


60/108

Example

Graph and contour plot off(x, y) =e(x2+y2)

2

0

2

2

0

2

0.0

0.5

1.0

3 2 1 0 1 2 3

3

2

1

0

1

2

3

Example
http://find/


61/108

Example

Graph and contour plot off(x, y) = arcsin(y x)

Example
http://find/


62/108

Example

Graph and contour plot off(x, y) = arcsin(y x)

2

1

0

1

22

1

0

1

2

1

0

1

2 1 0 1 2

2

1

0

1

2

Example
http://find/


63/108

Example

Graph and contour plot off(x, y) = arctan

yx

Example
http://find/


64/108

Example

Graph and contour plot off(x, y) = arctan

yx

2

1

0

1

22

1

0

1

2

1

0

1

2 1 0 1 2

2

1

0

1

2

The process
http://find/


65/108

To differentiate a function of several variables with respect to one

of the variables, pretend that the others are constant.

Examples
http://find/


66/108

Example

Letf(x, y) = 3x+ 2xy2 2y4. Find both the partial derivatives off.

Examples
http://find/


67/108

Example

Letf(x, y) = 3x+ 2xy2 2y4. Find both the partial derivatives off.

SolutionWe have

f

x= 3 + 2y2

f

y = 4xy 8y3
http://find/


68/108

ExampleLet w= sin cos. Find both the partial derivatives ofw.
http://find/


69/108

ExampleLet w= sin cos. Find both the partial derivatives ofw.

SolutionWe have

w

= cos cos

w

= sin sin

Example
http://find/


70/108

Let f(u, v) = arctan(u/v). Find both the partial derivatives off.

Example


71/108

Let f(u, v) = arctan(u/v). Find both the partial derivatives off.

SolutionFor this its important to remember the chain rule!

f

u =

1

1 + (u/v)2

u

u

v =

1

1 + (u/v)21

v

fu

= 1

1 + (u/v)2v

uv

= 1

1 + (u/v)2uv2

Another way to write this is

fu

= vu2 +v2

fv

= uu2 +v2

Example
http://find/


72/108

Example

Let u= x21 +x22 + +x2n . Find all the derivatives ofu.

Example
http://find/


73/108

Example

Let u= x21 +x22 + +x2n . Find all the derivatives ofu.

SolutionWe have a partial derivative for each index i, but luckily theyresymmetric. So each derivative is represented by:

uxi

= 1

2

x21 +x22 + +x2n

xi

(x21 +x22 + +x2n )

= xi

x21 +x22 + +x2n
http://find/


74/108

Example

Let f(x, y) = 3x+ 2xy2 2y4. Find all the second derivatives.
http://find/


75/108

Example

Let f(x, y) = 3x+ 2xy2 2y4. Find all the second derivatives.Solution

2

fx2

= 0 2

fxy

= 4y

2f

yx = 4y

2f

y2 = 24y2

Tangent Planes
http://find/


76/108

FactThe tangent plane to z=f(x, y) through (x0, y0, z0 =f(x0, y0))has normal vector(f1(x0, y0), f

2(x0, y0),1) and equation

f1(x0, y0)(x x0) +f2(x0, y0)(y y0) (z z0) = 0

Tangent Planes
http://find/


77/108



f1(x0, y0)(x x0) +f2(x0, y0)(y y0) (z z0) = 0

orz=f(x0, y0) +f

1(x0, y0)(x x0) +f2(x0, y0)(y y0)

Tangent Planes
http://find/


78/108



f1(x0, y0)(x x0) +f2(x0, y0)(y y0) (z z0) = 0

orz=f(x0, y0) +f

1(x0, y0)(x x0) +f2(x0, y0)(y y0)

This is the best linear approximation to f near (x0, y0).

is is the first-degree Taylor polynomial (in two variables) for f.

Outline
http://find/


79/108





DifferentiationThe Chain RuleImplicit Differentiation


Fact (The Chain Rule, version I)

When z = F (x , y ) with x = f (t) and y = g (t), then
http://find/


80/108

When z F(x, y) with x f(t) and y g(t), then

z

(t) =F

1(f(t), g(t))f

(t) +F

2(f(t), g(t))g

(t)

or

dz

dt =

F

x

dx

dt +

F

y

dy

dt

Fact (The Chain Rule, version I)

When z=F(x, y) with x=f(t) and y=g(t), then


81/108

e ( , y ) t (t) a d y g (t), t e

z

(t) =F

1(f(t), g(t))f

(t) +F

2(f(t), g(t))g

(t)

or

dz

dt =

F

x

dx

dt +

F

y

dy

dt

We can generalize to more variables, too. IfF is a function ofx1, x2, . . . , xn, and each xi is a function oft, then

dzdt

= Fx1

dx1dt

+ Fx2

dx2dt

+ + Fxn

dxndt

Tree Diagrams for the Chain Rule

F
http://find/


82/108

F

x

t

dxdt

Fx

y

t

dydt

Fy

To differentiate with respect to t, find all leaves marked t.Going down each branch, chain (multiply) all the derivativestogether. Then add up the result from each branch.

dz

dt =

dF

dt =

F

x

dx

dt +

F

y

dy

dt

Fact (The Chain Rule, Version II)

When z=F(x, y) with x=f(t, s) and y=g(t, s), then

F F
http://find/


83/108

z

t

=F

x

x

t

+F

y

y

tz

s =

F

x

x

s +

F

y

y

s

F

x

t s

y

t s

ExampleSupposez=xy2, x=t+s and y=t s. Find z

t and z

s at

(t, z) = (1/2, 1) in two ways:

(i) B i di l i f d b f
http://find/


84/108

(i) By expressing zdirectly in terms oft and s before

differentiating.(ii) By using the chain rule.

ExampleSupposez=xy2, x=t+s and y=t s. Find z

t and z

s at

(t, z) = (1/2, 1) in two ways:

(i) B i di tl i t f t d b f


85/108

(i) By expressing zdirectly in terms oft and s before

differentiating.(ii) By using the chain rule.

Solution (i)

We have

z= (t+s)(t s)2 =s3 ts2 t2s+t3

So

zt

= s2 2ts+ 3t2z

s = 3s2 2ts t2

Solution (ii)

W h
http://find/


86/108

We have

z=xy2

x=t+s y=t sSo

z

t =

z

x

x

t +

z

y

y

t

=y2 1 + 2xy 1 = (t s)2 + 2(t+s)(t s)z

s =

z

x

x

s +

z

y

y

s

=y2

1 + 2xy(

1) = (t

s)2

2(t+s)(t

s)

These should be the same as in the previous calculation.

Theorem (The Chain Rule, General Version)


87/108

Suppose that u is a differentiable function of the n variables

x1, x2, . . . , xn, and each xi is a differentiable function of the mvariables t1, t2, . . . , tm. Then u is a function of t1, t2, . . . , tm and

u

ti= u

x1

x1ti

+ u

x2

x2ti

+ + uxn

xnti

In summation notation

u

ti=

n

j=1u

xj

xjti

Implicit DifferentiationThe Big Idea
http://find/


88/108

FactAlong the level curve F(x, y) =c, the slope of the tangent line is

given by dy

dx =

dy

dx

F

= F/xF/x

= F

1(x, y)

F2(x, y)
http://find/


89/108

More than two variables

The basic idea is to close your eyes and use the chain rule:

E l


90/108

Example

Suppose a surface is given by F(x, y, z) =c. If this defines z as afunction ofx and y, find zx and z

y.

More than two variables

The basic idea is to close your eyes and use the chain rule:

E l
http://find/


91/108

Example

Suppose a surface is given by F(x, y, z) =c. If this defines z as afunction ofx and y, find zx and z

y.

SolutionSetting F(x, y, z) =c and remembering z is implicitly a function

of x and y, we get

F

x +

F

z

z

x

F

= 0 =z

x

F

= F

x

Fz

F

y +F

zzy

F

= 0 = zyF = FyFz

Tree diagram
http://find/


92/108

F

x y z

x

F

x

+F

z z

xF= 0 =

z

xF=

Fx

F

z

ExampleProblem 16.8.4
http://find/


93/108

ProblemLet D=f(r,P) denote the deman for an agricultural commoditywhen the price is P and r is the producers total advertising

expenditure. Let supply be given by S=g(w,P), where w is anindex for how favorable the weather has been. Assumegw(w,P)> 0. Equilibrium now requires f(r,P) =g(w,P).Assume that this equation defines P implicitly as a differentiablefunction of r and w. Compute Pwand comment on its sign.

Solution

We have
http://find/


94/108

f(r,P)

g(w,P)

0

f g

r w P

w

fP

Pw

f=g

gw g

PPw

f=g

Answer
http://find/


95/108

So P

w

f=g

=gw

f

P g

P

f

P0. We assumed that

g

w>0. So in this case,P

w

f=g


96/108


Eigenbusiness

Eigenvector and EigenvalueDiagonalizationThe Spectral Theorem



DifferentiationThe Chain Rule

Implicit DifferentiationOptimization

Unconstrained OptimizationConstrained Optimization

OptimizationLearning Objectives


97/108

Find the critical points of a function defined on an open set(so unconstrained)

Classify the critical points of a function Find the critical points of a function restricted to a surface

(constrained)
http://find/


98/108

Theorem (Fermats Theorem)Let f(x, y) be a function of two variables. If f has a localmaximum or minimum at(a, b), and is differentiable at (a,b), then

f

x(a, b) = 0

f

y(a, b) = 0

As in one variable, well call these points critical points.

Theorem (The Second Derivative Test)


99/108

Let f(x, y) be a function of two variables, and let(a, b) be acritical point of f . Then

If 2f

x22fy2

2fxy

2>0 and

2fx2

>0, the critical point is a

local minimum.

If 2

fx22

fy2 2fxy2 >0 and 2fx2


100/108

f(x, y) = 4xy x4 y4

Example

ProblemFind and classify the critical points of
http://find/


101/108

f(x, y) = 4xy x4 y4

SolutionWe have f

x = 4y

4x3 and f

y = 4x

4y3. Both of these are

zero when y=x3 and x=y3 So x9 =x. Since

x9 x=x(x8 1) =x(x4 + 1)(x2 + 1)(x+ 1)(x 1)

the real solutions are x= 0, x= 1, and x=

1. Thecorresponding y values are0, 1, and1. So the critical points are

(0, 0), (1, 1), (1,1)

The second derivatives are

2f

x2 = 12x2

2f

yx = 4
http://find/


102/108

2

fxy

= 4 2

fyx

= 12y2

So

H(x, y) = 43x2 11 3y2

At (0, 0), the matrix is

0 11 0

, which has determinant


103/108

2

1

0

1

2 2

1

0

1

2

30

20

10

0

2 1 0 1 2

2

1

0

1

Theorem (The Method of Lagrange Multipliers)
http://find/


104/108

Let f(x1, x2, . . . , xn) and g(x1, x2, . . . , xn) be functions of severalvariables. The critical points of the function f restricted to the setg= 0 are solutions to the equations:

f

xi

(x1, x2, . . . , xn) =g

xi

(x1, x2, . . . , xn) for each i= 1, . . . , n

g(x1, x2, . . . , xn) = 0.

Note that this is n+ 1 equations in the n+ 1 variables.x1, . . . , xn, .

ProblemFind the critical points and values of

f(x, y) =ax2 + 2bxy+cy2
http://find/


105/108

( , y ) y y

subject to the constraint that x2 +y2 = 1.

ProblemFind the critical points and values of

f(x, y) =ax2 + 2bxy+cy2
http://find/


106/108

( )

subject to the constraint that x2 +y2 = 1.

SolutionWe have

fx =g

x = 2ax+ 2by=(2x)fy =g

y = 2bx+ 2cy=(2y)

So the critical points happen when

a bb c

xy

=

xy

The critical values are


107/108

f(x, y) =

x ya b

b c

xy

=

x y

xy

=(x2 +y2) =

The critical values are


108/108

f(x, y) =

x ya b

b c

xy

=

x y

xy

=(x2 +y2) =

So

The critical points are eigenvectors!

The critical values are eigenvalues!
http://find/

midterm ii review session slides 119683075631104 2

Documents