midterm ii review session slides
DESCRIPTION
These are the slides from the review session. THE FILE IS BIG AND MAY HAVE BEEN CORRUPTED. IF YOU CAN'T SEE IT THROUGH THE FLASH INTERFACE, JUST CLICK THE "DOWNLOAD" LINK and view it on your own computer.TRANSCRIPT
Review for Midterm II
Math 20
December 4, 2007
Announcements
I Midterm I 12/6, Hall A 7–8:30pm
I ML Office Hours Wednesday 1–3 (SC 323)
I Old exams and solutions on website
Outline
Rank and other Linear AlgebraLinear dependenceRank
EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem
Functions of several variables
Graphing/Contour PlotsPartial Derivatives
DifferentiationThe Chain RuleImplicit Differentiation
OptimizationUnconstrained OptimizationConstrained Optimization
Rank and other Linear AlgebraLearning Objectives
I Determine whether a set of vectors is linearly independent
I Find the rank of a matrix
Linear Independence
DefinitionLet {a1, a2, . . . , an} be a set of vectors in Rm. We say they arelinearly dependent if there exist constants c1, c2, . . . , ck ∈ R, notall zero, such that
c1a1 + c2a2 + · · ·+ ckan = 0.
If the equation only holds when all c1 = c2 = · · · = cn = 0, thenthe vectors are said to be linearly independent.
Deciding linear dependence
We showed
a1, . . . , an LD ⇐⇒ c1a1 + · · ·+ cnan = 0 has a nonzero sol’n
⇐⇒(a1 . . . an
)︸ ︷︷ ︸A
c1...
cn
︸ ︷︷ ︸
c
= 0 has a nonzero sol’n
⇐⇒ system has some free variables
⇐⇒ rref(A) has a column with no leading entry to it
Example
Determine if the vectors101
,
3−22
,
021
are linearly dependent.
Solution
1 3 0
0 − 2 2
1 2 1
←−
−1
+
1 3 00 −2 20 −1 1
1 3 0
0 1 − 1
0 0 0
←−−3
+
1 0 3
0 1 − 1
0 0 0
So the vectors are linearly dependent.
Example
Determine if the vectors101
,
3−22
,
021
are linearly dependent.
Solution
1 3 0
0 − 2 2
1 2 1
←−
−1
+
1 3 00 −2 20 −1 1
1 3 0
0 1 − 1
0 0 0
←−−3
+
1 0 3
0 1 − 1
0 0 0
So the vectors are linearly dependent.
Example
Determine if the vectors101
,
3−22
,
021
are linearly dependent.
Solution
1 3 0
0 − 2 2
1 2 1
←−
−1
+
1 3 00 −2 20 −1 1
1 3 0
0 1 − 1
0 0 0
←−−3
+
1 0 3
0 1 − 1
0 0 0
So the vectors are linearly dependent.
Deciding linear independence
So
a1, . . . , an LI ⇐⇒ every column of rref(A) has a leading entry to it
⇐⇒ A ∼(
InO
)
Example
Determine if the vectors1011
,
3−220
,
021−1
are linearly dependent.
Solution
1 3 0
0 − 2 2
1 2 1
− 1 0 1
1 0 3
0 1 −1
0 0 0
− 1 0 1
←−
−1
+
1 0 3
0 1 −1
0 0 0
0 0 − 2
1 0 0
0 1 0
0 0 1
0 0 0
So the vectors are linearly independent.
Rank
DefinitionThe rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A.
If A is a zero matrix, wesay r(A) = 0.
Rank
DefinitionThe rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A. If A is a zero matrix, wesay r(A) = 0.
Example
Since
rref
1 3 00 −2 21 2 1
=
1 0 30 1 −10 0 0
this matrix has rank 2.
Example
Since
rref
1 3 00 −2 21 2 1−1 0 1
=
1 0 00 1 00 0 10 0 0
this matrix has rank 3.
Another way to compute rank
TheoremBook Theorem 14.1 The rank of A is the size of the largestnonvanishing minor of A.
Rank and consistency
FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b.Then the system of linear equations Ax = b has a solution (isconsistent) if and only if r(A) = r(Ab).
Rank and redundancy
FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b. Suppose that r(A) = r(Ab) = k < m (m is thenumber of equations in the system Ax = b).Then m − k of the equations are redundant; they can be removedand the system has the same solutions.
Rank and freedom
FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b. Suppose that r(A) = r(Ab) = k < n (n is thenumber of variables in the system Ax = b).Then n − k of the variables are free; they can be chosen at willand the rest of the variables depend on them, getting infinitelymany solutions.
Outline
Rank and other Linear AlgebraLinear dependenceRank
EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem
Functions of several variables
Graphing/Contour PlotsPartial Derivatives
DifferentiationThe Chain RuleImplicit Differentiation
OptimizationUnconstrained OptimizationConstrained Optimization
EigenbusinessLearning Objectives
I Determine if a vector is an eigenvalue of a matrix
I Determine if a scalar is an eigenvalue of a matrix
I Find all the eigenvalues of a matrix
I Find all the eigenvectors of a matrix for a given eigenvalue
I Diagonalize a matrix
I Know when a matrix is diagonalizable
Eigenbusiness
DefinitionLet A be an n × n matrix. The number λ is called an eigenvalueof A if there exists a nonzero vector x ∈ Rn such that
Ax = λx. (1)
Every nonzero vector satisfying (1) is called an eigenvector of Aassociated with the eigenvalue λ.
ExampleMidterm II, Fall 2006, Problem 4
Let A =
(4 −21 1
)Problem
Is
(21
)an eigenvector for A?
SolutionUse the definition of eigenvector:(
4 −21 1
)(21
)=
(63
)= 3
(21
)So the vector is an eigenvector corresponding to the eigenvalue 3.
ExampleMidterm II, Fall 2006, Problem 4
Let A =
(4 −21 1
)Problem
Is
(21
)an eigenvector for A?
SolutionUse the definition of eigenvector:(
4 −21 1
)(21
)=
(63
)= 3
(21
)So the vector is an eigenvector corresponding to the eigenvalue 3.
Let A =
(4 −21 1
)ProblemIs 0 an eigenvalue for A?
SolutionThe number 0 is an eigenvalue for A if and only if the determinantof A− 0I = A is zero. But
det A = 4 · 1− 1 · (−2) = 6.
So it’s not.
Let A =
(4 −21 1
)ProblemIs 0 an eigenvalue for A?
SolutionThe number 0 is an eigenvalue for A if and only if the determinantof A− 0I = A is zero. But
det A = 4 · 1− 1 · (−2) = 6.
So it’s not.
Methods
I To find the eigenvalues of a matrix A, find the determinant ofA− λI. This will be a polynomial in λ (called thecharacteristic polynomial of A, and its roots are theeigenvalues.
I To find the eigenvector(s) of a matrix corresponding to aneigenvalue λ, do Gaussian Elimination on A− λI.
Diagonalization Procedure
I Find the eigenvalues and eigenvectors.
I Arrange the eigenvectors in a matrix P and the correspondingeigenvalues in a diagonal matrix D.
I If you have “enough” eigenvectors so that the matrix P issquare and invertible, the original matrix is diagonalizable andequal to PDP−1.
Example
Problem
Let A =
(2 32 1
). Diagonalize.
SolutionTo find the eigenvalues, find the characteristic polynomial and itsroots:
|A− λI| =
∣∣∣∣2− λ 32 1− λ
∣∣∣∣ = (2− λ)(1− λ)− 6
= λ2 − 3λ− 4 = (λ+ 1)(λ− 4)
So the eigenvalues are −1 and 4.
Example
Problem
Let A =
(2 32 1
). Diagonalize.
SolutionTo find the eigenvalues, find the characteristic polynomial and itsroots:
|A− λI| =
∣∣∣∣2− λ 32 1− λ
∣∣∣∣ = (2− λ)(1− λ)− 6
= λ2 − 3λ− 4 = (λ+ 1)(λ− 4)
So the eigenvalues are −1 and 4.
To find an eigenvector corresponding to the eigenvalue −1,
A + I =
(3 32 2
)
(1 10 0
)
So
(1−1
)is an eigenvector.
To find an eigenvector corresponding to the eigenvalue 4,
A− 4I =
(−2 32 −3
)
(1 −3/20 0
)
So
(32
)is an eigenvector.
Let
P =
(1 3−1 2
)so
P−1 =1
5
(2 −31 1
)Then
A =
(1 3−1 2
)(−1 00 4
)1
5
(2 −31 1
)
The Spectral Theorem
Theorem (Baby Spectral Theorem)
Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.
Theorem (Spectral Theorem for Symmetric Matrices)
Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as
A = PDP′,
The Spectral Theorem
Theorem (Baby Spectral Theorem)
Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.
Theorem (Spectral Theorem for Symmetric Matrices)
Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as
A = PDP′,
Outline
Rank and other Linear AlgebraLinear dependenceRank
EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem
Functions of several variables
Graphing/Contour PlotsPartial Derivatives
DifferentiationThe Chain RuleImplicit Differentiation
OptimizationUnconstrained OptimizationConstrained Optimization
Functions of several variablesLearning Objectives
I identify functions, graphs, and contour plots
I find partial derivatives of functions of several variables
Types of functions
I linear
I polynomial
I rational
I Cobb-Douglas
I etc.
Examples
ProblemIn each of the following, find the domain and range of the function.Is it linear? polynomial? rational? algebraic? Cobb-Douglas?
(a) f (x , y) = y − x
(b) f (x , y) =√
y − x
(c) f (x , y) = 4x2 + 9y2
(d) f (x , y) = x2 − y2
(e) f (x , y) = xy
(f) f (x , y) = y/x2
(g) f (x , y) =1√
16− x2 − y2
(h) f (x , y) =√
9− x2 − y2
(i) f (x , y) = ln(x2 + y2)
(j) f (x , y) = e−(x2+y2)
(k) f (x , y) = arcsin(y − x)
(l) f (x , y) = arctan( y
x
)
Graphing/Contour Plots
A function of two variables can be visualized by
I its graph: the surface (x , y , f (x , y) in R3
I a contour plot: a collection of level curves
Example
Graph and contour plot of f (x , y) = y − x
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = y − x
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) =√
y − x
-2
-1
0
1
2-2
-1
0
1
2
0.0
0.5
1.0
1.5
2.0
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) =√
y − x
-2
-1
0
1
2-2
-1
0
1
2
0.0
0.5
1.0
1.5
2.0
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = 4x2 + 9y2
-2
-1
0
1
2 -2
-1
0
1
2
0
20
40
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = 4x2 + 9y2
-2
-1
0
1
2 -2
-1
0
1
2
0
20
40
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = x2 − y2
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = x2 − y2
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = xy
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = xy
-2
-1
0
1
2-2
-1
0
1
2
-4
-2
0
2
4
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = yx2
-2
-1
0
1
2-2
-1
0
1
2
-5
0
5
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = yx2
-2
-1
0
1
2-2
-1
0
1
2
-5
0
5
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) =1√
16− x2 − y2
-4
-2
0
2
4-4
-2
0
2
4
0.0
0.5
1.0
-4 -2 0 2 4
-4
-2
0
2
4
Example
Graph and contour plot of f (x , y) =1√
16− x2 − y2
-4
-2
0
2
4-4
-2
0
2
4
0.0
0.5
1.0
-4 -2 0 2 4
-4
-2
0
2
4
Example
Graph and contour plot of f (x , y) =√
9− x2 − y2
-2
0
2
-2
0
2
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) =√
9− x2 − y2
-2
0
2
-2
0
2
0
1
2
3
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) = ln(x2 + y2)
-2
0
2
-2
0
2
-1
0
1
2
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) = ln(x2 + y2)
-2
0
2
-2
0
2
-1
0
1
2
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) = e−(x2+y2)
-2
0
2
-2
0
2
0.0
0.5
1.0
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) = e−(x2+y2)
-2
0
2
-2
0
2
0.0
0.5
1.0
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
Example
Graph and contour plot of f (x , y) = arcsin(y − x)
-2
-1
0
1
2-2
-1
0
1
2
-1
0
1
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = arcsin(y − x)
-2
-1
0
1
2-2
-1
0
1
2
-1
0
1
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = arctan( y
x
)
-2
-1
0
1
2-2
-1
0
1
2
-1
0
1
-2 -1 0 1 2
-2
-1
0
1
2
Example
Graph and contour plot of f (x , y) = arctan( y
x
)
-2
-1
0
1
2-2
-1
0
1
2
-1
0
1
-2 -1 0 1 2
-2
-1
0
1
2
The process
To differentiate a function of several variables with respect to oneof the variables, pretend that the others are constant.
Examples
Example
Let f (x , y) = 3x + 2xy2 − 2y4. Find both the partial derivatives off .
SolutionWe have
∂f
∂x= 3 + 2y2 ∂f
∂y= 4xy − 8y3
Examples
Example
Let f (x , y) = 3x + 2xy2 − 2y4. Find both the partial derivatives off .
SolutionWe have
∂f
∂x= 3 + 2y2 ∂f
∂y= 4xy − 8y3
Example
Let w = sinα cosβ. Find both the partial derivatives of w .
SolutionWe have
∂w
∂α= cosα cosβ
∂w
∂β= − sinα sinβ
Example
Let w = sinα cosβ. Find both the partial derivatives of w .
SolutionWe have
∂w
∂α= cosα cosβ
∂w
∂β= − sinα sinβ
Example
Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .
SolutionFor this it’s important to remember the chain rule!
∂f
∂u=
1
1 + (u/v)2
∂
∂u
u
v=
1
1 + (u/v)2
1
v
∂f
∂u=
1
1 + (u/v)2
∂
∂v
u
v=
1
1 + (u/v)2
−u
v2
Another way to write this is
∂f
∂u=
v
u2 + v2
∂f
∂v=
−u
u2 + v2
Example
Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .
SolutionFor this it’s important to remember the chain rule!
∂f
∂u=
1
1 + (u/v)2
∂
∂u
u
v=
1
1 + (u/v)2
1
v
∂f
∂u=
1
1 + (u/v)2
∂
∂v
u
v=
1
1 + (u/v)2
−u
v2
Another way to write this is
∂f
∂u=
v
u2 + v2
∂f
∂v=
−u
u2 + v2
Example
Let u =√
x21 + x2
2 + · · ·+ x2n . Find all the derivatives of u.
SolutionWe have a partial derivative for each index i , but luckily they’resymmetric. So each derivative is represented by:
∂u
∂xi=
1
2√
x21 + x2
2 + · · ·+ x2n
∂
∂xi(x2
1 + x22 + · · ·+ x2
n )
=xi√
x21 + x2
2 + · · ·+ x2n
Example
Let u =√
x21 + x2
2 + · · ·+ x2n . Find all the derivatives of u.
SolutionWe have a partial derivative for each index i , but luckily they’resymmetric. So each derivative is represented by:
∂u
∂xi=
1
2√
x21 + x2
2 + · · ·+ x2n
∂
∂xi(x2
1 + x22 + · · ·+ x2
n )
=xi√
x21 + x2
2 + · · ·+ x2n
Example
Let f (x , y) = 3x + 2xy2 − 2y4. Find all the second derivatives.
Solution
∂2f
∂x2= 0
∂2f
∂x ∂y= 4y
∂2f
∂y ∂x= 4y
∂2f
∂y2= −24y2
Example
Let f (x , y) = 3x + 2xy2 − 2y4. Find all the second derivatives.
Solution
∂2f
∂x2= 0
∂2f
∂x ∂y= 4y
∂2f
∂y ∂x= 4y
∂2f
∂y2= −24y2
Tangent Planes
FactThe tangent plane to z = f (x , y) through (x0, y0, z0 = f (x0, y0))has normal vector (f ′1(x0, y0), f ′2(x0, y0),−1) and equation
f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0
orz = f (x0, y0) + f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)
I This is the best linear approximation to f near (x0, y0).
I is is the first-degree Taylor polynomial (in two variables) for f .
Tangent Planes
FactThe tangent plane to z = f (x , y) through (x0, y0, z0 = f (x0, y0))has normal vector (f ′1(x0, y0), f ′2(x0, y0),−1) and equation
f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0
orz = f (x0, y0) + f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)
I This is the best linear approximation to f near (x0, y0).
I is is the first-degree Taylor polynomial (in two variables) for f .
Tangent Planes
FactThe tangent plane to z = f (x , y) through (x0, y0, z0 = f (x0, y0))has normal vector (f ′1(x0, y0), f ′2(x0, y0),−1) and equation
f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0
orz = f (x0, y0) + f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)
I This is the best linear approximation to f near (x0, y0).
I is is the first-degree Taylor polynomial (in two variables) for f .
Outline
Rank and other Linear AlgebraLinear dependenceRank
EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem
Functions of several variables
Graphing/Contour PlotsPartial Derivatives
DifferentiationThe Chain RuleImplicit Differentiation
OptimizationUnconstrained OptimizationConstrained Optimization
Fact (The Chain Rule, version I)
When z = F (x , y) with x = f (t) and y = g(t), then
z ′(t) = F ′1(f (t), g(t))f ′(t) + F ′2(f (t), g(t))g ′(t)
or
dz
dt=∂F
∂x
dx
dt+∂F
∂y
dy
dt
We can generalize to more variables, too. If F is a function ofx1, x2, . . . , xn, and each xi is a function of t, then
dz
dt=∂F
∂x1
dx1
dt+∂F
∂x2
dx2
dt+ · · ·+ ∂F
∂xn
dxn
dt
Fact (The Chain Rule, version I)
When z = F (x , y) with x = f (t) and y = g(t), then
z ′(t) = F ′1(f (t), g(t))f ′(t) + F ′2(f (t), g(t))g ′(t)
or
dz
dt=∂F
∂x
dx
dt+∂F
∂y
dy
dt
We can generalize to more variables, too. If F is a function ofx1, x2, . . . , xn, and each xi is a function of t, then
dz
dt=∂F
∂x1
dx1
dt+∂F
∂x2
dx2
dt+ · · ·+ ∂F
∂xn
dxn
dt
Tree Diagrams for the Chain Rule
F
x
t
dxdt
∂F∂x
y
t
dydt
∂F∂y
To differentiate with respect to t, find all “leaves” marked t.Going down each branch, chain (multiply) all the derivativestogether. Then add up the result from each branch.
dz
dt=
dF
dt=∂F
∂x
dx
dt+∂F
∂y
dy
dt
Fact (The Chain Rule, Version II)
When z = F (x , y) with x = f (t, s) and y = g(t, s), then
∂z
∂t=∂F
∂x
∂x
∂t+∂F
∂y
∂y
∂t∂z
∂s=∂F
∂x
∂x
∂s+∂F
∂y
∂y
∂s
F
x
t s
y
t s
Example
Suppose z = xy2, x = t + s and y = t − s. Find ∂z∂t and ∂z
∂s at(t, z) = (1/2, 1) in two ways:
(i) By expressing z directly in terms of t and s beforedifferentiating.
(ii) By using the chain rule.
Solution (i)
We have
z = (t + s)(t − s)2 = s3 − ts2 − t2s + t3
So
∂z
∂t= −s2 − 2ts + 3t2
∂z
∂s= 3s2 − 2ts − t2
Example
Suppose z = xy2, x = t + s and y = t − s. Find ∂z∂t and ∂z
∂s at(t, z) = (1/2, 1) in two ways:
(i) By expressing z directly in terms of t and s beforedifferentiating.
(ii) By using the chain rule.
Solution (i)
We have
z = (t + s)(t − s)2 = s3 − ts2 − t2s + t3
So
∂z
∂t= −s2 − 2ts + 3t2
∂z
∂s= 3s2 − 2ts − t2
Solution (ii)
We havez = xy2 x = t + s y = t − s
So
∂z
∂t=∂z
∂x
∂x
∂t+∂z
∂y
∂y
∂t
= y2 · 1 + 2xy · 1 = (t − s)2 + 2(t + s)(t − s)
∂z
∂s=∂z
∂x
∂x
∂s+∂z
∂y
∂y
∂s
= y2 · 1 + 2xy(−1) = (t − s)2 − 2(t + s)(t − s)
These should be the same as in the previous calculation.
Theorem (The Chain Rule, General Version)
Suppose that u is a differentiable function of the n variablesx1, x2, . . . , xn, and each xi is a differentiable function of the mvariables t1, t2, . . . , tm. Then u is a function of t1, t2, . . . , tm and
∂u
∂ti=
∂u
∂x1
∂x1
∂ti+∂u
∂x2
∂x2
∂ti+ · · ·+ ∂u
∂xn
∂xn
∂ti
In summation notation
∂u
∂ti=
n∑j=1
∂u
∂xj
∂xj
∂ti
Implicit DifferentiationThe Big Idea
FactAlong the level curve F (x , y) = c, the slope of the tangent line isgiven by
dy
dx=
(dy
dx
)F
= −∂F/∂x
∂F/∂x= −F ′1(x , y)
F ′2(x , y)
Tree diagram
F
x y
x
∂F
∂x+∂F
∂y
(dy
dx
)F
= 0
More than two variables
The basic idea is to close your eyes and use the chain rule:
Example
Suppose a surface is given by F (x , y , z) = c . If this defines z as afunction of x and y , find z ′x and z ′y .
SolutionSetting F (x , y , z) = c and remembering z is implicitly a functionof x and y, we get
∂F
∂x+∂F
∂z
(∂z
∂x
)F
= 0 =⇒(∂z
∂x
)F
= −F ′xF ′z
∂F
∂y+∂F
∂z
(∂z
∂y
)F
= 0 =⇒(∂z
∂y
)F
= −F ′yF ′z
More than two variables
The basic idea is to close your eyes and use the chain rule:
Example
Suppose a surface is given by F (x , y , z) = c . If this defines z as afunction of x and y , find z ′x and z ′y .
SolutionSetting F (x , y , z) = c and remembering z is implicitly a functionof x and y, we get
∂F
∂x+∂F
∂z
(∂z
∂x
)F
= 0 =⇒(∂z
∂x
)F
= −F ′xF ′z
∂F
∂y+∂F
∂z
(∂z
∂y
)F
= 0 =⇒(∂z
∂y
)F
= −F ′yF ′z
Tree diagram
F
x y z
x
∂F
∂x+∂F
∂z
(∂z
∂x
)F
= 0 =⇒(∂z
∂x
)F
= −F ′xF ′z
ExampleProblem 16.8.4
ProblemLet D = f (r ,P) denote the deman for an agricultural commoditywhen the price is P and r is the producers’ total advertisingexpenditure. Let supply be given by S = g(w ,P), where w is anindex for how favorable the weather has been. Assumeg ′w (w ,P) > 0. Equilibrium now requires f (r ,P) = g(w ,P).Assume that this equation defines P implicitly as a differentiablefunction of r and w. Compute P ′w and comment on its sign.
Solution
We havef (r ,P)− g(w ,P) ≡ 0
f − g
r w P
w
∂f
∂P
(∂P
∂w
)f =g
− ∂g
∂w− ∂g
∂P
(∂P
∂w
)f =g
Answer
So (∂P
∂w
)f =g
=
∂g
∂w∂f
∂P− ∂g
∂P
∂f
∂P< 0 and
∂g
∂P> 0. We assumed that
∂g
∂w> 0. So in this case,(
∂P
∂w
)f =g
< 0,
meaning the price decreasing with improving weather.
Outline
Rank and other Linear AlgebraLinear dependenceRank
EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem
Functions of several variables
Graphing/Contour PlotsPartial Derivatives
DifferentiationThe Chain RuleImplicit Differentiation
OptimizationUnconstrained OptimizationConstrained Optimization
OptimizationLearning Objectives
I Find the critical points of a function defined on an open set(so unconstrained)
I Classify the critical points of a function
I Find the critical points of a function restricted to a surface(constrained)
Theorem (Fermat’s Theorem)
Let f (x , y) be a function of two variables. If f has a localmaximum or minimum at (a, b), and is differentiable at (a,b), then
∂f
∂x(a, b) = 0
∂f
∂y(a, b) = 0
As in one variable, we’ll call these points critical points.
Theorem (The Second Derivative Test)
Let f (x , y) be a function of two variables, and let (a, b) be acritical point of f . Then
I If ∂2f∂x2
∂2f∂y2 −
(∂2f
∂x∂y
)2> 0 and ∂2f
∂x2 > 0, the critical point is a
local minimum.
I If ∂2f∂x2
∂2f∂y2 −
(∂2f
∂x∂y
)2> 0 and ∂2f
∂x2 < 0, the critical point is a
local maximum.
I If ∂2f∂x2
∂2f∂y2 −
(∂2f
∂x∂y
)2< 0, the critical point is a saddle point.
All derivatives are evaluated at the critical point (a, b).
Example
ProblemFind and classify the critical points of
f (x , y) = 4xy − x4 − y4
SolutionWe have ∂f
∂x = 4y − 4x3 and ∂f∂y = 4x − 4y3. Both of these are
zero when y = x3 and x = y3 So x9 = x. Since
x9 − x = x(x8 − 1) = x(x4 + 1)(x2 + 1)(x + 1)(x − 1)
the real solutions are x = 0, x = 1, and x = −1. Thecorresponding y values are 0, 1, and −1. So the critical points are
(0, 0), (1, 1), (−1,−1)
Example
ProblemFind and classify the critical points of
f (x , y) = 4xy − x4 − y4
SolutionWe have ∂f
∂x = 4y − 4x3 and ∂f∂y = 4x − 4y3. Both of these are
zero when y = x3 and x = y3 So x9 = x. Since
x9 − x = x(x8 − 1) = x(x4 + 1)(x2 + 1)(x + 1)(x − 1)
the real solutions are x = 0, x = 1, and x = −1. Thecorresponding y values are 0, 1, and −1. So the critical points are
(0, 0), (1, 1), (−1,−1)
The second derivatives are
∂2f
∂x2= −12x2 ∂2f
∂y ∂x= 4
∂2f
∂x ∂y= 4
∂2f
∂y ∂x= −12y2
So
H(x , y) = 4
(−3x2 1
1 −3y2
)At (0, 0), the matrix is
(0 11 0
), which has determinant < 0. So
it’s a saddle point. At the other two points, the matrix is(−3 11 −3
), which has positive determinant. So those points are
local maxima.
Graph and contour plot of f (x , y) = 4xy − x4 − y4
-2
-1
0
1
2 -2
-1
0
1
2
-30
-20
-10
0
-2 -1 0 1 2
-2
-1
0
1
2
Theorem (The Method of Lagrange Multipliers)
Let f (x1, x2, . . . , xn) and g(x1, x2, . . . , xn) be functions of severalvariables. The critical points of the function f restricted to the setg = 0 are solutions to the equations:
∂f
∂xi(x1, x2, . . . , xn) = λ
∂g
∂xi(x1, x2, . . . , xn) for each i = 1, . . . , n
g(x1, x2, . . . , xn) = 0.
Note that this is n + 1 equations in the n + 1 variables.x1, . . . , xn, λ.
ProblemFind the critical points and values of
f (x , y) = ax2 + 2bxy + cy2
subject to the constraint that x2 + y2 = 1.
SolutionWe have
f ′x = λg ′x =⇒ 2ax + 2by = λ(2x)
f ′y = λg ′y =⇒ 2bx + 2cy = λ(2y)
So the critical points happen when(a bb c
)(xy
)= λ
(xy
)
ProblemFind the critical points and values of
f (x , y) = ax2 + 2bxy + cy2
subject to the constraint that x2 + y2 = 1.
SolutionWe have
f ′x = λg ′x =⇒ 2ax + 2by = λ(2x)
f ′y = λg ′y =⇒ 2bx + 2cy = λ(2y)
So the critical points happen when(a bb c
)(xy
)= λ
(xy
)
The critical values are
f (x , y) =(x y
)(a bb c
)(xy
)=(x y
)λ
(xy
)= λ(x2 + y2) = λ
So
I The critical points are eigenvectors!
I The critical values are eigenvalues!
The critical values are
f (x , y) =(x y
)(a bb c
)(xy
)=(x y
)λ
(xy
)= λ(x2 + y2) = λ
So
I The critical points are eigenvectors!
I The critical values are eigenvalues!