linear algebra: lecture notes from kolman and hill 9th edition.taylan.sengul/files/...linear...
TRANSCRIPT
Linear Algebra: Lecture notes from Kolman
and Hill 9th edition.
Taylan Sengül
April 6, 2020
Please let me know of any mistakes in these notes.
Contents
Week 1-2 31.1 Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . 31.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.4 Algebraic Properties of Matrix Operations . . . . . . . . . . . . . . . 101.5 Special Types of Matrices and Partitioned Matrices . . . . . . . . . . 11
Week 3-4 162.1 Echelon Form of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 16
1
2.2 Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 Elementary Matrices; Finding A−1 . . . . . . . . . . . . . . . . . . . 24
Week 5-6 283.1 Definition of Determinant . . . . . . . . . . . . . . . . . . . . . . . . . 283.2 Properties of Determinant . . . . . . . . . . . . . . . . . . . . . . . . 303.3 Cofactor Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.4 Inverse of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.5 Other Applications of Determinants . . . . . . . . . . . . . . . . . . . 44
Week 7 47
Week 8 514.1 Vectors in the plane and in 3-space . . . . . . . . . . . . . . . . . . . 514.2 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524.3 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Week 9 584.4 Span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.5 Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Week 10 65
Week 11 654.6 Basis and Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.7 Homogeneous Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 686.1 Linear Transformations and Matrices. Definition and Examples . . . 70
Week 12 72
2
7.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 72
Week 1-2
1.1 Systems of Linear Equations
1. Let a1, a2, . . . , an, b be constant numbers. The equation
a1x1 + a2x2 + · · · anxn = b
is called a linear equation with n unknowns x1, x2, . . . , xn.
2. A system of m linear equations in n unknowns is
a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...an1x1 + am2x2 + · · ·+ amnxn = bm
If the linear system has no solution, it is called inconsistent; if it has asolution it is called consistent. A linear system is called homogeneous ifb1 = b2 = · · · = bm = 0; otherwise it is called non-homogeneous.Example. 3x1 + 2x2 = 6 is a consistent, non-homogenous linear systemsince it has a solution x1 = 2, x2 = 0.Example. Write a system which is inconsistent.
3. A homogeneous system is always consistent because x1 = x2 = · · · = xn =
0 is a solution.
3
4. Two linear systems are called equivalent if they both exactly have thesame solutions.
5. To find a solution to a linear system, we use the method of elimination.Example. The linear system
x− 3y = −72x− 6y = 7
is inconsistent. To see, eliminate x from the second eq. to obtain
0 = 21
Example.x+ 2y + 3z = 6
2x− 3y + 2z = 14
3x+ y − z = −2
Eliminate x from the second and third equations by using the first equa-tion.
−7y − 4z = 2
−5y − 10z = −20
Eliminate y from the second eq.
10z = 30 =⇒ z = 3
Substitute to find y = −2 and x = 1.
4
By the elimination procedure we get the equivalent system
x+ 2y + 3z = 6
y + 2z = 4
z = 3
This system has a unique solution.Example.
x+ 2y − 3z = −42x+ y − 3z = 4
Eliminate x,−3y + 3z = 12
This system has solutions of the form
x = z + 4
y = z − 4
z = any real number
This system has infinitely many solutions.
6. These examples suggest that a linear system may have a unique solution,no solution or infinitely many solutions.
7.a1x+ a2y = c1b1x+ b2y = c2
The graph of each equation is a line. A solution must lie on both lines:
5
(a) If lines coincide: Infinitely many solutions.
(b) If lines intersect at exactly 1 point: unique solution.
(c) If lines are parallel and do not intersect: no solution.
8.a1x+ b1y + c1z = d1a2x+ b2y + c2z = d2a3x+ b3y + c3z = d3
The graph of each equation is a plane.
(a) If all 3 planes coincide: infinitely many solution.
(b) If planes intersect on a line: infinitely many solutions.
(c) If planes intersect at exactly 1 point: unique solution.
(d) If two of the planes are parallel and do not intersect: no solution.
6
9. Unlike system of linear equations, system of nonlinear equations may havemore than one but less then infinite solutions. Think about the intersectionof two circles.
10. Exercises 1.1: 1-23
1.2 Matrices
• Define: an m × n matrix is a rectangular array of mn real (or complex)numbers in m rows and n columns. The ith row of a matrix, the jth columnof a matrix. (i, j)- entry of a matrix.
• If m = n the matrix is called a square matrix. Main diagonal of a squarematrix: a11, a22, . . . , amm.
• An n× 1 matrix is called an n-vector.
• When are two matrices equal A = [aij ] and B = [bij ]? If they have the samesize and aij = bij for all i = 1, . . . ,m, j = 1, . . . , n.
7
• Matrix addition.
• Scalar multiplication of a matrix. Difference of two matrices.
• Linear combination of k matrices of size m× n.
• Transpose of a matrix.
• Exercises 1.2. 5, 7, 9, 11, 13
1.3 Matrix Multiplication
• Dot product of two n-vectors.Example.
Let a =
x
2
3
and b =
4
1
2
. If a · b = −4, find x
Answer: x = −3.
• Matrix multiplication. Which matrices can be multiplied? If Am×n andBp×r then AB is defined only if n = p.
(AB)ij =
n∑k=1
aikbkj
• If the matrix product AB is defined, BA may not be defined. If it is definedit may not be equal to AB.
8
• Matrix vector product.
Ac =
a11 a12 · · · a1na21 a22 · · · a2n...
......
am1 am2 · · · amn
c1c2...cn
=
(row1(A))
T · c(row2(A))
T · c...
(rowm(A))T · c
Ac = c1
a11a21...
am1
+ c2
a12a22...
am2
+ · · ·+ cn
a1na2n
...amn
Ac = c1 col1(A) + c2 col2(A) + · · ·+ cn coln(A)
• Express the linear system
a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2
...an1x1 + am2x2 + · · ·+ amnxn = bm
in the matrix formAx = b
with
A =
a11 a12 · · · a1na21 a22 · · · a2n...
......
am1 am2 · · · amn
, x =
x1x2·xn
, b =
b1b2...bm
9
The matrix A is called the coefficient matrix.
• Augmented matrix.
• Exercises 1.3: 1, 5, 9, 11, 15, 17, 19, 23, 30, 33, 37, 41, 51
1.4 Algebraic Properties of Matrix Operations
• Properties of matrix addition. If A, B, C are m×n matrices then (i) A+B =
B +A, (ii) A+ (B +C) = (A+B) +C, (iii) there is a unique zero matrix O,(iv) A+ (−A) = (−A) +A = O where −A = −1A.
• Properties of matrix multiplication. If A, B, C are matrices of appropriatesizes then (i)(AB)C = A(BC), (ii) A(B + C) = AB + AC, (iii) (A + B)C =
AC +BC.
• Properties of scalar multiplication. Theorem 1.3: If r, s are real numbersand A and B are matrices of appropriate sizes then (i) r(sA) = (rs)A, (ii)(r + s)A = rA+ sA, (iii) r(A+B) = rA+ rB, (iv) A(rB) = r(AB) = (rA)B.
• Properties of transpose. Theorem 1.4: If r is a scalar, A and B are matrices
of appropriate sizes then (i)(AT)T
= A, (ii) (A + B)T = AT + BT , (iii)(AB)T = BTAT , (iv) (rA)T = rAT . proof for (AB)T = BTAT .
(AB)Tij = (AB)ji =∑k
ajkbki =∑k
bTikaTkj = (BTAT )ij
Matrix multiplication has same properties unlike usual multiplication.
10
Example. If AB = 0 then it may happen that A 6= 0 and B 6= 0.
A =
[1 2
2 4
]B =
[4 −6−2 3
]AB =
[0 0
0 0
]Example. If AB = AC then it may happen that B 6= C.
A =
[1 2
2 4
]B =
[2 1
3 2
]C =
[−2 7
5 −1
]Then
AB = AC =
[8 5
16 10
]Thus for matrices of appropriate sizes
1. AB need not equal BA
2. AB may be the zero matrix with A 6= O and B 6= O
3. AB may equal AC with B 6= C.
• Exercises 1.4: 22, 30, 31, 32.
1.5 Special Types of Matrices and Partitioned Matrices
• A square matrix is called a diagonal matrix if its entries satisfy aij = 0 ifi 6= j. A scalar matrix is a diagonal matrix whose diagonal elements are
11
equal. Identity matrix In is a scalar matrix with diagonal elements equalto 1.
• An n× n square matrix is called the identity matrix In if its entries satisfy
aij =
{0, i 6= j
1, i = j
If A is m× n:AIn = A, and ImA = A
• For a square matrix, define powers of A
A0 = In, Ap = AA · · ·A︸ ︷︷ ︸p times
, p ≥ 1
Example. Let A be a diagonal matrix with diagonal entries {2,−3, 5}.Compute A4.
By definition, we set A0 = In. For any non-negative integers p, q
ApAq = Ap+q, (Ap)q = Apq
Note that in general,(AB)p 6= ApBp
HoweverAB = BA =⇒ (AB)p = ApBp = (BA)p
• An n × n matrix is called upper triangular if its entries satisfy aij = 0 ifi > j, that is all its entries below the main diagonal are zero. Similarly wecan define lower triangular matrices.
12
• A square matrix is symmetric if AT = A and skew-symmetric if AT =
−A. Write the general form of 3× 3 symmetric and skew symmetric matri-ces.Example. Any square matrix A can be written as a sum of symmetric andskew-symmetric matrices. This decomposition is unique.(
1 2
3 4
)= S +K
Write S =(a bb c
)and K =
(0 d−d 0
)and solve for a, b, c, d to find a = 1, b = 5/2,
c = 4, d = −1/2.
• Partitioned matrices. We skip this topic for now.
• If for an n × n matrix A there exists an n × n matrix B such that AB =
BA = In, then A called a nonsingular or invertible matrix and B iscalled an inverse of A. If A is not invertible then it is called singular ornon-invertible.
• The inverse matrix is unique. proof. Suppose
AB = BA = In and AC = CA = In
ThenB = BIn = B(AC) = (BA)C = InC = C.
Because of uniqueness, we write the inverse of a matrix as A−1.
• Example. Find the inverse of
A =
[1 2
3 4
]
13
Solution.
AA−1 =
[1 2
3 4
] [a b
c d
]= I2 =
[1 0
0 1
][a+ 2c b+ 2d
3a+ 4c 3b+ 4d
]=
[1 0
0 1
]Solve
a+ 2c = 1
2a+ 4c = 0
b+ 2d = 0
2b+ 4d = 1
to get
A−1 =
[−2 1
32 − 1
2
]• Theorem 1.6. If A and B have inverses, then so does AB. And its inverse
is (AB)−1 = B−1A−1. proof Check the inverse identity from left and right.
• Corollary 1.1. In general if A1, . . . , An are invertible then so is the product
matrix A1 · · ·An and (A1 · · ·An)−1 = A−1n · · ·A−11 .
• Theorem 1.7. (A−1)−1 = A proof. This is evident from the identity AA−1 =
A−1A = I.
• Theorem 1.8. (A−1)T = (AT )−1. proof.(A−1
)TAT = ITn = In and
(AT) (A−1
)T=
In
14
• Linear Systems and Inverses. Suppose A is a nonsingular matrix and Ax =
b.A−1(Ax) = A−1b(A−1A
)x = A−1b
Inx = A−1b
x = A−1b
We showed that if A is an n × n matrix, then the linear system Ax = b
has the unique solution x = A−1b. Moreover. if b = 0, then the uniquesolution to the homogeneous system Ax = 0 is x = 0.
• Example. Solve the systems
x+ 2y = 8
3x+ 4y = 6, and
x+ 2y = 10
3x+ 4y = 2
Solution. Write the system as Ax = b. Then pre-multiply with A−1.
• Example. Suppose A2x = b, A is nonsingular with
A−1 =
[3 0
2 1
]b =
[−12
]Find the solution x.
Solution.
A−1A−1A2x = A−1A−1b =⇒ x = (A−1)2b =
[9 0
8 1
] [−12
]=
[−9−6
]• Exercises 1.5: 17, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40
15
Week 3-4
2.1 Echelon Form of a Matrix
• An m× n matrix A is said to be in reduced row echelon form (RREF) ifit satisfies the following properties:
a) All zero rows, if there are any, appear at the bottom of the matrix.
b) The first nonzero entry from the left of a nonzero row is a 1. Thisentry is called a leading one of its row.
c) For each nonzero row, the leading one appears to the right and belowany leading ones in preceding rows.
d) If a column contains a leading one, then all other entries in that col-umn are zero.
An ,m× n matrix satisfying properties (a), (b), and (c) is said to be in rowechelon form (REF).
example. Give examples where a matrix breaks one of each rule.
• There are three types of elementary row operations
1. Interchange row i and row j: ri ↔ rj .
2. Replace row i by k 6= 0 times row i: kri → ri.
3. Replace row j by k time row i + row j: kri + rj → rj .
16
• If the matrix Bm×n can be obtained from the matrix Am×n by elementaryrow operations then B is called row equivalent to A.
• If A is row equivalent to B then B is row equivalent to A. proof. Since eachelementary row operation has an inverse which is again an elementary rowoperation. For example the inverse of the operation ri ↔ rj is itself, sinceapplying it twice gives the original matrix.
• Theorem 2.1. Every non-zero matrix is row-equivalent to a matrix in REF.The REF of a matrix may not be unique. Moreover, every matrix is equiv-alent to a unique matrix in RREF. proof. The proof for bringing to REF isthe below algorithm. Let A be an m× n matrix with m ≥ 2.
1. Let i = 1.
2. Let the first non-zero entry in the column i of A below row i be in therow j. If there are none then go to last step.
3. r1 ↔ rj .
4. By choosing suitable numbers k, make aji = 0 for j ≥ i+ 1 by the rowoperation kri + rj → rj .
5. If i = m− 1 then stop. If not increase i by 1.
• In the next section, we will solve examples of finding a matrix in REF whichis row equivalent to a given matrix.
• Exercises 2.1: 1, 3, 5
17
2.2 Solving Linear Systems
• Take a system of m equations and n unknowns. Then write it in matrixform Ax = b. Then write it in the augmented matrix form [A | b].
• Theorem. Let Ax = b and Cx = d be two linear systems. If the augmentedmatrices [A | b] and [C | d] are row equivalent then the linear systems areequivalent, i.e. they have the same solutions.
• To solve a linear system:
1. Write the linear system in the augmented form: [A | b]
2. Gaussian Elimination: Reduce [A | b] into row echelon form andsolve by back substitution.
3. Gauss-Jordan reduction: Reduce [A | b] into reduced row echelonform.
• Example 6 from Section 2.2 The system
x+ 2y + 3z = 6
2x− 3y + 2z = 14
3x+ y − z = −2
in augmented matrix form is 1 2 3 6
2 −3 2 14
3 1 −1 −2
Use the row operations
18
1. −2r1 + r2 → r2,
1 2 3 6
0 −7 −4 2
3 1 −1 −2
,
2. −3r1 + r3 → r3,
1 2 3 6
0 −7 −4 2
0 −5 −10 −20
,
3. −15 r3 → r3, r2 ↔ r3,
1 2 3 6
0 1 2 4
0 −7 −4 2
,
4. 7r2 + r3 → r3,
1 2 3 6
0 1 2 4
0 0 10 30
5. 110 r3 → r3,
1 2 3 6
0 1 2 4
0 0 1 3
Then solve by back substitution. That is
z = 3,
y = 4− 2z = −2x = 6− 2y − 3z = 1
• Examples of systems with exactly one solution, no solution, infinitely manysolutions with 1, 2 free parameters. If a system has no solutions we saythat the system is inconsistent, otherwise we say that the system is con-sistent.
19
Example. If [C d
]=
1 2 3 4 5
0 1 2 3 6
0 0 0 0 1
then Cx = d has no solutions since the last equation is
0x1 + 0x2 + 0x3 + 0x4 = 1.
Example. The solutions of the system1 2 3 4 5 6
0 1 2 3 −1 7
0 0 1 2 3 7
0 0 0 1 2 9
are in the form
x1 = −1− 10r
x2 = 2 + 5r
x3 = −11 + r
x4 = 9− 2r
x5 = r, any real number.
Example. The solutions of the system 1 1 2 0 − 52
23
0 0 0 1 12
12
0 0 0 0 0 0
20
are of the form
x1 =2
3− r − 2s+
5
2t
x2 = r
x3 = s
x4 =1
2− 1
2t
x5 = t
with r, s, t being real numbers.
• Application: quadratic interpolation. Find the quadratic polynomial pass-ing from the points (1, -5), (-1, 1), (2, 7).
• Consider a system of the form Ax = b, where Am×n is a matrix. The systemis called a homogeneous system if b = 0 where 0 is the m×1 zero vector.If b 6= 0 then the system is called a non-homogeneous system.
• A homogeneous system Ax = 0, where Am×n is a matrix, always has asolution. Namely the solution x = 0 which is the n × 1 zero vector. Thissolution is called the trivial solution. Hence, a homogeneous system isalways consistent. We discussed that for linear systems, there are either0, 1 or∞ many solutions. Hence for homogeneous systems the question iswhether the trivial solution is the only solution or there are infinitely manysolutions.
Theorem. A homogeneous system Ax = 0, where Am×n is a matrix, hasinfinitely many solutions if m < n, that is there are fewer equations thanunknowns. proof. Consider the RREF form B of A. Then each column of
21
B which does not have a leading one will be a free parameter. Since eachrow has at most 1 leading one, if there are more columns than there arerows, each column can not have a leading one.Example.
x+ y + z + w = 0
x+ w = 0
x+ 2y + z = 0
whose augmented matrix in REF is 1 0 0 1 0
0 1 0 −1 0
0 0 1 1 0
Thus the solution is
x = −ry = r
z = −rw = r, any real number.
•Example. Balance the chemical reaction
xNaOH+ yH2SO4 → zNa2SO4 + wH2O.
That is, find x, y, z.
22
Solution.Na: x = 2z
O : x+ 4y = 4z + w
H : x+ 2y = 2w
S : y = z1 0 0 −1 0
0 1 0 − 12 0
0 0 1 − 12 0
0 0 0 0 0
A non-trivial solution is
2NaOH+H2SO4 → Na2SO4 + 2H2O
Example. Section 2.2, exercise 14. Determine all values of a such thatthe resulting linear system has (a) no solution; (b) a unique solution; (c)infinitely many solutions:
x+ y − z = 2
x+ 2y + z = 3
x+ y +(a2 − 5
)z = a
Example. Section 2.2, exercise 27. Find an equation relating a, b, and cso that the linear system
2x+ 2y + 3z = a
3x− y + 5z = b
x− 3y + 2z = c
is consistent for any values of a, b, and c that satisfy that equation.
23
• The relationship between homogeneous and non-homogeneous linear sys-tems. If Ax = b has a particular solution xp and Ax = 0 has a solution xh,then xh + xp is also a solution to Ax = b.
• Exercises 2.2: 1, 3, 5, 7, 10, 15, 20, 26, 30, 39.
2.3 Elementary Matrices; Finding A−1
• An m×m matrix is called an elementary matrix if it can be obtained fromthe identity matrix Im by means of a single elementary row operation.
example. The following are elementary matrices
E1 = (I3)r1↔r3 =
0 0 1
0 1 0
1 0 0
, E2 = (I3)−2r2→r1 =
1 0 0
0 −2 0
0 0 1
E3 = (I3)2r2+r1→r1 =
1 2 0
0 1 0
0 0 1
, E4 = (I3)3r3+r1→r3 =
1 0 3
0 1 0
0 0 1
• Theorem. Let A be an m× n matrix. Then
(A)some row operation = (Im)same row operationA
example. [a b
c d
]r1↔r2
=
[c d
a b
]=
[0 1
1 0
] [a b
c d
]= (I2)r1↔r2A
24
• Theorem. If A and B are m × n matrices, then A is row equivalent to Bif and only if there exist elementary matrices E1, E2, . . . , Ek such thatB = EkEk−1 · · ·E2E1A.
• Theorem. An elementary matrix E is invertible, and its inverse is an ele-mentary matrix of the same type. proof. This follows from the fact thatevery row operation can be inverted.
• Theorem. An×n is nonsingular if and only if A is a product of elemen-tary matrices. proof. If A = E1 · · ·Ek then since Ei is invertible, A−1 =
E−1k · · ·E−11 . Conversely, if A is nonsingular, then Ax = 0 has only zero
solution since x = A−10 = 0. Then each column of RREF(A) must have aleading one, otherwise the system would have a non-trivial solution. ThusRREF (A) = In and A is row equivalent to In. By the above theoremA = Ek · · ·E1In.
• Summary. For An×n the following are equivalent.
1. A is nonsingular.
2. Ax = 0 has only the trivial solution.
3. A is row equivalent to In. (RREF of A is In)
4. The linear system Ax = b has a unique solution for every bn×1. proofx = A−1b.
5. A is a product of elementary matrices.
• Let A =
[1 2
2 4
]. Show that A is singular. Solution Show that RREF of A
25
is not I2.
• To find A−1 . Suppose A is nonsingular. Then there exists elementarymatrices E1, . . . , Ek such that Ek · · ·E1A = In. This means A−1 = Ek · · ·E1.From this observation, we get
(EkEk−1 · · ·E2E1) [A | In] = [EkEk−1 · · ·E2E1A | EkEk−1 · · ·E2E1In]
=[In | A−1
]Example. Find the inverse of
A =
1 1 1
0 2 3
5 5 1
if it exists. Solution. Write
[A I3
]=
1 1 1 1 0 0
0 2 3 0 1 0
5 5 1 0 0 1
Use elementary row operations −5r1 + r3 → r3, 1
2r2 → r2, − 14r3 → r3,
− 32r3 + r2 → r2, −r3 + r1 → r1, −1r2 + r1 → r1 to bring it into the form 1 0 0 13
8 − 12 − 1
8
0 1 0 − 158
12
38
0 0 1 54 0 − 1
4
=[I3 A−1
]
• Theorem. An×n is singular, if and only if it is row equivalent to a matrixwhich has a row of zeros. proof. In this case Ax = 0 has a nontrivial
26
solution and hence can not be invertible. example. Show that the matrix
A =
1 2 −31 −2 1
5 −2 −3
is singular.
• Theorem If A and B are n× n matrices such that AB = In, then BA = In.
Thus B = A−1.
proof. First, we will show that A is invertible. If not then RREF (A) = C
which has a row of zeros and C = Ek · · ·E1A. Hence CB = Ek · · ·E1AB =
Ek ·E1. Hence CB is invertible since it is a product of elementary matrices.But this is not possible since CB also has a row of zeros. Hence A−1 mustexist. Then B = A−1AB = A−1I = A−1. So B = A−1.
Remark By definition A and B are inverses of each other if AB = In andBA = In. This theorem says that to be inverses of each other, it sufficescheck only one equation AB = In.
• Exercises 2.3: 2, 7, 8, 9, 11, 17, 19, 21
27
Week 5-6
3.1 Definition of Determinant
• Define Sn = {1, 2, . . . , n}. A rearrangement (with no repetation of ele-ments) j1j2 . . . jn of the elements of Sn is called a permutation of Sn. Theset Sn has a total A permutation is said to have an inversion if a largerinteger comes before than a smaller integer. If the total number of inver-sions is even, then the permutation is called even otherwise it is calledodd.Example. The permutation 4132 of S4 has 4 inversions: 4 > 1, 4 > 3, 4 > 2,3 > 2 so it is even.Example. The permutations of the set S2 = {1, 2} are 12 and 21. Thepermutation 12 is even (zero inversions), and the permutation 21 is odd (1inversion).Example. The permutations of the set S3 = {1, 2, 3} are 123, 231, 312 whichare even and 132, 321, 213 which are odd.
Let A = [aij ] be an n× n matrix. The determinant of A is
det(A) =∑
(±)a1j1a2j2 · · · anjn
where the summation is over all permutations j1j2 · · · jn of the setS = {1, 2, . . . , n}. The sign is taken as + or − according to whetherthe permutation j1j2 · · · jn is even or odd.
28
Thus determinant of an n×n matrix is a sum of n! terms each of which is aproduct of n terms. Each term in a product comes from a distinct row andcolumn. That is no product contains terms from the same row or column.
• For a 2× 2
A =
[a11 a12a21 a22
]we have
det(A) = a11a22 − a12a21Example. For
A =
[2 −34 5
]det(A) = (2)(5)− (−3)(4) = 22.
• For
A =
a11 a12 a13a21 a22 a23a31 a32 a33
we have
det(A) =a11a22a33 + a12a23a31 + a13a21a32 − a11a23a32− a12a21a33 − a13a22a31
Example. The determinant of ∣∣∣∣∣∣0 3 7
5 6 5
−1 5 5
∣∣∣∣∣∣29
a11 a12 a13 a11 a12
a21 a22 a23 a21 a22
a31 a32 a33 a31 a32
Figure 1: Sarrus rule to compute 3×3 determinants. This method does not workfor 4× 4 matrices.
is 127.
• The definition of determinant is not practical when n is as large as 10.It involves the sum of 10! terms each of which is a product of 10 terms.Around 3.7× 107 operations.
• Exercises 3.1: 8, 11, 13
3.2 Properties of Determinant
det(A) = det(AT).
Example. |A| =∣∣∣∣ a b
c d
∣∣∣∣ = ∣∣∣∣ a c
b d
∣∣∣∣ = ∣∣AT∣∣ = ad− bc
30
det(Ari↔rj ) = −det(A) if i 6= j.
Example.
∣∣∣∣ a b
c d
∣∣∣∣ = − ∣∣∣∣ c d
a b
∣∣∣∣If two rows (or columns) of a are equal then det(a) = 0.
proof suppose rowi = rowj , i 6= j then a = ari↔rj so that det a = det ari↔rj =
− det a.
Example.
∣∣∣∣ a b
a b
∣∣∣∣ = 0 and
∣∣∣∣∣∣1 2 3
−1 0 7
1 2 3
∣∣∣∣∣∣ = 0
If a row (or column) of a consists entirely of zeros then det(a) = 0.
proof. This follows from the fact that each summand in the determinant sumformula contains exactly one element from each row (or column).
Example.
∣∣∣∣∣∣1 2 3
4 5 6
0 0 0
∣∣∣∣∣∣ = 0
det(Akri→ri) = k det(A).
Example.
∣∣∣∣ ka kb
c d
∣∣∣∣ = k(ad− bc) = k
∣∣∣∣ a b
c d
∣∣∣∣31
det(Ari+krj→ri) = det(A), i 6= j.
Example.
∣∣∣∣ a b
ka+ c kb+ d
∣∣∣∣ = (a(kb+ d)− b(ka+ c) =
∣∣∣∣ a b
c d
∣∣∣∣Example.
∣∣∣∣∣∣1 2 3
2 −1 3
1 0 1
∣∣∣∣∣∣ =∣∣∣∣∣∣5 0 9
2 −1 3
1 0 1
∣∣∣∣∣∣ = 4, obtained by adding twice the sec-
ond row to the first row.Example. If det(A) = 4 and B = Ar1+2r2→r1 then det(B) = 4.
If a row (or column) of a matrix is a multiple of another row then itsdeterminant is zero.
proof. Since the row which is a multiple of the other can be made zero bythe operation ri + krj → ri which gives a zero determinant.
Let A be an upper (or lower) triangular matrix. Then det(A) is equal tothe product of the diagonal entries of A.
proof. Recall that determinant is a sum of product of n terms each comingfrom a different row and column. Thus for an upper diagonal matrix, only theterms including a11 can be non-zero. But for any product containing a11, noother elements can be chosen from the first column. Thus the only productsincluding a11a22 can be non-zero. Going this way, we obtain the conclusion.
Since a diagonal matrix is both lower and upper triangular,
32
The determinant of a diagonal matrix is the product of elements on itsdiagonal.
Example. ∣∣∣∣∣∣3 1 0
0 8 −10 0 1
∣∣∣∣∣∣ = 3× 8× 1 = 24.
Example. Compute ∣∣∣∣∣∣3 4 5
5 2 0
−1 0 0
∣∣∣∣∣∣After r1 ↔ r3 determinant changes by −1 and the matrix is in triangular form.The answer is −1×−1× 2× 5 = 10.
Since det(A) = det(AT ), column operations can be used too instead ofrow operations. That is
1. det(A) = −det(Aci↔cj ), i 6= j.2. det(Akci→ci) = k det(A).3. det(Aci+kcj→ci) = det(A), i 6= j.
Example. Compute ∣∣∣∣∣∣3 6 5
4 8 0
−1 −2 1
∣∣∣∣∣∣Second column can be made zero by c2 − 2c1 → c2. So the determinant is zero.
33
Example. Compute det(A) for
A =
2 4 6
0 1 2
2 −3 1
After the operations
1. 1/2r1 → r1
2. −2r1 + r3 → r3,
3. 7r2 + r3 → r3
det(A) = 2
∣∣∣∣∣∣1 2 3
0 1 2
0 0 9
∣∣∣∣∣∣ = 2(1)(1)9 = 18
This method is called as the computation of determinant via reduction totriangular form.
Determinant of three types of elementary matrices:• det(Iri↔rj ) = −1,• det (Ikri→ri) = k,• det
(Ikrj+ri→ri
)= 1.
If E is an elementary matrix then det(EA) = det(E) det(A).
34
proof. Take E = Iri↔rj then EA is the matrix A with row i and j swapped.Thus det(EA) = −det(A). On the other hand det(E) det(A) = −det(A) sincedet(E) = −1. Thus det(EA) = det(E) det(A). The same is true for other types ofelementary matrices.
If B is row equivalent to A then det(A) is non-zero if and only if det(B) isnon-zero.
proof. Suppose B is row equivalent to A. We know B = EkEk−1 · · ·E1A.
det(B) = det(EkEk−1 · · ·E1A) = det(Ek) det(Ek−1 · · ·E1A)
= det(Ek) det(Ek−1) · · · det(E1) det(A)
Theorem
An×n is nonsingular if and only if det(A) 6= 0.
proof. If A is nonsingular then A is a product of elementary matrices andhence its determinant is non zero. If A is singular, then A is row equivalent to amatrix B that has a row of zeros. Hence its determinant is zero.
If A is n × n matrix, then Ax = 0 has a nontrivial solution if and only ifdet(A) = 0.
35
Example. Does the systemx+ 3y + 2z = 0
2x− 2y = 0
3x+ 9y + 6z = 0
have any non-trivial solution?
solution. Yes because 1 3 2
2 −2 0
3 9 6
its determinant is zero since the 3rd row is a multiple of the first row.Example. (
3 1
2 4
)(x1x2
)=
(0
0
)have a non-trivial solution. solution. No because the determinant of the matrixis 10. This system has only the solution (x1, x2) = (0, 0).Example. Let A be a 4× 4 matrix with det (A) = −2
1. Describe the set of all solutions to the homogeneous system Ax = 0.answer. since det (A) 6= 0, the homogeneous system has only the trivialsolution.
2. If A is transformed to reduced row echelon form B, what is B? answer. In.
3. Can the linear system Ax = b have more than one solution? Explain.answer. no the system has unique solution x = A−1b.
4. Does A−1 exist? answer. Yes.
36
Theorem
If A and B are n× n matrices then det(AB) = det(A) det(B).
proof. If A is nonsingular then A = Ek · · ·E1, product of elementary matrices.
det(AB) = det(Ek · · ·E1B) = det(Ek) · · · det(E1) det(B) = det(A) det(B)
If A is singular then A is row equivalent to C which has a row of zeros. Thus
det(AB) = det(Ek · · ·E1CB) = det(Ek) · · · det(E1) det(CB)
Notice that CB has a row of zeros and det(CB) = 0.Example. If det(A) = 5, det(B) = −3 then det(AB) = −15 and det(A2) = 25.
Theorem
If A is nonsingular then det(A−1) = 1det(A) .
proof. 1 = det(I) = det(AA−1) = det(A) det(A−1).
In general (most usually) det(A+B) 6= det(A) + det(B).
Exercises 3.2: 1-5, 8, 9, 10, 13, 14, 15, 17, 22, 26, 30, 31.
37
3.3 Cofactor Expansion
Let A = [aij ] be an n×nmatrix. LetMij be the (n−1)×(n−1) submatrix ofA obtained by deleting the ith row and jth column. Aij = (−1)i+j det(Mij)
is called the cofactor Aij of aij .
Example 1 3 2
2 −2 0
3 9 6
Find the cofactor A22.answer.
A22 = (−1)2+2 ×∣∣∣∣1 2
3 6
∣∣∣∣ = 0
Determinant as cofactor expansion theorem
Let A = [aij ] be an n × n matrix. Then expansion of det(A) along theith row is
det(A) = ai1Ai1 + ai2Ai2 + · · ·+ ainAin
and the expansion of det(A) along the jth column is
det(A) = a1jA1j + a2jA2j + · · ·+ anjAnj
38
Let us verify that the statement for the 3 × 3 case and for expansion alongthe first row.∣∣∣∣∣∣
a11 a12 a13a21 a22 a23a31 a32 a33
∣∣∣∣∣∣= a11(a22a33 − a23a32)− a12(a23a31 − a21a33) + a13(a21a32 − a22a31)= a11A11 + a12A12 + a13A13
Best way to compute a matrix determinant is to expand it along a row/columnwith most zeros.
Example
Use cofactor expansion method to compute the determinant.∣∣∣∣∣∣4 3 1
0 −2 0
5 −3 6
∣∣∣∣∣∣Solution. We expand along the second row.
(−1)2+1 × 0 + (−1)2+2(−2)∣∣∣∣4 1
5 6
∣∣∣∣+ (−1)2+3 × 0 = −38
Check that expansion along a different row or column gives the sameresult.
39
∣∣∣∣∣∣∣∣4 8 −9 10
0 0 1 2
0 3 1 0
0 5 0 0
∣∣∣∣∣∣∣∣ = 4× (−1)1+1
∣∣∣∣∣∣0 1 2
3 1 0
5 0 0
∣∣∣∣∣∣ = 4× 5× (−1)3+1
∣∣∣∣1 2
1 0
∣∣∣∣ = −40In the first determinant, we expand along the first column. In the seconddeterminant we expand along third row.
We can combine row operations with cofactor method to evaluate determi-nants.
Example. Find all values of t for which∣∣∣∣∣∣t− 1 0 1
−2 t+ 2 −10 0 t+ 1
∣∣∣∣∣∣ = 0
Solution. Expansion along the third row gives
(−1)3+3(t+ 1)((t− 1)(t+ 2)− 0) = 0
gives t = −2,−1, 1.
Exercises 3.3. 3, 11, 12
40
3.4 Inverse of a Matrix
If A = [aij ]is an n× n matrix then
ai1Ak1 + ai2Ak2 + · · ·+ ainAkn =
{det(A) if i = k
0 if i 6= k
where Akj is the cofactor of akj .
proof. If B is a matrix obtained from A by replacing the kth row of A byits ith row then B has two identical rows and det(B) = 0. On the other handdet(B) = ai1Ak1 + ai2Ak2 + · · ·+ ainAkn.Example.
A =
1 2 3
−2 3 1
4 5 −2
A21 = (−1)2+1
∣∣∣∣ 2 3
5 −2
∣∣∣∣ = 19
A22 = (−1)2+2
∣∣∣∣ 1 3
4 −2
∣∣∣∣ = −14A23 = (−1)2+3
∣∣∣∣ 1 2
4 5
∣∣∣∣ = 3
a11A21 + a12A22 + a13A23 = (1)(19) + (2)(−14) + (3)(3) = 0
a21A21 + a22A22 + a23A23 = (−2)(19) + (3)(−14) + (1)(3) = −77 = detA
41
a31A21 + a32A22 + a33A23 = (4)(19) + (5)(−14) + (−2)(3) = 0
Definition
Let A = [aij ] be an n× n matrix. The adjoint of A is the n× n matrix
adjA =
A11 A12 . . . A1n
A21 A22 . . . A2n
.... . .
...An1 An2 . . . Ann
T
,
where Aij is the cofactor of aji. (Be careful about the transpose)
Theorem
If A is n× n matrix
A(adjA) = (adjA)A = det(A)In.
Thus if det(A) 6= 0 then
A−1 =1
det(A)(adjA)
proof. This is a consequence of
ai1Ak1 + ai2Ak2 + · · ·+ ainAkn =
{det(A) if i = k
0 if i 6= k
42
where Akj is the cofactor of akj . The left hand side is the product of kth row ofadj(A) with ith row of A.Example. Let
A =
3 −2 1
5 6 2
1 0 −3
Compute adjA. Verify A(adjA) = (adjA)A = det(A)In.
answer.
adjA =
−18 −6 −1017 −10 −1−6 −2 28
Example. Use the adjoint method to find the inverse of4 1 2
0 −3 3
0 0 2
Solution. The answer is
1
24
6 2 −90 −8 12
0 0 12
This method of inverting is much less efficient than the method given inChapter 2: [A | In]→ [In | A−1].
43
Exercises 3.4: 2, 9, 10, 12
3.5 Other Applications of Determinants
Theorem. Cramer’s Rule
Leta11x1 + a12x2 + · · ·+ a1nxn = b1
a21x1 + a22x2 + · · ·+ a2nxn = b2
...
an1x1 + an2x2 + · · ·+ annxn = bn
If det(A) 6= 0, then the system has the unique solution
x1 =det (A1)
det(A), x2 =
det (A2)
det(A), . . . , xn =
det (An)
det(A)
where Ai is the matrix A with ith column replaced by b,
b =
b1b2...bn
44
proof. Since detA 6= 0, A−1 exists. Ax = b implies
x = A−1b =1
detAadj(A)b
For example, suppose A is 3× 3. Then
x1 =1
detA(A11b1 +A21b2 +A31bn)
=1
detA
((−1)1+1
∣∣∣∣a22 a23a32 a33
∣∣∣∣ b1 + ∣∣∣∣a12 a13a32 a33
∣∣∣∣ b2 + ∣∣∣∣a12 a13a22 a23
∣∣∣∣ b3)
=1
detA
∣∣∣∣∣∣b1 a12 a13b2 a22 a23b3 a32 a33
∣∣∣∣∣∣This is exactly the determinant det(A1) defined in the theorem.
Example. Cramer’s Rule
Use Cramer’s rule to solve
−2x1 + 3x2 − x3 = 1
x1 + 2x2 − x3 = 4
−2x1 − x2 + x3 = −3
solution.
|A| =
∣∣∣∣∣∣−2 3 −11 2 −1−2 −1 1
∣∣∣∣∣∣ = −245
x1 =
∣∣∣∣∣∣1 3 −14 2 −1−3 −1 1
∣∣∣∣∣∣|A|
=−4−2
= 2, x2 =
∣∣∣∣∣∣−2 1 −11 4 −1−2 −3 1
∣∣∣∣∣∣|A|
=−6−2
= 3
x3 =
∣∣∣∣∣∣−2 3 1
1 2 4
−2 −1 −3
∣∣∣∣∣∣|A|
=−8−2
= 4
Cramer’s rule is applicable only when there are n equations in n unknownsand the coefficient matrix A is nonsingular. Otherwise we must use the Gaus-sian elimination or Gauss-Jordan reduction methods. Cramer’s rule becomescomputationally inefficient for n ≥ 4.
Exercises from the book
• Exercises 3.5. 1, 3, 5.• Chapter 3 Supplementary Exercises. 1, 2, 3, 4, 5, 8.• Chapter 3 Quiz. 1, 2, 3, 4, 5.• This is optional. Read section 3.6 for computational complexity of
computing determinants with various methods, inverse matrix, etc.
46
Week 7
Example
Prove that for any ai, bi, c1, d1∣∣∣∣∣∣∣∣a1 a2 a3 a4b1 b2 b3 b4c1 0 0 0
d1 0 0 0
∣∣∣∣∣∣∣∣ = 0
Example
Find ∣∣∣∣∣∣∣∣0 1 2 0
1 3 4 5
−2 0 −1 1
1 −1 0 0
∣∣∣∣∣∣∣∣Example
Let B be the matrix obtained from A after the row operations 2r3 →r3, r1 ↔ r2, 4r1 + r3 → r3, and −2r1 + r4 → r4 have been performed. Ifdet(B) = 2 find det(A).
47
Example
Let A,B, and C be 2 × 2 matrices with det(A) = 3 det(B) = −2, anddet(C) = 4. Compute det
(6ATBC−1
).
Example
Show that if An = O, the zero matrix, for some positive integer n thendet(A) = 0.
True or False
1. det(A+B) = det(A) + det(B).2. det(A−1B) = det(B)
det(A) .
3. If det(A) = 0 then A has at least two equal rows.4. A is singular if and only if det(A) = 0.5. If B is the reduced row echelon form of A then det(A) = det(B).6. 1
c det(cA) = det(A).7. det(ABTA−1) = det(B).8. det(AAT ) ≥ 0.
48
Example
Verify ∣∣∣∣∣∣a− b 1 a
b− c 1 b
c− a 1 c
∣∣∣∣∣∣ =∣∣∣∣∣∣a 1 b
b 1 c
c 1 a
∣∣∣∣∣∣
Week 8
4.1 Vectors in the plane and in 3-space
1. 2-Vectors can be added and multiplied by scalars. There is a zero vectorand can take the difference of vectors. Same holds for 3-vectors.
49
2. If u,v, and w are vectors in R2 or R3, and c and d are real scalars, then thefollowing properties are valid:
(a) u+ v = v + u
(b) u+ (v +w) = (u+ v) +w
(c) u+ 0 = 0+ u = u
(d) u+ (−u) = 0
(e) c(u+ v) = cu+ cv
(f) (c+ d)u = cu+ du
(g) c(du) = (cd)u
(h) 1u = u
4.2 Vector Spaces
There are many structures satisfying the above conditions other then R2 and R3
equipped with vector addition and scalar multiplication.
1. Definition. A real vector space is a set V of elements on which we havetwo operations ⊕ and � defined with the following properties:
(a) If u and v are any elements in V, then u⊕v is in V . (We say that V isclosed under the operation ⊕.)
(1) u⊕ v = v ⊕ u for all u,v in V
50
(2) u⊕ (v ⊕w) = (u⊕ v)⊕w for all u,v,w in V
(3) There exists an element 0 in V such that u ⊕ 0 = 0 ⊕ u = u forany u in V .
(4) For each u in V there exists an element −u in V such that u ⊕−u = −u⊕ u = 0.
(b) If u is any element in V and c is any real number, then c � u is in V
(i.e. V is closed under the operation �)
(5) c� (u⊕v) = c�u⊕ c�v for any u,v in V and any real number c.
(6) (c+ d)� u = c� u⊕ d� u for any u in V and any real numbers cand d.
(7) c� (d�u) = (cd)�u for any u in V and any real numbers c and d
(8) 1� u = u for any u in V
The elements of V are called vectors; the elements of the set of realnumbers R are called scalars. The operation ⊕ is called vector ad-dition; the operation � is called scalar multiplication. The vector 0in property (3) is called a zero vector. The vector −u in property (4)is called a negative of u. It can be shown that 0 and −u are unique.If we allow the scalars to be complex numbers, we obtain a complexvector space.
2. example. Rn, the set of n-vectors with usual vector addition and scalarproduct is a real vector space for any integer n ≥ 1.
51
3. example. More generally, the set of all m × n matrices with usual matrixaddition and scalar product is a real vector space.
4. example. The set of all 2 × 2 matrices with trace equal to zero is a realvector space with usual matrix addition and scalar product.
5. example. The set Pn of all polynomials with degree ≤ n with polynomialaddition and multiplication by scalar is a real vector space.
6. example. More generally, the set of all real-valued functions defined onR1 form a real vector space with usual function addition and product offunctions with scalars.
7. example. Let V be the set of all real numbers with the operations u⊕ v =
u−v(⊕ is ordinary subtraction) and c�u = cu(� is ordinary multiplication).Is V a vector space? If it is not, which properties in definition fail to hold?answer. (2), (3), (4), (6). Thus V is not a vector space.
8. example. Let V be the set of all ordered triples of real numbers (x, y, z)
with the operations (x, y, z) ⊕ (x′, y′, z′) = (x′, y + y′, z + z′) ; c � (x, y, z) =
(cx, cy, cz). We can readily verify that properties (1), (3), (4), and (6) of Def-inition fail to hold.
9. example. Let V be the set of all integers; define ⊕ as ordinary addition and� as ordinary multiplication. Here V is not a vector space, because if u isany nonzero vector in V and c =
√3, then c�u is not in V. Thus (b) fails to
hold.
10. The following are not in axioms of a vector space but they hold for anyvector space.
52
Theorem 4.2 If V is a vector space, then
(a) 0� u = 0 for any vector u in V
(b) c� 0 = 0 for any scalar c
(c) If c� u = 0, then either c = 0 or u = 0
(d) (−1)� u = −u for any vector u in V
proof (a) 0� u = (0 + 0)� u = 0� u+ 0� u then subtract 0� u from bothsides.
11. Exercises 4.2.: 1-4, 7-12, 16-18
4.3 Subspaces
1. Let V be a vector space and W a nonempty subset of V. If W is a vectorspace with respect to the operations in V, then W is called a subspace ofV .
2. Let V be a vector space with operations ⊕ and � and let W be a nonemptysubset of V. Then W is a subspace of V if and only if the following condi-tions hold: (a) If u and v are any vectors in W, then u⊕ v is in W (b) If c isany real number and u is any vector in W, then c� u is in W
3. Every vector space has at least two subspaces, itself and the subspace {0}consisting only of the zero-vector, called the zero subspace.
53
4. P2 the set of all polynomials of degree ≤ 2 is a subspace of the vector spaceof all polynomials P .
5. Is the set of all polynomials of degree exactly 2 a vector space? No! Thesum of polynomials x2 + 1 and −x2 + x is a polynomial of degree 1.
6. Which of the following subsets of R2 with the usual operations of vectoraddition and scalar multiplication are subspaces?
(a) The set of all vectors of the form
(x
y
)where x ≥ 0.
(b) The set of all vectors of the form
(x
y
), where x = 0
7. Is the set
a
b
a+ b
, a subspace of R3 with usual operations?
8. Given two vectors v1 and v2 in a vector space V , the set {a1v1 + a2v2 : a1, a2 ∈ R}is a subspace of V . Prove that this set is closed with respect to vector ad-dition and scalar multiplication.
9. Let v1,v2, . . . ,vk be vectors in a vector space V. A vector v in V is calleda linear combination of v1,v2, . . . ,vk if v = a1v1 + a2v2 + · · · + akvk =∑k
j=1 ajvj for some real numbers a1, a2, . . . , ak.
10. In a previous example we showed that W, the set of all vectors in R3 of
the form
a
b
a+ b
where a and b are any real numbers, is a subspace of
54
R3. Let v1 =
1
0
1
and v2 =
0
1
1
.Then every vector in W is a linear
combination of v1 and v2, since av1 + bv2 =
a
b
a+ b
11. example Is the vector v =
2
1
5
a linear combination of the vectors v1 =
1
2
1
, v2 =
1
0
2
, and v3 =
1
1
0
.
solution. Yes if we can find real numbers a1, a2, and a3 such that a1v1 +
a2v2 + a3v3 = v which gives
a1
1
2
1
+ a2
1
0
2
+ a3
1
1
0
=
2
1
5
which is equivalent to
a1 + a2 + a3 = 2
2a1 + a3 = 1
a1 + 2a2 = 5
We can solve this system to find a1 = 1, a2 = 2 and a3 = −1 so thatv = v1 + 2v2 − v3.
55
12. Let Am×n be a matrix. Then the solution set Ax = 0 is a subset of Rn. Infact this is a subspace of Rn.
13. Exercises 4.3: 1-19, 23-35
Week 9
1 class hour was canceled due to national holiday.
4.4 Span
1. We have seen that the set of all linear combination of two vectors of avector space V is a subspace of V . More generally we have:
Theorem 4.4 The set of all possible linear combinations of vectors v1,v2, . . . ,vk
in V is a subspace of V . This subspace is known as the span of the vectorsv1,v2, . . . ,vk.
2. What is the span of vectors
S =
{[1 0 0
0 0 0
],
[0 1 0
0 0 0
],
[0 0 0
0 1 0
],
[0 0 0
0 0 1
]}in the vector space M2×3 of 2× 3 matrices.
56
answer. It consists of all matrices that can be written as
a
[1 0 0
0 0 0
]+ b
[0 1 0
0 0 0
]+ c
[0 0 0
0 1 0
]+ d
[0 0 0
0 0 1
]
where a, b, c, and d are real numbers, that is
[a b 0
0 c d
]3. example. Let S =
{t2, t
}be a subset of the vector space P2. Then span S is
the subspace of all polynomials of the form at2 + bt, where a and b are anyreal numbers.
4. example. Let
v1 =
2
1
1
, v2 =
1
−13
Is
v =
1
5
−7
in the span of {v1,v2}?
answer. The problem asks if we can find scalars a1, a2 such that a1v1 +
a2v2 = v, that is
a1
2
1
1
+ a2
1
−13
=
1
5
−7
57
The augmented form is 2 1 1
1 −1 5
1 3 −7
which has RREF 1 0 2
0 1 −30 0 0
The system is consistent with a1 = 2, a2 = 3. The answer is YES.
5. example. Do vectors
v1 =
1
2
1
, v2 =
1
0
2
, and v3 =
1
1
0
span R3?
answer Pick any vector
a
b
c
and determine whether there are constants
a1, a2, a3 such that a1v1 + a2v2 + a3v3 = v. This leads to the linear system
a1 + a2 + a3 = a
2a1 + a3 = b
a1 + 2a2 = c
which has solution
a1 =−2a+ 2b+ c
3, a2 =
a− b+ c
3, a3 =
4a− b− 2c
3
58
The answer is YES!
6. example. Let V be the vector space P2. Let v1 = t2 + 2t + 1 and v2 =
t2 + 2.Does {v1,v2} spanV ?
answer. No, see book example 9.
7. example. Suppose the RREF of augmented matrix of the equation Ax = 0
is
A =
1 1 0 2
−2 −2 1 −51 1 −1 3
4 4 −1 9
Find a set which spans the solution space of Ax = 0.
answer. Note that the solution is
x1 = −r − 2s, x2 = r, x3 = s, x4 = s
so that the general solution is
x = r
−11
0
0
+ s
−20
1
1
Hence the vectors
−11
0
0
and
−20
1
1
span the solution space.
8. 4.4 Exercises. 2-11
59
4.5 Linear Independence
.
1. We have seen that the set W of all vectors of the form a
b
a+ b
is a subspace of R3. Each of the following sets span W :
S1 =
1
0
1
, 0
1
1
, 3
2
5
S2 =
1
0
1
, 0
1
1
S2 is a more efficient spanning set since it has fewer elements.
2. We want to find a spanning set for a given vector space which contains theleast number of elements. For this we make the following definition.
definition. The vectors v1,v2, . . . ,vk in a vector space V are said to belinearly dependent if there exist constants a1, a2, . . . , ak, not all zero, suchthat
a1v1 + a2v2 + · · ·+ akvk = 0
60
Otherwise, v1,v2, . . . ,vk are called linearly independent. That is, v1,v2, . . . ,vk
are linearly independent if, whenever a1v1 + a2v2 + · · ·+ akvk = 0 we havea1 = a2 = · · · = ak = 0
3. example. Are the following vectors linearly independent? v1 =
3
2
1
, v2 =
1
2
0
, v3 =
−12−1
answer. Write a1
3
2
1
+a2 1
2
0
+a3 −12−1
=
0
0
0
, that is
3 1 −1 0
2 2 2 0
1 0 −1 0
.
The RREF is
1 0 −1 0
0 1 2 0
0 0 0 0
, which has a non-trivial solution
k
−2kk
, k 6=
0 so the vectors are linearly dependent.
4. example. Are the vectors v1 =[1 0 0
],v2 =
[0 1 0
]and v3 =[
0 0 1]
linearly independent? answer. YES!
5. example. Are the vectors v1 = t2 + t+ 2,v2 = 2t2 + t, and v3 = 3t2 + 2t+ 2
in P2 linearly independent? answer. No since v1 + v2 − v3 = 0
6. theorem 4.5 Let S = {v1,v2, . . . ,vn} be a set of n vectors in Rn (Rn) . LetA be the matrix whose columns (rows) are the elements of S. Then S islinearly independent if and only if det(A) 6= 0
proof. If S is linearly independent then RREF(A)=In and det(A) 6= 0. Con-
61
verse also holds.
7. example. Is S ={[
1 2 3],[0 1 2
],[3 0 −1
]}a linearly inde-
pendent set of vec- tors in R3?
solution. A =
1 2 3
0 1 2
3 0 −1
Since det(A) = 2, S is linearly independent.
8. Let S1 ⊂ S2 be two subsets of a vector space. If S1 is linearly dependentthen so is S2. If S2 is linearly independent then so is S1.
9. Geometric meaning of linear independence. Two non-zero vectors in R2
are linearly dependent if they are parallel (one is a scalar multiple of theother) and independent if not. In R3, three vectors are linearly dependentif one of the vectors is a linear combination of the other two. That meansthese three vectors lie in the same plane. And if they do no lie in the sameplane then they are independent.
10. example. Let V = R3 and also v1 =[1 2 −1
],v2 =
[1 −2 1
],v3 =[
−3 2 −1]
and v4 =[2 0 0
]. We find (verify) that v1 + v2 +
0v3 − v4 = 0 so v1,v2,v3, and v4 are linearly dependent. We then havev4 = v1 + v2 + 0v3.
11. Section 4.5 exercises 1-4, 11-17, 20-23
62
Week 10
1 class hour was canceled due to April 23rd holiday. 2 class hours was canceleddue to May 1st holiday.
Week 11
4.6 Basis and Dimension
1. The vectors v1,v2, . . . ,vk in a vector space V are said to form a basis forV if (a)v1,v2, . . . ,vk spanV and (b)v1,v2, . . . ,vk are linearly independent.
2. Remark. If v1,v2, . . . ,vk form a basis for a vector space V, then they mustbe distinct and nonzero.
3. Standard basis of R3:
1
0
0
, 0
1
0
, 0
0
1
. Standard basis for R3:
[1 0 0
],[0 1 0
],[0 0 1
]. Generalize this to Rn and Rn.
4. example. Show that the set S ={t2 + 1, t− 1, 2t+ 2
}is a basis for the
vector space P2.
63
solution. 1st step. Show that for each a, b, c we can find a1, a2, a3 suchthat
at2 + bt+ c = a1(t2 + 1
)+ a2(t− 1) + a3(2t+ 2)
This reduces toa1 = a
a2 + 2a3 = b
a1 − a2 + 2a3 = c
Show that this system has a solution. 2nd step. Show that the vectors givenare linearly independent. To show this, write a1
(t2 + 1
)+a2(t−1)+a3(2t+
2) = 0 and show that only solution is the trivial solution a1 = a2 = a3 = 0.This reduces to
a1 = 0
a2 + 2a3 = 0
a1 − a2 + 2a3 = 0
5. example. Find a basis for the subspace V of P2, consisting of all vectors ofthe form at2+ bt+ c, where c = a− b.
solution. Every vector in P2 is of the form at2+bt+a−b, or a(t2 + 1
)+b(t−1).
The vectors t2+1 and t−1 span V . These vectors are linearly independentsince neither one is a multiple of the other.
6. A vector space V is called finite-dimensional if there is a finite subset of Vthat is a basis for V. If there is no such finite subset of V, then V is calledinfinite-dimensional.
64
7. Theorem 4.8 If S = {v1,v2, . . . ,vn} is a basis for a vector space V, thenevery vector in V can be written in one and only one way as a linear com-bination of the vectors in S.
8. Theorem 4.9 Let S = {v1,v2, . . . ,vn} be a set of nonzero vectors in a vectorspace V and let W = spanS. Then some subset of S is a basis for W.
9. example. Find a basis for the span(S) where S = {v1,v2,v3,v4,v5} , wherev1 =
[1 0 1
], v2 =
[0 1 1
], v3 =
[1 1 2
], v4 =
[1 2 1
],
and v5 =[−1 1 −2
]. Find a subset of S that is a basis for R3.
solution. Step 1. Write a1v1 + a2v2 + a3v3 + a4v4 + a5v5 = 0.
Step 2.
a1 + a3 + a4 − a5 = 0
a2 + a3 + 2a4 + a5 = 0
a1 + a2 + 2a3 + a4 − 2a5 = 0
Step 3. RREF is
1 0 1 0 −2 0
0 1 1 0 −1 0
0 0 0 1 1 0
This has infinitely many solutions so the vectors are not linearly indepen-dent.
Step 4. If v3 and v5 were not present the RREF would be
1 0 0 0
0 1 0 0
0 0 1 0
and the system would be linearly independent.
10. We can have different basis for the same vector space. For example B1 =
65
{[1
0
],
[0
1
]}and B2 =
{[1
1
],
[1
2
]}are both basis for R2. But it
turns out that (see Corollary 4.1 in the book) the number of vectors in abasis set is always same for a given vector space. This number is calledthe dimension of the vector space.
11. The set S ={t2, t, 1
}is a basis for P2, so dim P2 = 3.
12. Let V be the subspace of R3 spanned by S = {v1,v2,v3} , where v1 =[0 1 1
]v2 =
[1 0 1
], and v3 =
[1 1 2
]. We find that S is
linearly dependent, and v3 = v1 + v2 (verify). Thus S1 = {v1,v2} alsospans V . Since S1 is linearly independent (verify), we conclude that it is abasis for V. Hence dimV = 2.
13. If vector space V has dimension n, then any subset of m > n vectors mustbe linearly dependent.
14. If vector space V has dimension n, then any subset ofm < n vectors cannotspan V .
15. 4.6 Exercises. 1-15, 19-24.
4.7 Homogeneous Systems
1. Let A be m×n matrix. Then the solution set of Ax = 0 is a subspace of Rn.This solution space is also called nullspace of A and its dimension is callednullity of A.
66
2. example. Suppose the RREF of Ax = 0 is
1 0 2 0 1 0
0 1 2 0 −1 0
0 0 0 1 2 0
0 0 0 0 0 0
0 0 0 0 0 0
Then
the solution space is x =
−2s− t−2s+ t
s
−2tt
where s, t are any real numbers. Let
x1 =
−2−21
0
0
and x2 =
−11
0
−21
. Then {x1,x2} is a basis for the solution
space of A with dimension 2. The nullity of A is 2.
3. example. Find a basis for the solution space of the homogeneous system
(λI3 −A)x = 0 for λ = −2 and A =
−3 0 −12 1 0
0 0 −2
.
solution. We form −2I3−A =
1 0 1
−2 −3 0
0 0 0
. RREF is
1 0 1 0
0 1 − 23 0
0 0 0 0
.
−123
1
is a basis for the solution space.
67
4. Warning The solution set of a non-homogeneous equation Ax = b, b 6= 0 isnot a vector space. Any solution of a non-homogeneous equation can bewritten as a sum of a particular solution and the solution of the associatedhomogeneous equation.
5. 4.7 Exercises. 1-20.
6.1 Linear Transformations and Matrices. Definition and Ex-amples
1. Let V and W be vector spaces. A function L : V → W is called a lineartransformation of V into W if(a) L(u+ v) = L(u) + L(v) for every u and v in V(b) L(cu) = cL(u) for any u in V, and c any real number.
2. Let A be an m × n matrix. Then L : Rn → Rm defined by L(u) = Au is alinear transformation.
3. example. Let L : R3 → R3 be defined by L
u1u2u3
=
[u1 + 1
2u2
]. Is L a
linear transformation?
answer. No because in general L(u+ v) 6= L(u) + L(v).
4. example. Let L : R2 → R2, L([
u1 u2])
=[u21 2u2
]. Is L a linear
transformation?
answer. No! Show that L(u+ v) 6= L(u) + L(v).
68
5. example. Let L : P1 → P2 be defined by L[p(t)] = tp(t). Show that L is alinear transformation.
6. Let L : V →W be a linear transformation. Then(a) L (0V ) = 0W where 0V is the zero vector in V and 0W is the zero vectorin W .(b) L(u− v) = L(u)− L(v), for u,v in V
7. L
u1u2u3
=
u1 + 1
2u2u3
is not a linear transformation because
L
0
0
0
=
1
0
0
.
8. example. Let L : R2 → R2 be a linear transformation for which weknow that L
([1 1
])=[1 −2
]and L([−1 1]) =
[2 3
]. What is
L([−1 5
])? What is L
([u1 u2
])?
9. Let L : Rn → Rm be a linear transformation and consider the natural basis{e1, · · · en}. Let A be the m × n matrix whose j th column is L (ej) . A
has the property L(x) = Ax for every x ∈ Rn. This matrix is called thestandard matrix representation of L.
10. example. Find the standar matrix representation of L where L
∣∣∣∣∣∣x1x2x3
∣∣∣∣∣∣ =[
x1 + 2x23x2 − 2x3
]. answer. A =
[L (e1) L (e2) L (e3]
]=
[1 2 0
0 3 −2
]
69
11. example. Let L : P1 → P1 be a linear transformation for which we knowthat L(t+ 1) = 2t+ 3 and L(t− 1) = 3t− 2. Find L(6t− 4). Find L(at+ b).
12. 6.1 Exercises: 1-4, 7-16, 20, 23
Week 12
7.1 Eigenvalues and Eigenvectors
1. Let A be an n × n matrix. If λ is a scalar (real or complex) and x 6= 0 is avector inRn or (Cn) such that Ax = λx then we say that λ is an eigenvalueof A and x is an eigenvector of A associated with A.
2. example 7.1.10.
3. definition 7.2
4. example 7.1.11.
5. example 7.1.12
6. example 7.1.13
7. exercises 7.1: 5-11
70