1

Lectures. MTH141: Linear Algebra Mathematics Department. Ryerson University

Instructor: Marcos Escobar (escobar@ryerson.ca, Eng213) Textbook: Contemporary Linear Algebra by Howard Anton and Robert C. Busby (Wiley, 2003) Course Website: Blackboard

2

What is linear algebra? Like calculus, it is an area of mathematics. Algebra – sets with binary operations and rules. Linear – additional conditions that make the structure “act like” straight lines. [hence the term “linear”] Recall that calculus started with two main problems or ideas.

1. Tangent line to a curve 2. Area under a curve.

These lead to the study of differential and integral calculus. We begin with the problem of solving linear equations.

3

CHAPTER 1 Vectors in the plane (R2) and three-space (R3). objects with length and

These vectors are the same since they have the same length and direction (equivalent).

direction

Vectors in the plane: Naming them: • (v1,v2 v = (v

) 1,v2)

Each vector in the plane corresponds in a one-to-one way with the points in the plane. Combining vectors: Geometrically and Algebraically Addition u = (u1,u2) v = (v1,v2

u u+v u + v = (u

)

1+v1 , u2+v2) v

4

Subtraction u u-v u - v = (u1-v1 , u2-v2

2u

) v Scalar Multiplication

u ku = k (u1,u2) = (ku1,ku2

) •

-u Vectors in 3-space. (u1,u2,u3

z

) Analogous geometric and algebraic rules.

(u1,u2,u3

y x Vectors from one point to another: See what vector you get when going from the point P=(1,5,-2) to Q=(3,4,5): Answer: QP=(2,-1,7). This vector is the same as performing u-v with u the vector (3,4,5) and v the vector (1,5,-2). Same in two dimensions.

) = u

5

Norm of a Vector and Vector Arithmetic Rules that hold for addition and scalar multiplication: 1. u+v =v+u 2. (u+v)+w = u+(v+w) 3. u+0=0+u=u 4. u+(-u)=0 5. a(bu)=ab(u) 6. a(u+v)=au+av 7. (a+b)u = au+bu 8. 1u = u These are the same as the rules for matrices. (recall we called nx1 and 1xn matrices column and row vectors.) Length of a vector v = (v1,v2) ||v|| = (v1

2 + v22 )1/2 for vectors in the plane

[ ||v|| = (v1

2 + v22 + v3

2 )1/2 for vectors in the 3-space ] Do problems 6 and 9 in class. 6. Find the length of (1/||v||) v [use the definition of length] Discuss the idea that normalizing v. 9. Use knowledge about triangles to show ||u+v|| ≤ ||u|| + ||v|| [look at the drawing we had of u+v earlier.

6

Dot Product (scalar product) (1.2) Geometrically u θ v u • v = ||u|| ||v|| cos θ if u ≠ 0 and v ≠ 0 0 u = 0 or v = 0 Algebraically: u • v = u1v1 + u2v2 in the plane u • v = u1v1 + u2v2 + u3v3

d) v • v > 0 if u ≠ 0, and v • v = 0 if v = 0.

in 3-space Observations If u ≠ 0 and v ≠ 0 then u ⊥ v if and only if u • v = 0. cos θ = u • v / ||u|| ||v|| OTHER PROPERTIES OF THE DOT PRODUCT a) u • v = v • u b) u • (v + w) = u • v + u • w c) k(u • v) = (ku) • v = u •(kv)

7

Orthogonal Projection u θ

proj

a

a u = u•a a ||a||

why? ||proj

2

( if θ>0 )

a u|| = cos θ = u•a ( solve for ||proja ||u|| ||u||||a|| What is the direction of ||proj

u|| )

a

proj

u|| ? Conclude

a u = u•a a ||a|| ||a||

8

EUCLIDEAN n-SPACE In previous slides we talked about vectors in 2 and 3 dimensional space. (a1,a2,a3) is point or vector in 3-space. This idea generalizes to higher dimensions. This is important since most applications in areas such as physics, economics, and statistics involve problems with more than three variables. Definition: If n is an positive integer and ai's are real numbers, then (a1,a2, . . . ,an) is an ordered n-tuple. The set of all possible n-tuples is called n-space (n dimensional space) or Rn

Suppose u = (u

. An n-tuple is called a point or vector in n-space. ALGEBRAIC DEFINITIONS AND RULES:

1,u2, . . . ,un) v = (v1,v2, . . . ,vn). We define u + v = u = (u1+v1, u2+v2, . . . , un+vn) ku = u = (ku1,ku2, . . . ,kun) 0 = (0,0, . . . ,0) It follows that -u = (-u1,-u2, . . . ,-un) u - v = u = (u1-v1, u2-v2, . . . , un-vn

9. u+v =v+u

) Theorem:

10.(u+v)+w = u+(v+w) 11.u+0=0+u=u 12.u+(-u)=0 13.a(bu)=ab(u) 14.a(u+v)=au+av

9

15.(a+b)u = au+bu 16.1u = u Generalized Dot Product - Euclidean Inner Product n

u • v = u1v1 + u2v2 + . . . + unvn = ∑ uivi i = 1 (5,4,1,2,3) • (-2,3,1,4,1) = -10 + 12 + 1 + 8 + 3 = 14 PROPERTIES OF DOT PRODUCT a) u • v = v • u b) (u +v) • w = u • w + v • w c) k(u • v) = (ku) • v = u •(kv) d) v • v > 0 if u ≠ 0, and v • v = 0 if v = 0. Proof of b) and d). (u+v)• w = ∑ (ui +vi ) wi = ∑ (ui wi + vi wi) = ∑ ui wi + ∑vi wi = u • w + v • w v • v = ∑ vivi = ∑ vi

2 > 0 v • v = 0 iff ∑ vi

2 = 0 iff vi = 0 for each i. Euclidean Norm (length) ||u|| = (u • u)1/2 = (u1

2 +u22 +. . . + un

2 )1/2 Distance d(u,v) = ||u-v|| = ((u1

-v1)2 +(u2 -v2)2. . . + (un

-vn)2 )1/2 Cauchy Schwarz Inequality (for Rn) If u and v are in Rn then |u • v| ≤ ||u|| ||v||

10

Proof: (Special Cases: 2 and 3 dimension.) |u • v| = | ||u|| ||v|| cos θ | ≤ ||u|| ||v|| (since |cos θ| ≤ 1) FACTS ABOUT LENGTH AND DISTANCE Length a) ||u|| ≥ 0 b) ||u|| = 0 iff u = 0 c) ||ku|| = |k| ||u|| d) ||u +v|| ≤ ||u|| + ||v|| Proof of d. (using the Cauchy-Schwarz inequality ||u +v||2 = (u +v)•(u +v) = u • u + 2u•v + v • v = ||u ||2 + 2u•v + ||v ||2 ≤ ||u ||2 + 2 ||u || || v|| + ||v ||2 = (||u|| + ||v||)2 Distance a) d(u,v) ≥ 0 b) d(u,v) = 0 iff u=v c) d(u,v) = d(v,u) d) d(u,v) ≤ d(u,w) + d(w,v) Proof of d. d(u,v) = ||u -v|| = ||u-w + w-v|| ≤ ||u -w|| + ||w -v|| = d(u,w) + d(w,v) Fact: u•v = 1/4 ||u +v||2 - 1/4 ||u -v||2 Proof: (see work done earlier in class to get the first equation) ||u +v||2 = ||u ||2 + 2u•v + ||v ||2

||u - v||2 = ||u ||2 - 2u•v + ||v ||2 Subtract second from first and solve for u•v.

11

ORTHOGONAL (generalization of perpendicular to n-space) The vectors u and v are called orthogonal iff u•v = 0. Theorem: u and v are orthogonal iff ||u +v||2 = ||u||2 + ||v||2 Proof: ||u +v||2 = (u +v)•(u +v) = u • u + 2u•v + v • v = ||u ||2 + 2u•v + ||v ||2 = ||u ||2 + 0 + ||v ||2 = ||u ||2 + ||v ||2

u

NOTATION FOR VECTORS IN n-SPACE

1 u = u2 (column vector) or u = (u1, u2, . . . , un) (row vector) . un Using column vectors then one gets the following u•v = vTu (do an example)

12

Lines and Planes (1.3) Lines in 2-space. Standard Equation of a line in R2: Ax+By+C=0. Where –C/B is the interception with the y-axis and A/B is the slope. Alternative expression (parametric equation): Assume a point on the line (x0,y0) and a vector v=(a,b) with the direction of the line are given then the equation of the line in vector notation is: P0P=tv or (x,y)= (x0,y0)+tv where P is any point in the line and t is a scalar called the parameter. Note the parametric equation implies (x- x0)/a=(y- y0)/b <=> (1/a) x + (-1/b) y + (y0/b- x0/a)=0 so we have a relationship between A,B,C and a,b,x0,y0. The parametric approach (vector notation) is beneficial because it is intuitive and it can be extended to n-dimensions. Lines and Planes in 3-space Straight lines in 3-space Given a point in the line (x0, y0, z0) and a vector v=(a,b,c) in the direction of the line then the parametric equation of the line in vector notation is: (x,y,z)=(x0,y0,z0)+t v.

13

x = x0 + at y = y0 + bt Line parallel to vector v=(a,b,c) through (x0 , y0 , z0 ) z = z0 + ct or (x - x0)/a = (y - y0)/b =(z- z0)/c Planes in 3-space. Equation of a plane in vector notation: n•P0P=0 where P0 is a point in the plane. P is any other point in the plane and n is a normal vector. (a,b,c)• (x-x0,y-y0,z-z0)=0 <=> a(x-x0)+b(y-y0)+c(z-z0

n

)=0 <=> ax+by+cz+d=0

P P0 . normal vector (a,b,c)=n If d=0 then it goes through the origin (0,0,0) Recall n•P0P=0, therefore nP0=0 if (0,0,0) in on the plane. Parametric equation of a Plane in vector notation: Assume a point P0 and two vectors v1 , v2 (both in the direction of the plane) are given, then the parametric equation is:

14

P0P=t1v1+t2v2. Where P is any point in the plane and t1 and t2 are the parameters. Examples: 1 – Equation of a Plane passing through the point P1=(1,2,-1) with normal n=(3,2,-5). 2 – Equation of a Plane passing through the point P1=(1,2,-1), P2=(2,3,1), P3=(3,-1,2). 3 - Equation of a line passing through the points P1=(2,4,-1) and P2=(5,0,7).

15

Chapter 2: Linear Equations (2.1) Variables x1 , x2 , x3 , ... , xn Constants a1 , a2 , a3 , ... , an , b Linerar equation a1 x1 + a2x2 + a3x3 + ... + anxn = b n ∑ aixi = b i=1 Remark: xi has to appear linear, i.e. xi

2 or sin(xi) are not allowed Definition: s1 , s2 , s3 , ... , sn is called a solution of the linear equation if a1 s1 + a2s2 + a3s3 + ... + ansn = b Example: 1, 1 is a solution to 3x+2y = 5 (or 3x1 + 2x2 = 5) Solving: x = 5/3 - 2/3 y Parametric form of the solution (general solution) x = 5/3 - 2/3 t y = t specific solutions (t=1) x =1 y = 1 (t=2) x = 1/3 y = 2

16

Example 2 x1 - 5x2 + x3 - 4x4 = 6 (or x1 = 3 + 5/2 x2 - x3/2 + 2x4 ) Solution x1 = 3 + 5/2 s - t/2 + 2u x2 = s x3 = t x4

= u System of Linear equations - finite set of linear equations A solution to the system must satisfy all the equations. A solution is the intersection of the solution sets of each equation. Example: 2 equations in 2 unknowns. Solution is the intersection of 2 straight lines in the plane

Consistent (one solution) Consistent (infinite solutions) Inconsistent (no solutions)

17

General System of Linear Equations a11 x1 + a12x2 + a13x3 + ... + a1nxn = b1 a21 x1 + a22x2 + a23x3 + ... + a2nxn = b2 . . am1 x1 + am2x2 + am3x3 + ... + amnxn = bm aij ith row jth

a

column Augmented Matrix (shorthand)

11 a12 a13 ... a1n b1 a21 a22 a23 ... a2n b2 am1 am2 am3 ... amn b

What is the augmented matrix for the following? x

m

1 - 2x2 + 3x3 = 5 2 x1 + x2 - 7x3 = 6 To solve systems of equations we manipulate these matrices.

18

Elementary Row Operations Three operations can be used to change equations (augmented matrices) and not change the solution set. 1) Multiply a equation (row) by a non-zero constant 2) Interchange 2 equations (rows). 3) Add a multiple of an equation (row) to another. Example: (Gauss Jordan Reduction) x1 + 2x2 = 3 3 x1 - x2

1 2 3

= -5

3 -1 -5 -3I+II 1 2 3 0 -7 -14 -1/7 II 1 2 3 0 1 2 -2II + I 1 0 -1 0 1 2 Solution x1 = -1 x2 = 2

19

Gaussian Elimination (2.2) A matrix is in reduced row-echelon form (RREF) if it has the following properties: 1. If any row is not all zeros then the first non-zero number is a 1.

2. All rows that are entirely zero are at the bottom of the matrix.

3. The leading 1 in any row is farther to the right than the leading

one in any row above it.

4. Each column with a leading 1 has zero everywhere else.

0 1 2 0 0 5 0 0 0 1 0 6 0 0 0 0 1 8 0 0 0 0 0 0

A matrix is in row-echelon form (REF) if it satisfies 1-3 above.

0 1 2 6 0 5 0 0 0 1 2 6 0 0 0 0 1 8 0 0 0 0 0 0

Remark: Gauss elimination takes a matrix to its REF while a Gauss-Jordan takes a matrix to its RREF.

20

Solving RREF matrices

1 2 0 0 3 0 4 0 0 1 0 1 0 5 0 0 0 1 2 0 6 0 0 0 0 0 1 2

x1 = 4 - 2x2 - 3x5 x3 = 5 - x5 x4 = 6 - 2x5 x6 = 2 x1 = 4 - 2s - 3t x2 = s x3 = 5 - t x4 = 6 - 2t x5 = t x6 = 2 Solving a matrix in REF. 1 2 4 5 -3 6 0 0 1 -1 2 5 0 0 0 0 1 3 x1 = 6 - 2x2 - 4x3 - 5x4 + 3x5 x3 = 5 + x4 - 2x5 x5 = 3 x1 = 6 - 2s - 4(t-1) + 5t + 3(3) = 19 - 2s + t x2 = s x3 = 5 + t - 2(3) = t-1 x4 = t

21

x5 = 3

22

Example of Gauss-Jordan Reduction (Process that leads to a RREF)

0 0 -2 0 7 6 3 6 0 9 18 12 2 4 5 6 -5 -2

switch I and II 3 6 0 9 18 12 0 0 -2 0 7 6 2 4 5 6 -5 -2

1/3 I 1 2 0 3 6 4 0 0 -2 0 7 6 2 4 5 6 -5 -2

-2I + III 1 2 0 3 6 4 0 0 -2 0 7 6 0 0 5 0 -17 -10

-1/2 II 1 2 0 3 6 4 0 0 1 0 -7/2 -3 0 0 5 0 -17 -10

-5II + III 1 2 0 3 6 4 0 0 1 0 -7/2 -3 0 0 0 0 1/2 5

2 III 1 2 0 3 6 4 0 0 1 0 -7/2 -3 0 0 0 0 1 10

-6 III + I 1 2 0 3 0 -56 0 0 1 0 -7/2 -3 0 0 0 0 1 10

7/2 III + II 1 2 0 3 0 -56 0 0 1 0 0 32 0 0 0 0 1 10

23

x1 = -56 - 2s - 3t x2 = s x3 = 32 x4 = t x5 = 10

24

Homogeneous Systems of Linear Equations a11 x1 + a12x2 + a13x3 + ... + a1nxn = 0 a21 x1 + a22x2 + a23x3 + ... + a2nxn = 0 . . am1 x1 + am2x2 + am3x3 + ... + amnxn

• A homogeneous systems stays homogeneous after applying an elementary row operation.

= 0

• A homogeneous system is always consistent (it has one or more solutions). All xi

This is called the = 0 works. trivial solution

• Either there is only the trivial solution or there are infinitely many solutions.

.

• Is there a way to tell if there are infinitely many solutions without solving.

Theorem: A homogeneous system of linear equations with more unknowns than equations has infinitely many solutions. [n>m] idea of proof: After applying Gauss-Jordan method, since there are more variables than equations, not all the variables are leading 1's so some will be free to vary (will be a parameter). 0 1 0 2 0 -2 0 x2 = -2x4 + 2x6 0 0 1 3 0 1 0 x3 = -3x4 - 2x6 0 0 0 0 1 0 0 x5 = 0 x1 = s x2 = -2t + 2u x3 = -3t - u x4 = t

25

x5 = 0 x6 = u