linear algebra. session 1roquesol/math_304_fall... · introduction sistems of linear equations...

IntroductionSistems of linear equations

Gaussian Elimination. Gauss-Jordan Reduction

Linear Algebra. Session 1

Dr. Marco A Roque Sol

08/01/2017

Dr. Marco A Roque Sol Linear Algebra. Session 1



Table of contentsIntroductionSystems of linear equationsGaussian elimination, Gauss-Jordan reductionMatrices, matrix algebraDeterminantsVector spacesLinear independenceBasis and dimensionCoordinates, change of basisLinear transformationsOrthogonalityInner products and normsThe Gram-Schmidt orthogonalization processEigenvalues and eigenvectorsMatrix exponentials and DiagonalizationTopics in applied linear algebra




Introduction

Linear Algebra

( Medical Imaging )Dr. Marco A Roque Sol Linear Algebra. Session 1



Introduction

Linear algebra is a cornerstone in undergraduate mathematicaleducation. It develops a general language used by all scientists andis interdisciplinary in essence. It hence, evolves naturally towardsabstraction. For most students, it is a first contact with modernmathematics.

Here are some concrete areas where we can find applications ofLinear Algebra.

Abstract Thinking

Chemistry

Coding theory

Coupled oscillations

Cryptography

Economics




Introduction

Classical Electromagnetism.

Geophysics.

Elimination Theory.

Games.

Genetics.

Geometry.

Graph theory.

Heat distribution.

Image compression.




Introduction

Linear Programming.

Markov Chains.

Networks.

Sociology

The Fibonacci numbers.

Eigenfaces.




Introduction

Definition (Linear Equation)An equation of the form ax + by = c (5x + 7y = 6) is called linearbecause its solution set is a straight line in R2

A solution of the equation is a pair of numbers (x0, y0) ∈ R2 suchthat ax0 + by0 = c (5x0 + 7y0 = 6).

For example, (1, 1/7) and (−1/5, 1), are solutions. In another way,we can write the first solution, as x = 1, y = 1/5 and the secondone, as x = −1/5, y = 1

And the graph of the lines is:




Introduction




Sistems of linear equations

General equation of a line

The General equation of a line is given by

ax0 + by0 = c

where x , y are variables and a, b, and c are constants (except forthe case a = b = 0 )

DefinitionA linear equation in the variables x1, x2, · · · , xn, is an equation ofthe form:

a1x1 + a2x2 + · · · anxn = b

where a1, a2, · · · an and b are constants.

A solution of the equation is an array of numbersα1, α2, · · ·αn ∈ Rn such that

a1α1 + a2α2 + · · ·+ anαn = b




System of linear equations


A System of linear equations is an expression of the form:a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2...

am1x1 + am2x2 + · · ·+ amnxn = bm

Here x1, x2, · · · xn are variables and aij , bj are constants.

In many applications, we will find such a systems of equations andit is here, where all the study of Linear Algebra Starts.




Systems of linear equations

Example 1.1

Find the point of intersection of the lines x − y = −2 and2x + 3y = in R2. You can find the solution of this problem bysetting up and solving the system of two equations in twounknowns {

x − y = −22x + 3y = 6

⇔ ( Equivalent System){x = y −2

2x + 3y = 6

⇔ {x = y − 2

2(y − 2) + 3y = 6





⇔ {x = y − 2

5y = 10

⇔ {x = y − 2y = 2

⇔ {x = 0y = 2

Thus, the solution is given by the point (0, 2)





{x − y = −2

2x + 3y = 6x = 0; y = 2





In a similar way, we have

{2x + 3y = 22x + 3y = 6

inconsistent system (no solution)





{4x + 6y = 122x + 3y = 6

2x+3y = 6 (infinitely many solutions)





Solving systems of linear equations

Elimination Method

Algorithm:(1) pick a variable, solve one of the equations for it, and eliminateit from the other equations;

(2) put aside the equation used in the elimination, and return tostep (1).

The algorithm reduces the number of variables (as well as thenumber of equations), hence it stops after a finite number of steps.After the algorithm stops, the system is simplified so that it shouldbe clear how to complete solution





Example 1.2 x − y = 2

2x − y− z = 3x + y+ z = 6

Solve the 1st equation for x :x = y + 2

2x − y− z = 3x + y+ z = 6

⇔Eliminate x from the 2nd and 3rd equations:

x = y + 22(y + 2) − y− z = 3(y + 2) + y+ z = 6





⇔Simplify:

x = y+ 2y− z = −12y + z = 4

In this way we have that, the whole system has been reduced to asystem (2nd and 3rd equations) of two linear equations in twovariables.

Solve the 2nd equation for y :x = y+ 2y = z− 12y + z = 4

⇔Dr. Marco A Roque Sol Linear Algebra. Session 1




Eliminate y from the 3rd equation:x = y+ 2y = z+ 1

2(z − 1) + z = 4

⇔

Simplify x = y+ 2y = z+ 13z = 6

⇔

Thus, the elimination process, has been completed. Now, thesystem is easily solved by back substitution





That is, we find z from the 3rd equation, then substitute it in the2nd equation and find y , then substitute y and z in the 1stequation and find x .

x = y + 2y = z + 1

z = 2

x = y + 2y = 1z = 2

x = 3y = 1z = 2

Finally, the System of linear equations:x − y = 2

2x − y− z = 3x + y+ z = 6

has as a solution, the point (x , y , z) = (3, 1, 2) .





Gaussian elimination


Remember that a System of linear equations is an expression ofthe form:

a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2...

am1x1 + am2x2 + · · ·+ amnxn = bm

Here x1, x2, · · · xn are variables and aij , bj are constants.

A solution of the system is a common solution of all equationspresent in the system.

A system of linear equations can have one solution, infinitely manysolutions, or no solution at all.





Using the Elimination method we had already shown the first caseand using it again we can illustrate the other two extra cases

Example 1.3 x + y− 2z = 1

y− z = 3−x + 4y− 3z = 14

Solve the 1st equation for x :⇔ ( Equivalent Systems)

x = −y+ 2z+ 1y − z = 3

−x + 4y− 3z = 14

Eliminate x from the 3rd equation :




⇔





⇔ x = −y + 2z + 1

y − z = 3−(−y + 2z + 1) + 4y − 3z = 14

Simplify :⇔

x = −y + 2z + 1y − z = 3

5y − 5z = 15⇔Solve the 2nd equation for y

x = −y+ 2z+ 1y = z+ 35y − 5z = 15





Eliminate y from the 3rd equation:x = −y+ 2z+ 1y = z+ 3

5(z + 3)− 5z = 15

⇔

Simplify : x = −y+ 2z+ 1y = z + 3

0 = 0

Now, the elimination process is completed. The last equation isactually 0z = 0. Hence z , is a free variable, that is, it can beassigned an arbitrary value. Then, x and y are found by backsubstitution .





z = t (a parameter)

y = z + 3x = −y + 2z + 1

⇔ z = t

y = t + 3x = t − 2

Thus, the system x + y− 2z = 1

y − z = 3−x + 4y− 3z = 14





has the General Solution

(x , y , z) = (t − 2, t + 3, t)

or in vector form

(x , y , z) = (−2, 3, 0) + t(1, 1, 1)

The set of all solutions is a straight line in R3 passing through thepoint P = (−2, 3, 0) in the direction v =< 1, 1, 1 > .





Example 1.4

x + y− 2z = 1

y− z = 3−x + 4y− 3z = 1

Solve the 1st equation for x :

⇔ (Equivalent System )x = −y + 2z + 1

y − z = 3−x + 4y − 3z = 1





⇔Eliminate x from the 3rd equation:

x = −y + 2z + 1y − z = 3

−(−y + 2z + 1) + 4y − 3z = 1⇔Simplify:

x = −y + 2z + 1y − z = 3

5y − 5z = 2⇔Solve the second equation for y :

x = −y + 2z + 1y = z + 3

5y − 5z = 2Dr. Marco A Roque Sol Linear Algebra. Session 1




⇔Eliminate y from the 3rd equation:

x = −y + 2z + 1y = z + 3

5(z + 3)− 5z = 2⇔Simplify:

x = −y + 2z + 1y = z + 3

15 = 2

Now, the elimination process is completed. The last equationactually is giving us a contradiction. Hence, there is no solution forthis system.





Thus, the system x + y− 2z = 1

y− z = 3−x + 4y− 3z = 1

has the General Solution

φ = empty set





Gaussian elimination

Gaussian elimination is a modification of the elimination methodthat allows only so-called elementary operations .

Elementary operations for systems of linear equations:

(1) to multiply an equation by a nonzero scalar;

(2) to add an equation multiplied by a scalar to another equation;

(3) to interchange any two equations.





The next result, is very important, because it ensures that thisoperations will not modify the solution set of the originall system.

Theorem

(i) Applying elementary operations to a system of linear equationsdoes not change the solution set of the system.

(ii) Any elementary operation can be undone by anotherelementary operation.





Operation 1: Multiply the ith equation by r 6= 0

a11x1 + a12x2 + · · ·+ a1nxn = b1...

ai1x1 + ai2x2 + · · ·+ ainxn = bi...

am1x1 + am2x2 + · · ·+ amnxn = bm

⇒

a11x1 + a12x2 + · · ·+ a1nxn = b1...

(rai1)x1 + (rai2)x2 + · · ·+ (rain)xn = rbi...

am1x1 + am2x2 + · · ·+ amnxn = bm

To undo the operation, multiply the ith equation by r−1





Operation 2: Add r times the ith equation to the jth equation

...ai1x1 + ai2x2 + · · ·+ ainxn = bi

...aj1x1 + aj2x2 + · · ·+ ajnxn = bj

...

⇒

...ai1x1 + ai2x2 + · · ·+ ainxn = bi

...(aj1 + rai1)x1 + (aj2 + rai2)x2 + · · ·+ (ajn + rain)xn = bj + rbi

...

To undo the operation, add −r times the ith equation to the jthequation .





Operation 3: interchange the ith and jth equations.

...ai1x1 + ai2x2 + · · ·+ ainxn = bi

...aj1x1 + aj2x2 + · · ·+ ajnxn = bj

...

⇒

...aj1x1 + aj2x2 + · · ·+ ajnxn = bj

...ai1x1 + ai2x2 + · · ·+ ainxn = bi

...

To undo the operation, apply it once more .





Example 1.5 x − y = 2

2x − y − z = 3x + y + z = 6

Add −2 times the 1st equation to the 2nd equation:x − y = 2

y − z = −1x + y + z = 6

R2 := R2 − 2 ∗ R1

Add −1 times the 1st equation to the 3rd equation:x − y = 2

y − z = −12y + z = 4

R3 := (−1) ∗ R1 + R3





Add −2 times the 2nd equation to the 3rd equation:x − y = 2

y − z = −13z = 6

R3 := (−2) ∗ R2 + R3

Note: At this point, the elimination process is completed, and wecan solve the system by back substitution. However, we cancontinue with elementary operations

Multiply the 3rd equation by 1/3:x − y = 2

y − z = −1z = 2

R3 := (1/3) ∗ R3





Add the 3rd equation to the 2nd equation:x − y = 2

y = 1z = 2

R2 := R2 + R3

Add the 2nd equation to the 1st equation:x = 3

y = 1z = 2

R1 := R2 + R1

Thus, the system x − y = 2

2x − y − z = 3x + y + z = 6

has as the solution set, the point (3, 1, 2)Dr. Marco A Roque Sol Linear Algebra. Session 1



Row echelon form. Gauss-Jordan Reduction

Matrices

Definition. A matrix is a rectangular array of numbers.

The dimension of a matrix is given by

dimensions = (number of rows) X ( number of columns)

Thus we have

n × n : Square matrixn × 1 : Column vector1× n : Row vector





Examples of matrices are:

2 7−1 03 3

(3× 2)

(2 7 0−1 1 5

)(2× 3)

358

(3× 1)

(2 4 9

)(1× 3)

(−2 01 5

)(2× 2)





From a system of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1

a21x1 + a22x2 + · · ·+ a2nxn = b2...

am1x1 + am2x2 + · · ·+ amnxn = bm

We can find the coefficient matrix and column vector of theright-hand sides:

a11 a12 · · · a1na21 a22 · · · a2n

...am1 am2 · · · amn

b1

b2...

bm





and also associated to the linear system we have the Augmentedmatrix:

a11 a12 · · · a1n b1

a21 a22 · · · a2n b2...

am1 am2 · · · amn bm

Now, rember that using Gaussian elimination, the solution of asystem of linear equations splits into two parts:(A) Elimination and(B) Back substitution.





Both parts can be done by applying a finite number of elementaryoperations:

(1) to multiply a row by a nonzero scalar;

(2) to add the ith row multiplied by some r ∈ R to the jth row;

(3) to interchange two rows.

Notation





Augmented matrix:

a11 a12 · · · a1n b1

a21 a22 · · · a2n b2...

am1 am2 · · · amn bm

=

v1v2...

vm

where vi = (ai1 ai1 ai1 · · · ai1) is a row vector.





Elementary row operations on Matrices

Operation 1: To multiply the ith row by r 6= 0

v1...

vi...

vm

⇒

v1...

rvi...

vm





Operation 2: To add the ith row multiplied by r to the jth row

v1...

vi...

vj...

vm

⇒

v1...

vi...

vj + rvi...

vm





Operation 3: To interchange the ith row with the jth row

v1...

vi...

vj...

vm

⇒

v1...

vj...

vi...

vm





Row echelon form

Definition. Leading entry of a matrix is the first nonzero entry ina row.

The goal of the Gaussian elimination is to convert the augmentedmatrix into row echelon form

1) Leading entries shift to the right as we go from the first row tothe last one;

2) Each leading entry is equal to 1.





Thus, we have this example in the echelon form

1 −1 4 1 7 9 6 1 3 9 −50 1 3 −2 7 1 5 3 3 4 −10 0 0 0 1 5 6 −3 1 2 10 0 0 0 0 0 1 8 1 7 20 0 0 0 0 0 0 1 −2 7 −30 0 0 0 0 0 0 0 1 −2 90 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0





Row echelon form

General augmented matrix in row echelon form

� ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗� � � ∗ ∗ ∗ ∗ ∗ ∗ ∗

� � ∗ ∗ ∗ ∗ ∗� ∗ ∗ ∗ ∗

� � ∗ ∗� ∗

�

1) Leading entries are boxed (all equal to 1);

2) All the entries below, the (imaginary) staircase line are zero;





3) Each step of the (imaginary) staircase has height 1;

4) Each circle marks a column without a leading entry thatcorresponds to a free variable.

Strict triangular form

Is a particular case of row echelon form that can occur for systemsof n variables :





� ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗� ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

� ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗� ∗ ∗ ∗ ∗ ∗ ∗ ∗

� ∗ ∗ ∗ ∗ ∗ ∗� ∗ ∗ ∗ ∗ ∗

� ∗ ∗ ∗ ∗� ∗ ∗ ∗

� ∗ ∗� ∗

1) No zero rows.

2) No free variables.





Consistency check

The original system of linear equations is consistent if there is noleading entry in the rightmost column of the augmented matrix inrow echelon form.

Augmented matrix of an inconsistent system

� ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗� � � ∗ ∗ ∗ ∗ ∗ ∗ ∗

� � ∗ ∗ ∗ ∗ ∗� ∗ ∗ ∗ ∗

� � ∗ ∗� ∗

�





The goal of the Gauss-Jordan reduction is to convert theaugmented matrix into reduced row echelon form :

1 ∗ ∗ ∗ ∗ ∗1 � � ∗ ∗ ∗

1 � ∗ ∗1 ∗ ∗

1 � ∗1 ∗

1) All entries below the staircase line are zero; 2) Each boxed entryis 1, the other entries in its column are zero; 3) Each circlecorresponds to a free variable.





Example 1.6From a previous example, we have

x − y = 1

2x − y− z = 3x + y+ z = 6

⇒

1 −1 0 22 −1 −1 31 1 1 6

Row echelon form (also strict triangular):

x − y = 2y − z = −1

x + y+ z = 2⇒

1 −1 0 2

0 1 −1 −1

0 0 1 2





Reduced row echelon form :x = 3

y = 1z = 2

⇒

1 0 0 3

0 1 0 1

0 0 1 2

Example 1.7

x + y− 2z = 1

y− z = 3−x + 4y− 3z = 6

⇒

1 1 −2 10 1 −1 3−1 4 −3 1





Row echelon form:

x + y− 2z = 1

y− z = 30 = 1

⇒

1 1 −2 1

0 1 −1 3

0 0 0 1

Reduced row echelon form:

x + − z = 0y− z = 0

0 = 1⇒

1 0 −1 0

0 1 −1 0

0 0 0 1

Inconsistent system





How to solve a system of linear equations

1) Order the variables.

2) Write down the augmented matrix of the system.

3) Convert the matrix to row echelon form

4) Check for consistency.

5) Convert the matrix to reduced row echelon form Example 7

6) Write down the system corresponding to the reduced rowechelon form.





7) Determine leading and free variables.

8) Rewrite the system so that the leading variables are on the leftwhile everything else is on the right.

9) Assign parameters to the free variables and write down thegeneral solution in parametric form.

Example 1.7 {x2 + 2x3 + 3x4 = 6

x1 + 2x2 + 3x3 + 4x4 = 10

In this case the variables are x1, x2, x3, x4. and the Augmentedmatrix is :





(0 1 2 3 61 2 3 4 10

)To get it into row echelon form, we exchange the two rows:(

1 2 3 4 100 1 2 3 6

)Consistency check is passed. To convert into reduced row echelonform, add −2 times the 2nd row to the 1st row:(

1 0 −1 −2 −2

0 1 2 3 6

)The leading variables are x1 and x2 ; hence x3 and x4 are freevariables





Back to the system:{x1 − x3 − 2x4 = −1x2 + 2x3 + 3x4 = 6

⇒{

x1 = x3 + 2x4 − 2x2 = −2x3 − 3x4 + 6

and the general solution is given byx1 = t + 2s − 2x2 = −2t − 3s + 6x3 = tx4 = s

(t, s ∈ R)

In vector form,

(x1, x2, x3, x4) = (−2, 6, 0, 0) + t(1,−2, 1, 0) + s(2,−3, 0, 1)





Example 1.8 y + 3z = 0

x + y − 2z = 0x + 2y + az = 0

a ∈ R

The system is homogeneous (all right-hand sides are zeros).Therefore it is consistent ( x = y = z = 0 is a solution )The augmented matrix is: 0 1 3 0

1 1 2 01 2 a 0





Since the 1st row cannot serve as a pivotal one, we interchange itwith the 2nd row: 0 1 3 0

1 1 −2 01 2 a 0

⇒

1 1 −2 00 1 3 01 2 a 0

Now we can start the elimination. First subtract the 1st row fromthe 3rd row: 1 1 −2 0

0 1 3 01 2 a 0

⇒

1 1 −2 00 1 3 00 1 a + 2 0





The 2nd row is our new pivotal row. Subtract the 2nd row fromthe 3rd row: 1 1 −2 0

0 1 3 00 1 a + 2 0

⇒

1 1 −2 00 1 3 00 0 a− 1 0

At this point row reduction splits into two cases.Case 1: a 6= 1. In this case, multiply the 3rd row by (a− 1)−1 1 1 −2 0

0 1 3 00 0 a− 1 0

⇒

1 1 −2 00 1 3 00 0 1 0





The matrix is converted into row echelon form. We proceedtowards reduced row echelon form. Subtract 3 times the 3rd rowfrom the 2nd row: 1 1 −2 0

0 1 3 00 0 1 0

⇒

1 1 −2 00 1 0 00 0 1 0

Add 2 times the 3rd row to the 1st row: 1 1 −2 0

0 1 0 00 0 1 0

⇒

1 1 0 00 1 0 00 0 1 0





Finally, subtract the 2nd row from the 1st row: 1 1 0 00 1 0 00 0 1 0

⇒

1 9 0 00 1 0 00 0 1 0

Thus x = y = z = 0 is the only solution





Case 2: a = 1. In this case, the matrix is already in row echelonform: 1 1 −2 0

0 1 3 00 0 a− 1 0

⇒

1 1 −2 00 1 3 00 0 0 0

To get reduced row echelon form, subtract the 2nd row from the1st row: 1 1 −2 0

0 1 3 00 0 0 0

⇒

1 0 −5 00 1 3 00 0 0 0





Then, z is a free variable, and the solution is given by{x − 5z = 0y + 3z = 0

⇒{

x = 5zy = −3z

Thus, the System of linear equations:x + 3z = 0

x + y − 2z = 0x + 2y + az = 0

has as a solution:

if a 6= 1 then (x , y , z) = (0, 0, 0).if a = 1 then (x , y , z) = t(5,−3, 1) t ∈ R.


linear algebra. session 1roquesol/math_304_fall... · introduction sistems of linear equations...

Documents