System of Linear EquationsDefinitions :

System of Linear Equations

1. Augmented Matrix : M = [A b]

2. Degenerate Equation : All coefficients aLj = 0 for equation “L”

Theorem :

Consider a system of linear equations that contains a degeneratey q g

equation L, say with constant b:

(i) If b = 0 => system has no solution (why ?)

(ii) If b = 0 => equation “L” can be deleted (why ?)

Equivalent Systems of EquationsEquivalent Systems of EquationsObservation : multiplying both sides from left by a matrix T ychanges A & b but x is not changed !

How to Choose T ?Theorem :

How to Choose T ?

Two systems of linear equations are said to be equivalent (have same solution) if and only if each equation in one system is obtained by applying elementary operations to the equations in the other systemapplying elementary operations to the equations in the other system (as in multiplying by matrix T from the left)

Definition :

Elementary operations include

(i) Interchanging 2 equations Ri Rj(ii) Replace equation by a non-zero multiple of other equation plus itself

jji RRKR

Elementary OperationsElementary Operations

MATLAB f(A) l f iMATLAB : rref(A) ; see also rrefmovie

Idea : Apply elementary operations to original system of linear

equations to transform it to another system whose solution can be

easily obtained by back substitution. This transformation

is called Gaussian Elimination

Example 1(Unique Solution)Example 1(Unique Solution)

2 428342623623

221

RRRzyzyxzyxzyx

313235863 331 RRRzyzyx

2314742

623

RRRzzyzyx

ofsystemoriginalinsolutionsubstitutealways(1;3;2

23 147 332

xyzRRRz

)!answer!your check toequationsofsystemoriginalin solution substitute always(

Example 2 (Infinite Solutions)

xxxxx522

152462 54321

system of equations in

echelon form

xxxxx

first)(except equation each in unknown Leading693

522

54

543

whileablespivot vari called are } x, x, x{ equation preceding in theunknown leading theofright the tois

)( pqg

431

byedparametrizsolutionsofnumber infinitean has equations of system thisHence, any value.

assumecan i.e. ; variablesfree called are } x,{x 52

tttxtxtxs

813/)96(225 ; 3/)96(numbers realany are t and s re whe ; x

by edparametriz solutions of

34

52

ststttx 3942/)6)81()3/)96(4(215()(;)(

1

34

Gaussian Elimination ProcedureGaussian Elimination Procedure

Find the first unknown in the system with a non-zero coefficient

(i) Arrange so that a11 is not equal to 0 (by interchanging equations)

(ii) Use a11 as the “pivot” to eliminate X1 from all equations except the first, i.e. multiply by m = -ai1/a11 ii RRmR 1p y y i1 11

to avoid fractions use iii

ii

RRaRa 1111

1

(i) Repeat elimination step for the other pivots to put system in echelon form where (1) the non-zero rows lie above any zero rows and (2) the first non-zero entry (pivot) in a non-zero row lies to the ( ) y (p )right of the first non-zero entry (pivot) in the row above it

Echelon Form

formthehasformechelon in equationsofsystemslinear A :Definition

qy

121222 2222

11212111

nnjjjj

nn

bxaxaxabxaxaxa

11

rnrnjrjjrj bxaxaxarrrr

ablespivot vari called are x, , x, x variablesThen r and zeronot are sa' theand j j 1 where

r2 jj1

r2

i blf)(tlbitiequations than unknowns more n r If (ii)

solution unique n r If (i)

variablesfreer)-(n tovaluesarbitrary assign

Rank of a MatrixRank of a Matrix

D fi iti Th k f t i A itt k(A) i l t thDefinition : The rank of a matrix A, written rank(A), is equal to the

number of pivots in an echelon form of A or equivalently the number of non-zero rows in an echelon form for A

Facts : 1) Reducing the augmented matrix M = [A b] to echelon form

solves the system of equations Ax = b by back-substitution

2) For an m x n matrix A;

nrank(A) and mrank(A) n)min(m,rank(A)0 Hence

Classifying Solutionsy g

• Theorem : Let M=[A b] be the m x (n+1)Theorem : Let M [A b] be the m x (n 1) augmented matrix of an m x n system of linear equations (S), then :q ( )

1) If rank (A) < rank (M) , then S has no solutions2) If rank (A)=rank (M)=n, S has unique solution2) If rank (A) rank (M) n, S has unique solution3) If rank (A)=rank (M)<n, than S has infinitely

many solutionsmany solutions

Remark : note that )()(0 MrankArank Remark : note that )()(0 MrankArank

Example 1

rank(M)rank(A) :1 Case] [

bAM

2,4

( )( )

nm

21

21

01

21

21

01

11

11

11

312

002

000

~

332

022

000

~

651

341

321

M

3rank(M)2rank(A)

300300633

solution No nt!Inconsiste 3rank(M)2rank(A)

Example 2

2n 4,mnrank(M)rank(A) :2 Case

001111

,

01

01

00

~41

41

21

M

000333

0x 1;x :solution Unique nrank(M)2rank(A)

12

12

Example 3

3n 3,m

nrank(M)rank(A) :3 Case

330011421

,

M 01

013

01

00~

31

65

125

52

n 32rank(M)rank(A)

We have 1 free variable

tt

n

303 x;131 xt; xby edparametriz solutions Infinite

32rank(M)rank(A)

123

;; 123

Homogeneous System of EquationsHomogeneous System of Equations

A 0 i i l f A b P t it i h l f thAx=0 is a special case of Ax=b. Put it in echelon form, then

1) If r = n (full rank), system has only the zero solution (unique sol.)) ( ), y y ( q )2) If r < n (rank-deficient), system has infinite solutions because

we have free variables

Question : Can a homogeneous system have no solution ? Why ?

Note that when a homogeneous system of equations has more unknowns than equations, then there are infinite solutions while qif the number of equations equals the number of unknowns, either we have infinite solutions or only the zero solution

Example 1p

More equations than unknowns (over-determined)q ( )

Question : for a homogeneous system, do we need to apply the elementary Operation to the “b” vector as well ? Why ?

Example 2

02.04.0 21 xx 02.04.0 21 xx

00

5.01

2.04.02.04.0

A

11

221

25 RR

RRR

variablefree 1r-n2;n1;rank(A)r002.04.0

2

numberrealaiswhere50;xby edparametriz solutions Infinite

ttxt numberreala is where5.0;x 12 ttxt

Application : Electrical Networks Analysis Kirchhoff’s Laws : (1) All currents flowing into a junction must flow out of it (2) Sum of (current x resistance) terms around a closed path equals total voltage

Application : Forces on a Trussnode 3

13F23F

3W

node 3

# Vector sum of forces at each node is equal to 0

# D fi f t b iti if th

1HF

node 1node 2

# Define forces to be positive if they- act to the right- act in an upward direction

1V 2V12F# acts from node i to node j

0i1

FVNode

20cos0sin

13121

131

NodeFFHFV

:FFFHVV

unknown

30cos0sin

2312

232

NodeFFFV 231312121 ,,,,, FFFHVV

180coscos

sinsin3

2313

32313

FFWFF

Node

Example : Forces on a Truss

00

0cos10100sin0001

1

1

HV

• Write in Ax=b form

000

cos01000sin00100

0cos1010

12

2

1

FVH

bAx

0coscos0000sinsin0000 3

23

13 WFF

• if :

002/10001 1V

100,3/,6/ 3 W

000

2/1010002/300100

02/31010

12

2

1

FVH

bAx

19

0100

0

2/12/300002/32/10000

2/101000

23

13

12

FFF

Example : Forces on a Truss• Let’s use MATLAB for two different values for W3

00

00

000

000

21 bandb

075

0100

0000

02/3101002/10001

2,11,1

2,11,1

HHVV

0000

2/1010002/300100

2,121,12

2,21,2

FFVV

bAx

20

0075100

2/12/300002/32/10000

2,231,23

2,131,13

FFFF

Example : Forces on a Trussx =% Applying Gaussian Eliminiation

function to Truss

sa = sin(pi/6); ca = cos(pi/6);

sb = sin(pi/3); cb = cos(pi/3);

x =

25.0000 18.750086.6025 64.951975.0000 56.2500sb = sin(pi/3); cb = cos(pi/3);

A = [ 1 0 0 0 sa 0

0 1 0 1 ca 0

0 0 1 0 0 sb

75.0000 56.2500-43.3013 -32.4760-50.0000 -37.5000-86.6025 -64.9519

0 0 1 0 0 sb

0 0 0 1 0 -cb

0 0 0 0 -sa -sb

0 0 0 0 -ca cb ];

>>

Q i h db = [ 0 0

0 0

0 0

0 0

RUNQuestion : what does a negative force value mean ?

100 75

0 0 ];

x = linsolve(A,b);

21display(x);

Matrix InversionDefinition : AA-1 = A-1A = I Unique MATLAB : inv(A)Definition : AA = A A = I Unique

proof (by contradiction): suppose both B and C are inverses of A

Consider : B(AC) = (BA)C BI = IC B = C

MATLAB : inv(A)

Consider : B(AC) (BA)C BI IC B C

1) Inverse only defined for square matrices nxn

2) Inverse exists iff elimination produces “n” pivots (rank = n)

3) Consider Ax = b, if A is invertible x = A-1b

Special Case : if b = 0 (homogenous) only zero solution4) Applying elimination to [A I] to reduce it to [T.A T.I]=[I A-1] gives inverse

5) Inverse of a

bd

bddba 1

1

Both forward and backward li i ti d d2x2 matrix

adbcb

adbcdababa

acbcaddc

00101f

elimination needed

adbca

adbcc

adbcadbc

cab

caddc 10

~10

~10

:Proof

Matrix Inverse (Cont’d)( )

6) Inverse of a diagonal matrix is diagonal : A = diag(di) A-1 = diag((di)-1)

(hint : start from fact that product of diagonal matrices is a diagonal matrix)

7) Inverse of lower (upper) triangular is lower (upper) triangular (why ?)

8) (AB)-1 = B-1A-1 prove it using definition of inverse !

9) (A-1)T = (AT)-1 prove it using definition of inverse !

10) AAAAp ....... (p times) matrix powerqpqp AAA pqqp AA )( TppT AA )()( AAA AA )( AA )()(

Example on Calculating Inverse

0121100 0 1 2 0 1

0103120 0 1 2 0 1

2212 RRR

1 0 4- 0 1 00 1 2- 1- 1- 0 ~

1 0 0 8 1 40 1 0 3 1- 2

3314 RRR

1-0401-02 2 11- 0 0 1

~012-1-1-00 0 1 2 0 1

232 RRR

1- 1- 6 1 0 0

1- 0 4 0 1- 0 ~1 1 6- 1- 0 00 1 2- 1- 1- 0

1132 RRR

1 0 4- 0 1 02 2 11- 0 0 1

~

(3 pivots Inverse exists) -R2 R2

1- 1- 6 1 0 0

A-1Always multiply AA-1 to check your answer!

Matrix Inversion Application : Decoding Digital Signals in Cell PhoneDigital Signals in Cell Phone

In cellular communications, digital data is transmitted in blocks. Each block received by your cell phone (after sampling and digitization) can be representedreceived by your cell phone (after sampling and digitization) can be representedmathematically in the form

HXY

Z

+where X is the transmitted blocks, H a matrix representing the effects of the multipath distortion of the channel (attenuation, reflections, scattering) on the p ( g)transmitted signal, and Z represents additive noise (e.g. due to cell phone analog front-end circuit imperfections, interference from other cell towers).

The transmitted signal can be decoded by the following matrix operation

Triangular/Cholesky/LU Factorizationg yMATLAB : [L,U] = lu(A)

Theorem: Suppose A is a non-singular (i.e. invertible with non-zero pivots) matrix that is reduced to upper-triangular form using Gaussian elimination, i.e.,

A ~ U TA=U A=inv(T).U then we can factor it as A = L U

where L is a lower triangular matrix with ones on the diagonal

U is upper triangular with the pivots on the main diagonal

Remark :

Elementary operations

(…..E32 E31 E21) A = U A = (E21-1 E31

-1 E32-1) U=LU

Elementary MatricesDefinition :

Let “e” be an elementary row operation. The elementary matrix E corresponding to “e” is constructed by applying “e” to thecorresponding to e is constructed by applying e to the

Identity matrix and is denoted E = e(I), where I is the identity matrix

Examples : 1 0 00 0 1

(i) R2 R3E = 0 0 1

0 1 0

E1 0 00 6 0

Remark : except for rows interchanges, all other elementary matrices are lower triangular

(ii) R2 -6R2

(iii)R3 -4R1 + R3

E = 0 -6 00 0 1

E =1 0 00 1 0( ) 3 1 3 E = 0 1 0-4 0 1

0150 0 1

0150 0 1 1

Remark : Inverse of elimination

1 0 0

0 1 51 0 0 0 1 5-Remark : Inverse of elimination

matrix is easy to compute !

LU Factorization (Cont’d)1) L contains the multipliers that take A to U in the elimination procedure

2) The elimination multipliers (with sign inversion) are below diagonal of L2) The elimination multipliers (with sign inversion) are below diagonal of L

3) If A is symmetric then U = LT (why ?)

4) All elementary matrices (except for interchange of 2 rows) are lower triangular and so are their inverses and products. Since we are considering invertible matrices (non-zero pivots), there is no need for row interchanges

Application : Solving System of Equations Using LU Factorization :

bLUxbAx cUx

bLUxbAx bLc

(1 System of Linear

(2 triangular systems)= Solve for c

firstEquations) first

Example on LU FactorizationExample on LU Factorization

321

ionfactorizatULfind

5121343A

700420321

~130420321

~A331

221

2

3

RRR

RRR

33223 RRR

700130

Exercise : write down the elementary matrices E1, E2, and E3

210321

020001

420321

;013001

UL

1007007001232

Always check by multiplying L U to get A

Application : Fitting Curvespp cat o tt g Cu es

equation The

011-n

1-nn

n

points m the through passes It curve. polynomial a defines yaxa...xaxa

1nn

mmm222111

: consistent is equations of system following the iff )y,(xP ..., , )y,(xP, )y,(xP :by given

20211-n

21-nn

2n

10111-n

11-nn

1n

yaxa...xaxa

yaxa...xaxa

m0m11-n

m1-nn

mn yaxa...xaxa

... ... ...

Application from : Fitting CurvesApplication from : Fitting Curves

:is 2))(nm size (of M matrix augmented The

y1xxxy1xxx

AM

:is 2))(nm size (of M matrix augmented The

221-n

2n2

111-n

1n1

hfhy1xxx

yy AM

mm1-n

mnm

2222

solution unique mrank(A) 1nm If ii)solutions infinite mrank(A) 1nm If i)

then j,i xx If :Theorem ji

solution unique 1nrank(A)solution no rank(M)rank(A)

1nm If iii)

solution unique mrank(A) 1nm If ii)

There are (n+1) unknowns and rank(A)=# of pivots

Fitting Curves ExampleFitting Curves Example

(1,2)P, (0,1)P 21

1n32nm distinct. values xparabola yaxaxap(x) 01

22

11

100011

~ 21

111100

M

t1avariable), (free ta

1,a

1

0

y 1txt)x(1 axaxap(x)

t1a

201

22

2

y 1txt)x(1

Application : Car Rentalspp cat o Ca e ta s

(D) Downtown (A), Airport :locations 3 has agency rentalcar A

is month each during cars of tionredistribu Thelocations. 3 the atcars 400 & 600,500 were there operation initial .At(S) Suburb &(D) Downtown (A), Airport :locations 3 has agency rentalcar A

:diagram following the by described

0.1A D

0.1

0.10.20.1 0.1

0.70.6

S

0.6

Application : Car RentalsApplication : Car Rentals

:tionInterpreta

been has 20% remaining TheS to returned & rented 10%another and D to returned & rented 10% A). to returned & rentedor

rented not(either A at still are A at cars of 60% end,-month At

service. from retired been has 20% remaining TheS. to returned & rented 10%another

500600

da

u & da

u

k. month each of beginning at cars of no. denote s &d,a Let

0

0

0k

k

k

kkk

400ss 0

00

k

kk

Application : Car RentalsApplication : Car Rentals

a0 20 10 6a

sda

0.60.10.10.10.70.10.20.10.6

sda

k

k

k

1k

1k

1k

A AA A k2

u

k

Au

1k

k1k

0 950.87

u e g

uAu uAAuu ,Auu 0k

k 02

1201

(why?) months 44after left cars no

0.690.95u e.g. 44

( y )

2 Equations in 2 Unknowns2 Equations in 2 Unknowns1212111

bXaXabXaXa

(Each

equation Solution Unique(i) Case2222121 bXaXa represents

a line)

22

12

21

11 SlopesDifferent

q( )

aa

aa

Solution21122211

S l tiN(ii)C

0 :Condition aaaa Solution

This is the determinant of matrix A !

2

1

22

12

21

11 :Condition

Solution No (ii) Case

bb

aa

aa

Parallel Lines

No Intersection

22221

coincide lines 2 Solutions ofNumber Infinite (iii) Caseinterceptsdifferent but Slope Same

2

1

22

12

21

11 :Condition bb

aa

aa

Exercise : Use Gaussian elimination to derive these 3 conditions

Review QuestionsReview Questions

1) If A is a diagonal matrix then An is also diagonal for any integer n (T)

True or False

1) If A is a diagonal matrix, then An is also diagonal for any integer n (T)

2) If A & B are invertible then AB is invertible. (T)

Answer the following short questionsAnswer the following short questions

1) Let A be a 4x4 matrix with , then the (2,3) element of A2 is equal to

2

|| jiaij

[1 0 1 2] = 2 + 2 = 42101

2) Let A & B be 4x4 matrices with trace(B)=2

Then trace(A 1BTA)

1

Then, trace(A-1BTA) = ……

Review Problem

32 3121

Gaussian elimination procedure applied directly to the augmented matrix

523452

32

zyxzyx

zyx

51-2-34- 1- 5 23 1 2 1

M523 zyx 5 1 2 3

103103 1 2 13313 RRR

103103 1 2 1

4- 4- 8- 010- 3- 1 0

2212 RRR

84- 28- 0 010- 3- 1 0

3328 RRR

110910338428

yyzyzz

233232 xxzyx