CE 601: Numerical MethodsLecture 5

Course Coordinator:

Dr. Suresh A. Kartha,

Associate Professor,

Department of Civil Engineering,

IIT Guwahati.

Elimination Methods

• For a system [A]{x} = {b} where [A] is an n X n matrix, the number of operations = (2/3)n3+(3/2)n2-(7/6)n.

• Note: The formula (2/3)n3+(3/2)n2-(7/6)n will always yield integer values.

• Like Gauss elimination, you might have studied Gauss-Jordan elimination method. The objective of Gauss-Jordan scheme was to convert [A]{x} = {b} => [I]{x} = {x}, [I] -> Identity matrix, through systematic elimination process.

• We are not going to discuss on this method in the class and request you to work on your our referring literature.

• The Gauss-Jordan scheme is computationally less efficient than the Gauss elimination scheme. The no. of operations involved in Gauss-Jordan scheme = n3+n2-n.

• Like Gauss elimination, Gauss-Jordan, etc. you might have studied matrix inverse methods to solve linear systems

i.e. [A]{x} = {b}

so, {x} = [A]-1{b}

There are two evaluations in this scheme

→ First evaluate the inverse of matrix [A]

→Second, to evaluate the product of [A]-1{b}

• In both the evaluations, there are arithmetic operations involved. For matrix inverse it takes 2n2-n operations i.e. total of 2n3 operations.

• Based on the significant digits assigned to the variables or components, round-off errors may creep in the solutions while using Gauss elimination scheme.

→ These errors can be minimized by performing partial pivoting or scaled partial pivoting.

• LU Decompositiono We have discussed that matrix can be factored i.e., it can

be given as product of two different matrix. [A] = [B][C]

o There can be many possibilities of obtaining factor matrices.→ However if we specify the diagonal elements of either

[L] or [U], then we will have a unique a factorization for [A].

→The LU decomposition methods like Doolittle and Croutwork on these principles.

o In a similar tone, one can also factorize [A] as product of [L] and [U] i.e., [A]= [L][U] where [L] is lower triangular and [U] is a upper triangular matrix.

11 12 1 11 11 12 1

21 22 2 21 22 22 2

1 2 1 2

0 0

0 0

0 0

n n

n n

n n nn n n nn nn

a a a l u u u

a a a l l u u

a a a l l l u

• Property of LU decomposition

o [A]{x} = {b}

[L][U] {x} = {b}

Multiply left and right side by [L]-1

[L]-1[L][U] {x} = [L]-1 {b}

[I][U]{x} = [L]-1 {b}

[U}{x} = [L]-1 {b}

o Let us define the system [U]{x} = {c}

So, we get {c} = [L]-1 {b}

or, [L]{c} = {b}

o The meaning here is:

Lower triangular matrix [L] will transform RHS vector {b} to {c} using the relation [L]-1{c} = {b}.

On obtaining {c} , using the relation [U]{x} = {c}, we obtain the solution vector.

So if we can suitable algorithm such that process involved in LU decomposition are”

Factorize [A] = [L][U]

Forward substitution to evaluate {c}

Backward substitution to evaluate {x]

If the no. of arithmetic operations in factorization can be reduced then LU decomposition becomes efficient.

• The Advantage of LU Decomposition

o If there are many linear systems with same coefficient matrix i.e.,

[A]{x} = {b}

[A]{y} = {c}

[A]{z} = {d}

o Then if we adopt Gauss elimination, for each system it may take (2/3)n3+(3/2)n2-(7/6)n operations. The purpose may become tedious.

o If we do LU decomposition, then you may see that only once we need to decompose [A]=[L][U]

o After that for each system, we need to utilize only the forward and backward substitutions.

• Doolittle LU Decomposition

• [A] = [L][U]

• If we factorize in such a way that

i.e. diagonal elements of [L] are 1, then the approach is Doolittle’s method.

• If

the approach is called Crout’s method.

11 12 1

21 22 2

1 2

0 0 1

0 0 1[ ]

0 0 1

n

n

n n nn

l u u

l l uA

l l l

11 12 1 11 12 1

21 22 2 21 22 2

1 2 1 2

1 0 0

1 0 0

1 0 0

n n

n n

n n nn n n nn

a a a u u u

a a a l u u

a a a l l u

Algorithm for Doolittle’s LU Decomposition

• Doolittle algorithm is developed using knowledge of Gauss elimination.

• Recall Gauss elimination at any step k

• You can now write

( ) ( 1) ( 1)

( 1) ( 1)1,2,3,..., 1; , 1,..., ; 1, 2,...,

k k k

ij ij ik kj

k k

ik ik kk

a a l ak n j k k n i k k n

l a a

( ) ( 1) ( 1)

( ) ( 2) ( 1) ( 2)

( 1) ( 1)

( ) ( 1)

1

( ) ( 1)

1

i.e.,

Extending,

or, ; 1, 2,..., ; , 1,...,

k k k

ij ij ik kj

k k k k

ij ij ik kj i k k j

kk m

ij ij im mj

m

kk m

ij ij im mj

m

a a l a

a a l a l a

a a l a

a a l a i k k n j k k n

• This is nothing but

• So you get

( ) ( 1)

1

[ ] [ ][ ]

where elements of lower triangular matrix

; for ; 1,2,3,..., 1

such that 1 ; for

0 ; for

kk m

ij ij im mj

m

ij

ik

ij

A L U

a a l a

l

l i k k n

l i j

i j

11 12 13 1

(1) (1) (1)

21 22 23 2

(2) (2)

31 32 33 3

( 1)

1 2 3

(1) (1)

21 21 11 32 32 22

1 0 0 0

1 0 0 0

[ ] 1 0 ,[ ] 0 0

1 0 0 0

where , , etc.

n

n

n

n

n n n nn

a a a a

l a a a

L l l U a a

l l l a

l a a l a a

• So the steps involved are:

o No. of steps as in Gauss elimination

k = 1,2,3,…,n-1

o At any k , i= k+1,k+2,…,n and j=k,k+1,…,n

( ) ( 1) ( 1)

( 1) ( 1)

1

0;

0

;

kk

ik

k k k

ij ij ij ik kj

ij

k k

ik ik kk

l

l i k

u a a l a

u

l a a i k

o Forward substitution for c

o Back substitution for x

1 1

1

1

; 2,3, ,k

i i im m

m

c b

c b l c i n

1 ( 1)

1

( 1)( 1)

1In genral, ; ( 1),( 2),...,2,1

n n nn

n n n n

n

n n

n

i ij j

j i

i

ii

x c u

c u xx

u

c u x

x i n nu

• Example1

2

3

11 12 13

21 22 23

31 32 33

21 31 32

11 12 13

21 22

80 20 20 20

20 40 20 20

20 20 130 20

1 0 0

1 0 , 0

1 0 0

1 / 4, 1 / 4, 5 / 7

80, 20, 20

0, 40 ( 1 / 4)

x

x

x

A L U

u u u

L l U u u

l l u

l l l

u u u

u u

23

31 32

2( 1)

33 3

1

( 20) 35

20 ( 1 / 4)( 20) 25

0

130

750 / 7

km

m mj

m

u

u u

u l a

Forward Substitution:

Now you can do back substitution easily.

1 1

2

3

1

2

3

20

20 ( 20 / 4) 25

20 ( 20 / 4) ( 5 25 / 7) 300 / 7

[ ]{ } { }

80 20 20 20

i.e., 0 35 25 25

0 0 750 / 7 300 / 7

c b

c

c

U x c

x

x

x

• Number of operations

• In Doolittle’s algorithm,

o to evaluate lik for any kth step for i> k, it takes (n-k) operations.

o To evaluate uij = aij(k) it takes 2(n-k)2

operations.

o In (n-k) elimination steps to form [L] and [u], it takes, 1

1

23

( )(2 2 1)

2( 1)(4 1)

6 3 2 6

n

p

n p n p

n n nn n n

o To perform forward substitution,

as c1 = b1 (No operation required)

There are 2i operations for each i.

So, no. of operations =

o To perform backward substitution, it requires n2 operations (as in Gauss)

o Total no. of operations =

1

1

; 2,3,4, ,i

i i im m

m

c b l c i n

2

2

2n

i

i n n

23 2

3 2

22

3 2 6

2 3 7

3 2 6

n nn n n

n n n

• No. of operations is same as Gauss elimination method.

• However if there are many systems involving [A], then you may need to just add the no. of operations for forward & backward substitution for each system.