iterative methods for systems of linear equations

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 1/17

Iterative methodsfor systems of linear

equationsAstrid Xiomara

Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin

eales_iterativos

Iterative methodsfor systems of linear

equationsAstrid Xiomara

Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin

eales_iterativos

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 2/17

Iterative methods for systems

of linear equations

Iterative methods for systems

of linear equations

Introduction

Heat EquationJacobi Method

Gauss-Seidel Method

Overrelaxation methodCapacitor Problem

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 3/17

Direct methods versus

iterative methods

Direct methods versus

iterative methodsDIRECT

Ax =b

x = A\ b

Moderate size

Alter the structure

Rounding error

ITERATIVE

x = Cx + d

x(k+1) = Cx(k) + d

Big size

Zeros preservedTruncation error

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 4/17

Iterative methods compared with direct, we donot guarantee a better approach, however, aremore efficient when working with large matrices

In the resolution for numerical partial differentialequations often appear linear equation systemswith even 100 000 unknowns, in these systemsthe coefficient matrix is sparse, ie a highpercentage of matrix elements are equal to 0. If 

there are any patterns in the nonzero elements(eg tridiagonal systems), then an iterativemethod can be very effective.

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 5/17

Heat Equation

Heat Equation

System Eq. linear. Associated matrix

T (T T ) / 2T (T T ) / 2

T (T T ) / 2

T (T T ) / 2

1 0 2

2 1 3

3 2 4

n n-1 n+1

!

!

!

!

/

2 -

- 2 -

- 2

-

- 2

1

1 1

¨

ª

©©©©

©©

 ¸

 º

¹¹¹¹

¹¹

T0 T1 T2 . . . Tn Tn+1

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 6/17

J acobi methodJ acobi method System of linear equations

a x

a x

a x

a x

a x  a x  a x

a x a x  a x

a x  a x  a x

a x  a x  a x

b

b

b

b

11 1

21 1

31 1

n1 n

12 2 13 3 1n n

22 2 23 3 2n n

32 2 33 3 3n n

n2 2 n3 3 nn n

1

2

3

n

!

!

!

!

¾

¿

±±±

À

±±±

/

.

.

.

/ / /

.

/

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 7/17

Of all the iterative methods, Jacobi's is theeasiest to implement and understand, however,

is not very efficient in terms of obtainingsolutions.Consider the system:

we can write equivalently as:

Jacobi's method is to use the above formulas asfixed point iteration.

±°

±¯

®

!

!

!

1552

2184

74

 z  y x

 z  y x

 z  y x

±±±

°

±±±

¯

®

!

!

!

5

2158

421

4

7

 y x z 

 z  x y

 z  y x

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 8/17

Fixed point equationFixed point equation

x (bx (b

x (b

x (b

a x  a x  a x ) / aa x a x  a x ) / a

a x  a x  a x ) / a

a x  a x  a x ) / a

1 1

1 2

2 3

n n

12 2 13 3 1n n 11

21 1 23 3 2n n 22

31 1 32 2 3n n 33

n1 1 n2 2 n,n 1 n 1 nn

! !

!

!

¾

¿

±±±

À

±±±

/

.

.

.

/ / 1 /

.

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 9/17

J acobi IterationJ acobi Iterationx (b

x (b

x (b

x (b

a x a x a x ) /a

a x a x a x ) /a

a x a x a x ) /a

a x a x a x )/a

1

(k+1)

1

2

(k+1)

2

3

(k+1)

3

n

(k+1)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k)

23 3

(k)

2n n

(k)

22

31 1

(k)

32 2

(k)

3n n

(k)

33

n1 1

(k)

n2 2

(k)

n,n 1 n 1

(k)

nn

!

!

!

!

¾

¿

À

/

.

.

.

/ / 1 /

.

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 10/17

At each step of the Jacobi iteration yields a vectorwith n coordinates

P0 = (x1 (0), x2 (0), ..., xn (0)), ..., Pk = (x1 (k), x2(k), ..., xn (k))

where the initial estimate (x1 (0), x2 (0), ..., xn (0))should be chosen. When you do not have a clue aboutthe solution is usually taken xi (0) = bi / aii

In the example above, if we take P0 = (x (0), y (0), z(0)) = (1,2,2)In the first iteration is obtained

P1 = (1.75, 3375, 3.00)

Generating the sequence of iterations of Jacobi notesthat converges to (2, 4, 3).

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 11/17

Many times the Jacobi iteration method does notwork. Here is an example of rearranging theequations above example.Example:

Now the iteration formula is

and notes that the sequence of Jacobi diverges.Note that the system matrix is not strictlydiagonal dominant.

±

°

±¯

!

!

!

74

2184

1552

 z  y x

 z  y x

 z  y x

±±

±

°

±±±

¯

¡ 

!

!

!

 y x z 

 z  y y

 z  y x

47

8

421

2

515

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 12/17

G auss-Seidel iterationG auss-Seidel iteration

x (b

x (b

x (b

x (b

a x a x a x )/a

a x a x a x )/a

a x a x a x )/a

a x a x a x )/a

1

(k+1)

1

2

(k+1)

2

3

(k+1)

3

n

(k+1)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k+1)

23 3

(k)

2n n

(k)

22

31 1

(k+1)

32 2

(k+1)

3n n

(k)

33

n1 1

(k+1)

n2 2

(k+1)

n,n 1 n 1

(k+1)

nn

!

!

!

!

¾

¿

±

/

.

.

.

/ / /

.

±±

À

±±±

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 13/17

The Gauss-Seidel method is a modification of theJacobi method which accelerates the convergence of the latter.Note that the Jacobi method generates a sequence foreach unknown(X1 (k)), ..., (xn (k)). Since xi (k +1) is probablybetter approximated by xi (k) instead of xi (k) in thecalculation of xi +1 (k +1) we use xi (k +1).Apply this strategy to the example 1 and comprubesethe speed of convergence.

The Gauss-Seidel substantially cut the number of iterations to make for some precision in thesolution. Obviously the convergence criteria aresimilar to those of Jacobi.

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 14/17

Overrelaxation method

Overrelaxation method

1)+(k

i

(k)

i

1)+(k

i

(k)

i

1)+(k

ii

i

(k)

i

1)+(k

i

i(k)i

1)+(ki

xww)x(1x

xxz

2<w<0 ;wzxx

:tionIverrelaxa

zxx

:SeidelGauss

Ö

Ö

Ö

!

!

!

!

xik

zi xik+1

ik+1x

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 15/17

Overrelaxation method reducesthe number of iterations in thecalculations of solutions of linear

systems by Gauss-Seidel. It isbased on each iteration to obtain aweighted average (only for vectorelements before the position

calculation) for the solution of theJacobi method and the solution of the Gauss-Seidel.

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 16/17

P ass

Overrelaxation

P ass

Overrelaxation

x (1 )x (b

x (1 )x (b

x (1 )x (b

x (1 )x (b

a x a x a x )/a

a x a x a x ) /a

a x a x a x ) /a

1

(k+1)

1

(k)

1

2

(k+1)

2

(k)

2

3

(k+1)

3

(k)

3

n

(k+1)

n

(k)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k+1)

23 3

(k)

2n n

(k)

22

31 1

(k+1)

32 2

(k+1)

3n n

(k)

33

!

!

!

!

[ [

[ [

[ [

[ [

/

.

.

.

/ / 1 /

.a x a x a x )/an1 1

(k+1)

n2 2

(k+1)

n,n 1 n 1

(k+1)

nn

¾

¿

±±±±

À

±

±±±

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 17/17

SummarySummary Iterative methods are applied to large

and sparse matrices.

The cost per iteration is O (n2) or less if you take advantage of the dispersityAre expected to converge in less than nsteps.

The matrix has to fulfill certainconditions for the method to converge.