systems of linear equations iterative methods 1daniel baur / numerical methods for chemical...

1Daniel Baur / Numerical Methods for Chemical Engineers / Iterative Methods

Systems of Linear EquationsIterative Methods

x bA

Daniel Baur

ETH Zurich, Institut für Chemie- und Bioingenieurwissenschaften

ETH Hönggerberg / HCI F128 – Zürich

E-Mail: [email protected]

http://www.morbidelli-group.ethz.ch/education/index


Iterative Methods

The main idea behind iterative methods is that we can reformulate the problem in the following way

Now we can proceed iteratively

This is advantageous if S has a structure that makes it particularly easy to factorize or solve (e.g. diagonal, tridiagonal, triangular) and if S can be considered constant

Even if b or A (and S) change slightly, the solution will be similar and thus the iteration will converge quickly if the previous solution is used as a starting point

( )

( )

x x b

x b x

A S A S

S S A

( 1) ( )( )k kx b x S S A


Convergence of Iterative Methods

Let us rewrite the convergence loop as

As we have seen with the convergence loop of a predictor corrector ODE-solver scheme, such a loop will only converge if the spectral radius of the iteration matrix is smaller than 1, i.e.

This is the case if S and A are similar in some sense:

1( 1) ( )

1

k kx x cc b

M S S AM

S

1max 1 M I S A

1 1 1 S A S A I


Solution Procedure

If we look at the iteration equation

we can formulate it in a more convenient way by treating the right hand side as a constant vector

Since we can choose S to have a structure that is easy to solve, there is no need to calculate S-1 or do Gauss elimination to solve this equation system; We can use more direct (even analytical) approaches instead

( 1) ( )( )k kx b x S S A

( 1)kx c S


Jacobi’s Method

The Jacobi method chooses S to be a diagonal matrix

The procedure is therefore given by

This method is guaranteed to converge if A is diagonally dominant, i.e.

( 1) ( )( )

( )

k kx b x

diag

D D A

D A

(0)

,

( 1) ( ),

, ,

1

ii

i i

k kii i j j

j ii i i i

bx

a

bx a x

a a

, ,i i i jj i

a a

1,1 1,2 1,3 1,

2,1 2,2 2,3 2,

1,1 1, 2 1, 1 1,

,1 , 2 , 1 ,

N

N

N N N N N N N

N N N N N N N

a a a a

a a a a

a a a a

a a a a

A

D


The Gauss-Seidel Method

In this case, S is chosen to be lower triangular

The procedure is therefore given by

This method is guaranteed to converge if A is symmetric positive-definite or diagonally dominant

( 1) ( )( )

( )

k kx b x

tril

L L A

L A

1,1 1,2 1,3 1,

2,1 2,2 2,3 2,

1,1 1, 2 1, 1 1,

,1 , 2 , 1 ,

N

N

N N N N N N N

N N N N N N N

a a a a

a a a a

a a a a

a a a a

A

L

( 1) ( ) ( 1), ,

,

1k k ki i i j j i j j

j i j ii i

x b a x a xa


Another way of Implementation

At every iteration, a linear system of equations is solved:

This is equivalent to

It is therefore possible to compute M and c and once in the beginning and then do simple matrix multiplications and vector additions for the iteration

( 1) ( )( )k kx b x S S A

1

1

( 1) ( )k k

c b

x x c

M S S A

S

M


The Conjugate Gradient Method

We say that two non-zero vectors are conjugate (wrt A) if

If A is symmetric and positive definite, this corresponds to an inner product of u and v

If we now find a series of N mutually conjugate vectors p, the solution of the linear system Ax = b must be contained in the space that these vectors span, i.e.

With the coefficients

0Tu v A

1

N

i ii

x p

1

TNk

i i k Ti k k

p bb p

p p

AA


The Conjugate Gradient Method (Continued)

If we find the vectors p one after the other, we can proceed iteratively

The main idea for the transformation into an iterative method is that the unique minimizer of the cost function

is the same as the solution of the equation Ax = b The negative gradient of f in point xk, which denotes the

steepest descent direction, reads

However to ensure that our next step be conjugate to the previous one, we will have to correct this search direction

1( )

2T Tf x x x x b A

k kr b x A


Stability and Convergence of the CG-Method

The CG-Method is only stable if A is symmetric and positive definite; However, if we have an asymmetric matrix, we can use the equivalent normal equations

where C = ATA is symmetric positive definite for any non-singular matrix A

The convergence speed of the CG-Method is determined by the condition number of A; The larger it is, the slower the method converges and unfortunately

T Tx b

x z

A A A

C

2T A A A


Preconditioned Conjugate Gradient

We can improve the convergence speed by preconditioning the equation system, i.e. replace the system by an equivalent system

so that κ(M-1A) < κ(A) Some choices of M are

Jacobi-preconditioning where M = D; D is the diagonal matrix of A Incomplete Cholesky factorization where M = KKT and KKT ≈ A SSOR-preconditioning where for ω = (0,...,2)

where L is the strictly lower triangular matrix of A

1 1x b x b A M A M

11 1 1 1

2

T

M D L D D L


Algorithm for the PCG-Method1. Choose a starting point x0 and calculate

2. Proceed from xk to xk+1 using pk as the direction

3. Correct the search direction

0 0 0 0; \ ;r b x z r A M

1

1

Tk k

k Tk k

k k k k

k k k k

r z

p p

x x p

r r p

A

A

1 1

1 1

1 1

\k k

Tk k

k Tk k

k k k k

z r

r z

r z

p z p

M

Iterate 2 and 3 until norm(rk+1) or norm(Axk – b)

is sufficiently small, or 5*length(A) steps have been taken


Exercise: 2-D Heat Transport

Consider a spherical catalyst particle where a chemical reaction takes place

Assumptions: The reaction rate is independent of concentration and temperature Thermal diffusivity is independent on temperature No convective heat transport Perfect heat sink at the boundary, i.e. T = 0

λ


Exercise (Continued)

The heat transfer can be described as a PDE:

where is the Laplace operator, k is the heat produced by the reaction and r is the particle radius

2 22

2 2

2 2 2 , 0

, , 0 0

Q Q QQ k

t x y

Q x y r t

Q x y t

2


Exercise (Continued) Substituting the temperature:

At steady state we get

This equation can be solved by using a discretized Laplace operator:

2

P

P

Q c T

T kT

t c

2

P

kT

c

P

kT

c D


Assignment

1. Implement the PCG-algorithm as a funciton (see slide 12)

2. Use it to solve the discrete steady state Laplace equation for the catalyst particle, discretizing with N = 50 points

Assume

Create a disk shaped grid using: G = numgrid('D',N); Check the grid with spy(G) Use D = delsq(G); to create a negative 5th-order discrete

Laplace operator, take a look at the operator again with spy(D) Create the right hand side using b = ones(size(D,1),1); Choose an initial guess, e.g. x0 = zeros(size(D,1),1);

3. Solve the system without preconditioning Set M = eye(size(D));

1P

k

c


Assignment (continued)

4. Plot the solution using U = G;U(G>0) = full(x(G(G>0)));clabel(contour(U))

5. Try out different preconditioning methods: Jacobi preconditioning, i.e. M = diag(diag(D)); SSOR preconditioning with ω = 1.5; Use L = tril(D,-1); to

get the strictly lower triagonal part and E = diag(diag(D)); to get the diagonal matrix of D

Incomplete cholesky factorization; This can be done usingM = C*C';withC = ichol(D); ORC = cholinc(D,'0');

How many iterations are needed in all cases?

systems of linear equations iterative methods 1daniel baur / numerical methods for chemical...

Documents