lecture 11

Computational Fluid Dynamics IPPPPIIIIWWWW

Numerical Methods forElliptic Equations

Instructor: Hong G. ImUniversity of Michigan

Fall 2001


Direct Methods• Cramer’s rule• Gaussian elimination• Tridiagonal matrix algorithm (Thomas algorithm)• Block tridiagonal matrix algorithm

Iterative Methods• Jacobi, Gauss-Seidel iteration• Successive overrelaxation (SOR)• Multigrid method

Outline

Solution Methods for Elliptic Equations


SfSy

fx

f =∇=∂∂+

∂∂ 2

2

2

2

2

;

Poisson equation (Laplace equation if S = 0).

A model equation for elliptic problems

On the boundaries (BC)

),(

),(

0

0

yxgnf

yxff

=∂∂= Dirichlet

Neumann


ωψ −=∇ 2

Examples of Elliptic Equations

2-D stream function equation

Projection method(Step 2)

tjihjih t

P ,,2 1 u⋅∇

∆=∇

kqT&

−=∇ 2 Steady conduction equation


Sy

fxf =

∂∂+

∂∂

2

2

2

2

Applying central differencing

Solving the Poisson equation

for which the modified equation becomes

jijijijijijiji S

yfff

xfff

,21,,1,

2,1,,1 22

=∆

+−+

∆+− −+−+

[ ] L+∆+∆−=+ 22

121 yfxfSff yyyyxxxxyyxx

(x, y)

i−1 i i+1

j+1j

j−1

Computational Fluid Dynamics IPPPPIIIIWWWWIf ⇒=∆=∆ hyx jijijijijiji Shfffff ,

2,1,1,,1,1 4 =−+++ −+−+

=

−

−−

−

−−

JI

JI

J

JI

JI

J

SS

SSS

SS

h

ff

fff

ff

,

1,

2,2

1,2

,1

2,1

1,1

2

,

1,

2,2

1,2

,1

2,1

1,1

410

1001001

101410100141010014

M

M

M

M

M

M

M

M

LLLL

LLLL

LLLL

Sparse matrix: only 5 non-zero entries in each row

Computational Fluid Dynamics IPPPPIIIIWWWWUltimately, the difference form of the Poisson equationboils down to solving for

[ ] bf =AHence,

[ ] bf 1−= A

Direct method:- Solving inverse matrix directly (Cramer’s rule)- Inverting matrix more cleverly (Gaussian elimination)- Other (L-U decomposition, Thomas algorithm)

Computational Fluid Dynamics IPPPPIIIIWWWWCramer’s Rule

- Check elementary linear algebra book- Extremely time consuming for large matrices

Operation count:For example, 11×11 matrix:

Using 1 Gigaflops (109 flops) computer

Direct Method - 1

)!1( +n

221075.4)!199( ×=+×

!!!!!! years1020.3 100×

Computational Fluid Dynamics IPPPPIIIIWWWWGaussian Elimination

- Pivoting: rearranging equations to put the largestcoefficient on the main diagonal.

- Eliminate the column below main diagonal.- Repeat until the last equation is reached.- Back-substitution

- Special case: tri-diagonal matrix - Thomas algorithm

Direct Method - 2

nnn cfafa

cfafacfafa

=++

=++=++

LL

MM

LL

LL

2211

2222121

1212111

nnnn cfa

cfacfafa

=′

=+′=++

M

LL

LL

2222

1212111

⇒

Computational Fluid Dynamics IPPPPIIIIWWWWDiscretized Poisson Equation

Iterative Method

jijijijijiji Shfffff ,2

,1,1,,1,1 4 =−+++ −+−+

Rearranging for jif ,

[ ]jijijijijiji Shfffff ,2

1,1,,1,1, 41 −+++= −+−+

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,1,,1,1

1, 4

1 −+++= −+−++ Jacobi

[ ]jinji

nji

nji

nji

nji Shfffff ,

211,1,

1,1,1

1, 4

1 −+++= +−+

+−+

+ Gauss-Seidel

SOR


,1,1,,1,1 4 =−+++ −+−+

[ ] ( ) njiji

nji

nji

nji

nji

nji fShfffff ,,

211,1,

1,1,1

1, 1

4ββ −+−+++= +

−++

−++

Computational Fluid Dynamics IPPPPIIIIWWWWA One Dimensional Example

Convergence – 1-D Problem (1)

)(xFx =

for which convergence is achieved if

An equation in the form

can be solved by iterative procedure:

)(1 nn xFx =+

nn xx ≈+1 or ε<−+

11

n

n

xx


The above figure suggests that

Convergence – 1-D Problem (2)

1≤dxdF for convergence

nx

1+nx

1<dxdF

1>dxdF

)(xFx =

Computational Fluid Dynamics IPPPPIIIIWWWWFor multi-dimensional problems

Convergence – Multi Dimensional (1)

fIt can be shown that the space of can be “spanned”by eigenvectors

[ ] nn M ff =+1

[ ] NMiM iii ×== L,1,qq λsuch that

NMNMyyy ××+++= qqqf L2211

Hence, [ ] [ ] ( )NMNMyyyMM ××+++= qqqf L2211

[ ] [ ] L++= 2211 qq MyMy

L++= 222111 qq λλ yy

Computational Fluid Dynamics IPPPPIIIIWWWWTherefore, the problem


is equivalent to

[ ] nn M ff =+1

or

LL ++=++ ++2221112

121

11 qqqq λλ nnnn yyyy

M

nn

nn

yyyy

221

2

111

1

λλ

==

+

+

yielding a convergence condition

1max ≤λ

Computational Fluid Dynamics IPPPPIIIIWWWWGeneral iterative procedure


[ ] [ ] [ ]21 AAA −=Let

[ ] bf =A

[ ] [ ] bff += 21 AA

An iterative scheme is constructed as

[ ] [ ] bff +=+ nn AA 21

1

For example,

[ ] [ ] [ ] [ ] [ ] [ ] [ ]DABAIDA −==−== 121 ,41 Jacobi

[ ] [ ] [ ] [ ] [ ]UALDA =−= 21 , Gauss-Seidel

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Multi Dimensional (4)

Requirements:

1. should be invertible.2. Iteration should converge, i.e.

[ ] [ ] bff +=+ nn AA 21

1

Define error at n-th iteration

ff =∞→

n

nlim

nn ffε −=

[ ]1A

[ ] [ ] nn AA εε 21

1 =+ [ ] [ ] nn AA εε 21

11 −+ =

[ ] [ ]( ) 02

11 εε

nn AA −=

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Multi Dimensional (5)

Condition for convergence

which requires

This is achieved if the modulus of the eigenvalues are less than unity.

Therefore, the convergence condition becomes:

0lim =∞→

n

nε

1max

≤= iλρ

[ ] [ ]( ) 0lim 21

1 =−

∞→

n

nAA

iλρ Spectral radius of convergence

Eigenvalues of matrix [ ] [ ]211 AA −

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Jacobi Method (1)

Jacobi Iteration Method for Poisson Equation(2nd order central difference with uniform mesh)

1,,1,1,,1,coscos21 −=−=

+= NnMm

Nn

Mm

mn LLππλ

and using a discrete analog of separation of variables, it can be shown that the eigenvalues of are

Therefore, and the Jacobi method converges.

[ ] [ ]IA =−11

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,1,,1,1

1, 4

1 −+++= −+−++

[ ]2A

1≤mnλ

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Jacobi Method (2)

For a large matrix

L+

+−≅

+=

2

2

2

2

max,

411

coscos21

NM

NMmn

ππ

ππλ largest eigenvalues for

Thus, for a large matrix, is only slightly lessthan unity.

⇒ Very slow convergence

1,1 == nm

max,mnλ

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Gauss-Seidel (1)

Gauss-Seidel Iteration Method for Poisson Equation(2nd order central difference with uniform mesh)

[ ]jinji

nji

nji

nji

nji Shfffff ,

211,1,

1,1,1

1, 4

1 −+++= +−+

+−+

+

[ ] [ ] [ ] [ ] [ ]UALDA =−= 21 ,

−

=

−

−−

−

+

+

+

NMn

NM

n

n

nJI

n

n

S

SS

h

f

ff

f

ff

,

2,1

1,1

2

,

2,1

1,1

1,

12,1

11,1

000000

010000010000010

4100

04100004100004100004

M

M

M

M

M

M

L

MMM

MMMMM

L

L

L

M

M

M

LL

MLMMM

L

L

L

L

A1 A2

Computational Fluid Dynamics IPPPPIIIIWWWWConvergence – Gauss-Seidel (2)

It can be shown that the eigenvalues of matrixAre simply square of the eigenvalues of Jacobi method

[ ] [ ]211 AA −

2

max, coscos41

+=

NMmnππλ

Thus, Gauss-Seidel method is twice as fast as the Jacobi method.

Computational Fluid Dynamics IPPPPIIIIWWWWSuccessive Overrelaxation - 1

Successive Overrelaxation

[ ] [ ]( ) [ ] bff +=− + nn ULD 1

Consider the Gauss-Seidel method

If Gauss-Seidel is an attempt to change the solution as

dff +=+ nn 1

Can we accelerate the change by a parameter?

1,1 >+=+ ββdff nn


Hence, SOR first uses Gauss-Seidel to compute intermediate solution,

[ ] [ ]( ) [ ] bff +=− nULD ~or

( )nnn ffff −+=+ ~1 β

Then accelerate the next iteration solution

( ) nff ββ −+= 1~

f~

[ ] [ ] [ ] bfff ++= nULD ~~

[ ] [ ]( ) ( ) nnn ULc

fbfff ββ −+++=+ 1~1

[ ] [ ]IcD =

Combining the two steps

Note that [ ] [ ] 1~ += nLL ff


Example: Poisson Equation (2nd order CS, uniform mesh)

[ ] [ ] [ ] bfff ++= nULD ~~

[ ] [ ]( ) ( ) nnn ULc

fbfff ββ −+++=+ 1~1


,1,1,,1,1 4 =−+++ −+−+

[ ] ( ) njiji

nji

nji

nji

nji

nji fShfffff ,,

21,,1

11,

1,1

1, 1

4ββ −+−+++= ++

+−

+−

+

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,,1

11,

1,1

1, 4

1 −+++= +++−

+−

+

G-S

Combining


Convergence of SOR

[ ] [ ] [ ] bfff ++= nULD ~~

[ ] [ ] [ ]( ) ( )[ ] [ ] [ ]{ }[ ] [ ] [ ]( ) [ ] b

ff111

1111 1−−−

−−−+

−+

+−−=

DLDI

UDILDI nn

ββ

βββ

From

Eliminating and solving for

( )nnn ffff −+=+ ~1 β1+nff~

[ ]SORM

Convergence depends on the eigenvalues of [ ]SORM


For the discretized Poisson operator, it can be shown that

where is an eigenvalue of the Jacobi matrix

Note that if (Gauss-Seidel)

( )[ ]1421 222/1 −−+= ββλλβµ

λ[ ] [ ] [ ] [ ]( )ULDM J += −1

2λµ = 1=β

Minimum occurs atµ

2max

opt112λ

β−+

=Typically

9.1~7.1opt ≈β1max ≈λ


For problems with irregular geometry and non-uniform mesh, must be found by trial and error.

Typical Comparison Chart

optβ

290.7906SOR

6250.9890Gauss-Seidel

12500.9945Jacobi

Iterations( )maxmax µλ

Ferziger, J. H., Numerical Method for Engineering Application (1981)


[ 1,,11,,1, 4 ++−− +++= jijijijiji fffff β

] ( ) jiji fSh ,,2 1 β−+−

Coloring Scheme (Red & Black)

In large computer application (vector or parallel platform),SOR faces difficulties in using constantly updated values. Remedy: Two separate grid system (red & black)

[ 1,,11,,1, 4 ++−− +++= jijijijiji fffff βdo i =1, nx, 2

do i=2, nx, 2

] ( ) jiji fSh ,,2 1 β−+−

enddo

enddo

Computational Fluid Dynamics IPPPPIIIIWWWWSuccessive Line Overrelaxation (SLOR) - 1

Line Relaxation Method (Line Gauss-Seidel Method)

Adding one more coupling (E)

⇒ Thomas algorithm

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,,1

11,

1,1

1, 4

1 −+++= +++−

+−

+

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,

1,1

11,

1,1

1, 4

1 −+++= ++

++−

+−

+

[ ]jinji

nji

nji

nji

nji Shfffff ,

21,

11,

1,1

1,

1,1 4

141

41 −+=−+− +

+−

++

++−

New

New New

Old

Old

New

New New

Old

New


SLOR = Line Relaxation + Overrelaxation

Apply line relaxation for intermediate solution

and then overrelax

which is no more complicated than line relaxation.

( ) njiji

nji fff ,,

1, 1~ ββ −+=+

[ ]jinji

njijijiji Shfffff ,

21,

11,,1,,1 4

1~41~~

41 −+=−+− +

+−+−


Notes on SLOR

- Exact eigenvalues are unknown.- To ensure convergence, - Converges approximately twice as fast as Gauss-Seidel.- May be faster than pointwise SOR, but each iterationtakes longer with Thomas algorithm.

- Improved convergence is due to the direct effect of theboundary condition in each row.

2≤β

Computational Fluid Dynamics IPPPPIIIIWWWWAlternating-Direction Implicit - 1

ADI for elliptic equation is analogous to ADI in parabolicequation

Sy

fx

ftf −

∂∂+

∂∂=

∂∂

2

2

2

2

α

[ ]Sfftff yyxxnji

nji −+∆=−+ δδα,

1,

In discrete form

and take it to the limit to obtain the steady solution.

=∂∂ 0

tf

( ) ( ) ( ) ( )[ ]jijijixx x ,1,,12 21+− +−

∆=δ

( ) ( ) ( ) ( )[ ]1,,1,2 21+− +−

∆= jijijiyy y

δ


ADI for [ ] Sfftff yyxxnji

nji −+∆=−+ δδα,

1,

is written as

( ) ( )[ ]jinji

nji

nji

nji

nji

nji

nn Sffffffh

tff ,1,,1,2/1

,12/1

,2/1

,122/1 22

2−+−++−∆=− −+

+−

+++

+ α

( ) ( )[ ]jinji

nji

nji

nji

nji

nji

nn Sffffffh

tff ,11,

1,

11,

2/1,1

2/1,

2/1,12

2/11 222

−+−++−∆=− +−

+++

+−

+++

++ α

or( ) ( ) ji

nyyn

nxxn S

htff ,2

2/1

211 ∆−+=− + αδρδρ

( ) ( ) jin

xxnn

yyn Sh

tff ,22/11

211 ∆−+=− ++ αδρδρ

∆=

2n

ntαρ


Convergence of ADI

- Iteration parameter usually varies with iteration- For example, Wachspress

lower bound eigenvalue

2n

nt∆= αρ

( ) ( )nk

bab

ht

h

nkkk ,,1,

2

1/1

22 L=

=∆=

−−αρ

ab upper bound eigenvalue

- Comparison with SOR is difficult- ADI can be efficient if appropriate parameters are found.


1,if=

Boundary Conditions for Iterative Method

Dirichlet conditions are easily implemented.For Neumann condition, the simplest approach is

00 1,0, =−⇒=∂∂

ii ffnf

(1st order)

Update interior points and then setL,,, 3,2,1, iii fff 1,0, ii ff =

⇒ This generally does not converge.

Instead, incorporate BC directly into the equations

[ ]jiiiiii Shfffff ,2

0,2,1,11,11, 41 −+++= +−

[ ]jiiiii Shffff ,2

2,1,11,11, 31 −++= +−

Computational Fluid Dynamics IPPPPIIIIWWWWMultigrid Method - 1

Basic Idea

Suppose we want to solve a 1-D elliptic equation

02

2

=−∂∂ g

xf

An iterative process is analogous to solving an unsteadyproblem (recall ADI)

−

∂∂=

∂∂ g

xf

tf

2

2

α

And take it to the limit ∞→t


Analytic Solution Behavior

The equation can be solved by Fourier series

∑=k

ikxk etaf )(

and assume that can be expanded as

−

∂∂=

∂∂ g

xf

tf

2

2

α

Substituting into the equation,

g

∑=k

ikxk ekbg 2

( ) ikx

kkk

ikx

k

k ekbtaedt

da 2)(∑∑ −−= α


Solving for each

Therefore,

Rate of convergence

kk bta =∞→ )(

( )kkk btak

dtda −−= )(2α

k

( ) tkkkkk ebabta

2

)0()( α−−=−

= 2

2 1~k

k c ατα

High wave number modes damps out faster.

( )tkk e

2

0αεε −=


For iterative method to solve elliptic equations,there are high wave number and low wave number components of errors that need to be damped out.

Consider a domain discretized by points1=L N

1 N

kL=2π

kH=2πN


For explicit time step, stability condition yields

If the error at various wave number decays at

where

αα

2;

21 2

2

hth

t =∆=∆

tke2

0αεε −=

α2

2nhtnt =∆=

( )2/exp/ 220 nhk−=εε⇒

( ) NkhNn HH ππεε 2forexp 222

0

=−=

( ) ππεε 2forexp 22

0

=−= LL khn

Short-wave errorsdecay much faster


1 Nt = 0

t = 5∆t

ε

1 N

ε

1 N

ε

t = 500∆t


Multigrid Method – the Idea

- A low wave number component on a fine grid becomesa high wave number component on a coarse grid

- Use a coarse grid system to converge low wave numbercomponent of the solution rapidly

- Map it onto the fine grid system to converge high wavenumber component.

( ) ( )222220 /expexp/ NLnhnL ππεε −=−=


Suppose we are solving a Laplace equation:

jijijijijijiji

ji Ry

fffx

fffLf ,2

1,,1,2

,1,,1,

22=

∆+−

+∆

+−= −+−+

and iterate until residual becomes smaller than tolerance

ε<→ jiji RR ,, or0

Define ji

njiji fff ,,, ∆+=

ji

nji

ji

ff

f

,

,

,

∆

Converged solutionSolution after n-th iterationCorrection


Since

we have

0,,, =∆+= jinjiji fLLfLf

and ji

nji RLf ,, =

jiji RfL ,, −=∆ Coarse grid correction

Steps: Fine grid solution ⇒ Interpolation onto coarse grid (restriction)⇒ Coarse grid correction⇒ Interpolated onto fine grid (prolongation)⇒ Converge on fine grid


Two-Level Multigrid Method - (nx+1, ny+1); (nx/2+1, ny/2+1)

1. Do n iterations on the fine grid (n = 3~4) using G-S

2. If then stop.3. Interpolate the residual onto the coarse grid

(restriction (injection if nx/2))4. Iterations on the coarse grid

5. Interpolate the correction onto the fine grid(prolongation)

6. Go to 1.

jinji RLf ,, =

jiR ,

jiji RIfL ,2

1,2 )( −=∆jif ,2 )(∆

ε<jiR ,

jinjiji fIff ,2

12,, )(∆+=

jiRI ,2

1

jifI ,212 )(∆


Details on ProlongationCoarse GridFine grid

jif ,∆ jif ,2+∆

2, −∆ jif

jif ,2−∆

2, +∆ jif ( )jijiji fff ,2,,1 21

++ ∆+∆=∆

( )2,,1, 21

++ ∆+∆=∆ jijiji fff

( )

( )2,22,,2,

1,22,11,,1

1,1

4141

++++

++++++

++

∆+∆+∆+∆=

∆+∆+∆+∆=

∆

jijijiji

jijijiji

ji

ffff

ffff

f


The process can be extended to multi-level grids(nx+1,ny+1), (nx/2+1,ny/2+1), (nx/4+1,ny/4+1), …, (3,3)

Various Strategies

V-Cycle W-Cycle

exact solution

Fine

Coarse


Example: 4-Level V-Cycle

1. Iterate on Grid 1 for n times2. Restriction by injection to Grid 23. Iterate on Grid 2

(Save )4. Restriction by injection to Grid 35. Iterate on Grid 3

(Save )6. Restriction by injection to Grid 47. Iterate on Grid 4

1,, 0 ji

nji RLf ⇒=

1,

21 jiRI

1,

21,2 )( jiji RIfL −=∆

1,

21,2 ,)( jiji RIf∆n

jijiji fLRIR ,21,

21

2, )(∆+=

2,

32 jiRI

2,

32,3 )( jiji RIfL −=∆

2,

32,3 ,)( jiji RIf∆

Fine ⇒⇒⇒⇒ Coarse

njijiji fLRIR ,3

2,

32

3, )(∆+=

3,

43,4 )( jiji RIfL −=∆

3,

43 jiRI


1. Prolongate from Grid 4 to Grid 3

2. Iterate on Grid 3 (saved)3. Prolongate from Grid 3 to Grid 2

4. Iterate on Grid 2 (saved)5. Prolongate from Grid 2 to Grid 1

6. Iterate on Grid 17. If then stop. Else, repeat the entire cycle.

njifI ,4

34 )(∆

2,

32,3 )( jiji RIfL −=∆

Coarse ⇒⇒⇒⇒ Fine

nji

nji

nji fIff ,4

34,3,3 )()()( ∆+∆=∆

njifI ,3

23 )(∆n

jin

jin

ji fIff ,323,2,2 )()()( ∆+∆=∆

1,

21,2 )( jiji RIfL −=∆

njifI ,2

12 )(∆

nji

nji

nji fIff ,2

12,, )(∆+=

0, =jiLfε<jiR ,


Test Problem (Tannehill, p. 170)- Laplace equation in a square domain - Dirichlet conditions on four boundaries- 5 Levels of resolution

9×9, 17×17, 33×33, 65×65, 129×129- 4 Methods

1. Conventional Gauss-Seidel (GS)2. Gauss-Seidel-SOR with optimal ω (GSopt )3. Multigrid with 2-level grids (MG2)4. Multigrid with maximum levels of grid (MGMAX)


217322826826129×12920258137228265×6520957571533×3319404021517×17171919629×9

MGMAXMG2GSωoptGSGrid size


The multigrid method is the fastest techniquecurrently available for general elliptic equations, and are said to compare favorably with the fastest direct solvers for special problems.

MUDPACKhttp://www.scd.ucar.edu/css/software/mudpack/FISHPACKhttp://www.scd.ucar.edu/css/software/fishpack/

lecture 11

Documents