DiscretizationDiscretizationand Solution of and Solution of ConvectionConvection--Diffusion ProblemsDiffusion Problems

Howard ElmanUniversity of Maryland

1

Overview

1. The convection-diffusion equationIntroduction and examples

2. Discretization strategiesFinite element methodsInadequacy of Galerkin methodsStabilization: streamline diffusion methods

3. Iterative solution algorithmsKrylov subspace methodsSplitting methodsMultigrid

2

The Convection-Diffusion Equation

3,2,1 ,in 2 =⊂Ω=∇⋅+∇− dfuwu dRε

Boundary conditions:

NN

DD

gnu

gu

Ω∂=∂∂

Ω∂=

on

on

D

N

Ω∂Ω∂Ω

Inflow boundary: ∂Ω+ = {x∈∂Ω | w· n > 0}

Characteristic boundary:∂Ω0 = {x∈∂Ω | w· n = 0}

Outflow boundary:∂Ω- = {x∈∂Ω | w· n < 0}

3

The Convection-Diffusion Equationfuwu =∇⋅+∇− 2ε

Challenging / interesting case: ε 0

Reduced problem: w·∇ u = f , hyperbolic

Streamlines: parameterized curves c(s) in Ω s.t. c(s)has tangent vector w(c(s)) on c

fscudsdwu ==⋅∇⇒ ))(()(

Solutionu to reduced problem = solution to ODE

If u(s0)∈ inflow boundary ∂Ω+, and u(s1)∈∂Ω, say outflow ∂Ω− ,then boundary values are determined by ODE

4

Consequence 2 fuwu =∇⋅+∇−ε

For small ε, solution to convection-diffusion equation often hasboundary layers, steep gradients near parts of ∂Ω.

Αlso: discontinuities at inflow propagate into Ω along streamlines

Simple (1D) example of first phenomenon:

1near except 11

)(

/1

/)1)(/1(

=≈

−−−

=−

−−

xxeee

xxux

xε

εε

Solution

0)1( ,0)0(

,)1,0(on 1'''

===+−

uu

uuε

Solution to reduced equation

5

Additional Consequence

These layers (steep gradients) are difficultto resolve with discretization

6

Conventions of Notation

fLuwWLu

=∇⋅

+∇− εε

2

***2

In normalized variables:

,εWLP ≡ Peclet number, characterizes relative contributions

of convection and diffusion

L = characteristic length scale in boundarye.g. length of inflow boundary

= x/L in normalized domain

W = normalization for velocity (wind) w,e.g. w = W w*, where ||w*||=1

x̂

7

Reference Problems

−

−= −−−

ε

ε

/2

/)1(

11),(

eexyxu

y

2 fuwu =∇⋅+∇−ε

1. w=(0,1)Dirichlet b.c.analytic solution

2. w=(0,(1+(x+1)2/4)Neumann b.c. at outflowcharacteristic boundary layers

8

Reference Problems 2 fuwu =∇⋅+∇−ε

3. w: 30o left of verticalinterior layer from

discontinuous b.c.downstream boundary layer

4. w=recirculating(2y(1-x2),-2x(1-y2)characteristic boundary layersdiscontinuous b.c.

9

Weak Formulation 2 fuwu =∇⋅+∇−ε

}on 0 | {)(

}on | { )(

)(

),( allfor s.t. )( Find

1

1

11

0

0

DE

DDE

N

EE

vvH

gvvH

vgfvvuwvu

HvHu

N

Ω∂==Ω

Ω∂==Ω

+=∇⋅+∇⋅∇

Ω∈Ω∈

∫∫∫Ω∂ΩΩ

ε

Shorthand notation: a(u,v) =l(v) for all v

Can show:

a(u,u) ≥ ε ||∇u||2 = ε∫Ω∇ u ·∇ u

a(u,v) ≤ (ε+||w||∞ L) ||∇u|| ||∇v||

l(v) ≤ C ||∇v||

Lax-Milgram lemmaexistence and uniquenessof solution

10

Approximation by Finite Elements

a(uh,vh) =l(vh) for all vh

Typically: finite element spaces are defined by low-orderbasis functions, e.g. — linear or quadratic functions on triangles— bilinear or biquadratic functions on quadrilaterals

∫∫∫∫ Ω∂ΩΩΩ +=∇⋅+∇⋅∇∈∈

⊂⊂

NNhhhhhh

hh

hEh

Eh

EhE

gvvfvuwuu

SvSu

HSHS

)(

, allfor such that find

, , ldimensiona finiteGiven

0

10

1

0

ε

11

What happens in such cases?

Problem 1, accurate

Problem 2, accurate

Problem 1, inaccurate

Problem 2, inaccurate

12

Explanations

small is if large ,1 1

,)(inf)(

w

h

εε

εPWL

vuuu hSvw

h hE

+=+=Γ

−∇Γ≤−∇∈

1. Error analysis: discrete solution is quasi-optimal:

2. Mesh Peclet number: 2

2 εWh

LPhPh ==

If Ph>1, then — there are oscillations in the discrete solution— these become pronounced if mesh does not resolve layers— oscillations propagate into regions where solution is smooth— problem is most severe for exponential boundary layers

13

Revisit two examples

Problem 1, exponential layer, width ~ ε

Problem 2, characteristic layer, width ~ ε1/2

14

Fix: The Streamline Diffusion Method

Petrov-Galerkin method: change the test functions

Galerkin: a(uh,vh) =l(vh) for all vh

Petrov-Galerkin: a(uh,vh+δ w·∇ vh) =l(vh+δ w·∇ vh) for all vhδ is a parameter

Result: asd(uh,vh) =lsd(vh)

)()()(

)()(

))(()(),(

2

∫∫∫

∑ ∫∫∫∫

Ω∂ΩΩ

∆

ΩΩΩ

∇⋅++∇⋅+=

∇⋅∇−

∇⋅∇⋅+∇⋅+∇⋅∇=

N

k

Nhhhhh

hk h

hhhhhhhhsd

gvwvvwfvfvl

vwu

vwuwvuwuuvua

δδ

δε

δε

Streamline diffusion term

0 for linear/bilinear

15

The Streamline Diffusion Method Explained

Augment finite element space:

Bh: bubble functions, with support local to element

hBSS += hhˆ

We could pose the problem on the augmented space:find uh in s.t. a(uh,vh) =l(vh) for all vh in

Then: decouple unknowns associated with bubble functionsfrom system new problem on original grid

Principle: augmented space places basis functions inlayers not resolved by the grid

hŜ

hŜ hŜ

16

The Streamline Diffusion Method Explained

Under appropriate assumptions: this new problem is

)()(

))((

)(),(

∑ ∫∫

∑ ∫

∫∫

∇⋅+=

∇⋅∇⋅+

∇⋅+∇⋅∇=

∆Ω

∆ ∆

ΩΩ

k hkhh

hhk

hhhhhhsd

vwfvfvl

vwuw

vuwuuvua

k

k k

δ

δ

ε

asd(uh,vh) =lsd(vh)

k determined from elimination of bubble functionsStreamline diffusion

17

Compare Galerkin and Streamline Diffusion

Middle: bilinear elements, Galerkin,32× 32 grid

Top: accurate solution, ε=1/200

Bottom: bilinear elements, streamline diffusion,32× 32 grid

18

Error Bounds

)(inf)(h

hSvw

h vuuu hE

−∇Γ≤−∇∈ε

For Galerkin: as noted earlier, quasi-optimality:

More careful analysis: for linear/bilinear elements,

)( 2uDChuu h ≤−∇ Large in exponential boundary layers for small ε

For streamline diffusion: use norm

( ) 1/222 vwvvsd

∇⋅+∇≡ δε

22/3 uDChuusdh

≤−Then

19

These bounds do not tell the whole story

4.98e-74.98e-72.392.3964×64

(1)

1.11e-55.30e-23.233.8132×32

(2)

1.64e-51.484.014.9116×16

(4)

8.16e-73.254.345.628×8(8)

Str.Diff.Ω∗

GalerkinΩ*

Str.Diff.Ω

GalerkinΩ

Grid (Ph )

For one example (Problem 1, ε=1/64), compareerrors ||∇(u-uh)|| on Ω and Ω* = (-1,1)×(-1,3/4) (to exclude boundary layer)

20

Choice of parameter δδδδ

∑ ∫

∫∫

∆ ∆

ΩΩ

∇⋅∇⋅+

∇⋅+∇⋅∇=

k khhk

hhhhhhsd

vwuw

vuwuuvua

))((

)(),(

δ

εMade element-wise:

( )

≤

>−=

1 if 0

1 if /11||2

kh

kh

kh

k

k

k

P

PPw

hδ

21

Matrix Properties

nila

u

SS

jjij

j jjh

hE

nnnj

hnjj

,...,2,1 ),(u),(

such that }{u find : becomes

problem , u isfunction element Finite

for }by{ extended ,for }{ basis aGiven

j

j

101

==

=

∑

∑∂+

+=

ϕϕϕ

ϕϕϕ

or asd

Leads to matrix equation Fu=f,

F=ε A + N (+ S)

22

Matrix Properties

Matrix equation Fu=f, F=ε A + N (+ S)

A=[aij], aij=∫Ω ∇φj·∇φi, , discrete Laplacian, symmetric positive-definite

N=[nij], nij=∫Ω (w·∇φj)φi , discrete convection operator,skew-symmetric (N=-NT)

S=[sij], sij=∫Ω (w·∇φj)(w·∇φi) , discrete streamlineupwinding operator, positive semi-definite

23

End of Part I

Next: how to solve Fu=f ?

24

Iterative Solution Algorithms: Krylov Subspace Methods

System Fu=f

• F is a nonsymmetric matrix, so an appropriate Krylov subspacemethod is needed

• Examples:• GMRES• GMRES(k) restarted• BiCGSTAB• BiCGSTAB(l)

• Our choices: • Full GMRES for optimal algorithm, or • BiCGSTAB(2) for suboptimal

25

Properties of Krylov Subspace Methods

Drawback of GMRES: work & storage requirements at step kare proportional to kN

BiCGSTAB: Fixed cost per step, independent of k

Drawback: No convergence analysis

Variant: BiCGSTAB(l), more robust for complex eigenvalues,somewhat higher cost per step (l=2), but still fixed

26

Convergence of GMRES

GMRES: Starting with u0, with residual r0=f-Fu0, computesuk ∈ span{ r0, Fr0, …,Fk-1r0}

for which rk = f-Fuk satisfies

||rk|| = minpk(0)=1 || pk(F)r0||.

Consequence:

Theorem: For diagonalizable F=VΛV-1, ||rk|| ≤ ||V|| ||V-1|| minpk(0)=1 maxλ∈σ(F)|pk(λ)| ||r0||.(||rk||/||r0||)1/k ≤ (||V|| ||V-1||)1/k (minpk(0)=1}maxλ∈σ(F)|pk(λ)|)

1/k

ρ̂Loosely speaking: residual is reduced by factor of at each step

Want eigenvalues to lie in compact setρ̂

27

Convergence of GMRES

)(11

ˆ 12

22

FF RQad

daa −=≤−+−+≈ ρρ

0 1

.

.1+d

1+a

Size of convergence factor ρ̂

28

Key for Fast Convergence: PreconditioningSplitting operators

SeekQF≈ F such that• the approximation is good, and• it is inexpensive to apply the action of Q-1 to a vector

Splitting: F=QF-RF stationary iteration

Error ek = u-uk satisfies

)(11 fuRQu kFFk +=−

+

( ) )()(/)(

)(

))(()(

1/11/1

0

01

01

111

FQIFQIee

eFQIe

eFQIe

uuFQIuuRQuu

F

kkF

k

k

kFk

kFk

kFkFFk

−−

−

−

−−+

−≈−≤

−≤

−=

⇒−−=−=−

ρ

29

Preconditioning / Splitting operators

Thus: want to be as small as possibleEquivalently: eigenvalues of

= eigenvalues of

)( 1FQI F−−ρ

FQF1−

1−FFQ

as close to 1 as possible

0 1

)( 1 FF RQ−ρ

This is similar to the requirement for rapid convergence of GMRES

Solve uQufuFQfQFuQ FFFF ˆ ,ˆor 1111 −−−− ===

30

Examples of splitting operators

=FQ lower triangle of AGauss-Seidel=FQ block lower triangle of ALine Gauss-Seidel

FFF ULQ =Symmetric versionsIncomplete LU factorization

FQLUF =≈

Comments:

• All depend on ordering of underlying grid

• Symmetric versions (symmetric GS, ILU) take some account of underlying flow

• Line/block versions can handle irregular grids

31

Convergence Analysis (Parter & Steuerwalt)

Seek maximal eigenvalue of uuRQ FF λ=−1 or uRuQ FF = λ

Subtract uRF λ from both sides

uRhuRuRQ FFFFh

)()( 2211

=

=− −−λ

λλ

λ

uRhFu Fh )(2µ=

Suggests relation to uu µ= LL uwuu ∇⋅+∇−= 2εR to be determined

R

For many examples of splittings:)( ),,(),(2 xrrvurvuRh hhF =≈operator"tion multiplicaweak " a is 2 FRh

and (2) of eigenvalue minimal )0( =→ µµh

(1)

(2)

(defines R)

32

are known:

+

+=⇒

+

++=

=

222)0(

22222

2/2/),(

222

22)(

)sin()sin( 21

εεπεµ

εεπεµ

ππ

yx

yxjk

ywxwkj

wwr

wwkj

r

ykexjeu

Consequence: 2)0(1 1)( hRQ FF µρ −=−

For model problems:(i) r is constant (will demonstrate in a moment)(ii) on square domains, eigenvalues, eigenvectors of

uu µ= L R

To find r: consider centered finite differences

+−2hwxε

−−2hwxε

−−

2hwyε

+−

2hwyε

ε4

For horizontal line Jacobi splitting, = block tridiagonal:FQ

)(222

][ 21,1, hOuuhw

uhw

uR ijjiy

jiy

ijF +=

++

−= −+ εεε

,2ε=⇒ r

Key point: convection terms lead to smaller convergence factors

2

222

222

211 h

ww yx

+

+−= εεπρ

34

Comments / Extensions

• Similar results obtained from matrix/Fourier analysis

• Young:

• “Multi-line” (k-line) splittings r =

• Can extend to other splittings via matrix comparison theorems

(Varga-Woźnicki):

k/2ε

2operator) Jacobi (lineoperator) Seidel-Gauss (line ρρ =

)()( 1-112

-12

11

12 RQRQQQ ρρ ≤⇒≥

−−

35

x x x x x x

x x x x x x

x x x x x x

x x x x x x

x x x x x x

x x x x x x

6

5

4

3

2

1

Limitations of analysis above: It does not discriminateamong different orderings

Natural ordering of gridLeft-to-right, bottom-to-topPlus resulting matrix structure:

Horizontal line red-black Ordering and matrix structure:

x x x x x x

x x x x x x

x x x x x x

x x x x x x

x x x x x x

x x x x x x

6

3

5

2

4

1

Young theory: spectral radii (Jacobi or Gauss-Seidel) independent of ordering

Performance of GS: depends on ordering

36

Example: Problem 1 60Pelements,linear piecewise ,)1,0( 22 ==+∇− fuu yε

Four solution strategies: line Gauss-Seidel iteration withnatural line ordering, following the flow (bottom-to-top)natural line ordering, against the flow (top-to-bottom)red-black line ordering, with the flowred-black line ordering, against the flow

10-5

10-4

10-3

10-2

10-1

100

101

102

0 10 20 30 40 50 60

*o

*o*o

k

||e^(

k)||

Naturalwith flow

Red-black

Naturalagainst flow

With

Against

37

Ordering effects

⇒=−= − 01 )( satisfies Error eRQeuue kFFkkk

01

01 )()( eRQeRQe kFF

kFFk

−− ≤=⇒

“Classical” analysis only provides insight in asymptotic sense: 1

)(lim

/11

=−

∞→ ρ

kkFF

k

RQ

E. & Chernesky:bounds for for 1D problems

kFF RQ )(

1−

38

Practical consequencesFor nonconstant flows: inherent latencies if sweeps don't follow flowPossible fixes:• flow-directed orderings (Bey & Wittum, Kellogg, Hackbusch, Xu)• iterations based on multi-directional sweeps2D version:

Speeds convergence when recirculations are present

)(

)(

)(

)(

4/31

44/31k

2/11

32/13/4k

4/11

24/11/2k

111/4k

+−

++

+−

++

+−

++

−+

−+=

−+=

−+=

−+=

kk

kk

kk

kk

FufQuu

FufQuu

FufQuu

FufQuu

Contours of stream function

39

Summarizing with an experiment:

Eigenvalues of line-GSpreconditioned operator,vertical flow, P=40, h=1/32

Asymptotic convergence rateis faster with Krylov acceleration

However: does not overcome latencies

40

MultigridFlow-following methods are effective for convection-dominatedproblems:

But: ultimately, solvers discussed above are mesh dependent

GMRES performance for Problem 4

41

Multigrid

V-cycle multigrid:

end

iteration)next for (update

h)(postsmoot )( steps, for

update) and correction (prolong ˆ

ˆˆ problem coarse tosystem multigridapply

residual)(restrict )(r̂

)(presmooth )( steps, for

econvergenc until 0for

Choose

1

11

2

T

11

0

ii

FiFi

ii

h

i

FiFi

uu

fQuFQIum

ePuu

reF

FufP

fQuFQIuk

i

u

←+−←

+←=

−=

+−←=

+

−−

−−

42

Bottom Line: Performance

43214321Grid

23332533128× 128

3333243364×64

5433243332× 32

6633333416× 16

Example Example

ε=1/25 ε=1/200

Multigrid iterations for ||rk||/||r0||

43

For this to happen:

)( :Smoothing 1. 11 fQuFQIu FiFi−− +−←

Two things have to be done correctly

reF h ˆˆ :solve grid Coarse 2. 2 =

Smoother must take underlying flow into account

For results above for Problem 4 (recirculating wind): smoother is 4-directional

Coarse grid operator must be stable

Even if fine grid is “fine enough,” coarse grid operators should include streamline diffusion

44

Example / effect of smoother

After one four-directionalGauss-Seidel step

After four one-directionalGauss-Seidel steps

45

Concluding Remarks

• Discretization requires stabilization for convection-dominatedproblems

• The best solution algorithms combine • general techniques of iterative methods • splitting strategies coupled to the underlying physics• stabilization when needed

46

References

• J. J. H. Miller, E. O’Riordan and G. I. Shishkin,Fitted Numerical Methods for Singularly Perturbed Problems,World Scientific, 1995.

• K. W. Morton, Numerical Solution of Convection-DiffusionProblems, Chapman & Hall, 1996.

• H.-G. Roos, M. Stynes and L. Tobiska, Numerical Methods for Singularly Perturbed Differential Equations, Springer,1996.

• H. C. Elman, D. J. Silvester and A. J. Wathen, Finite Elementsand Fast Iterative Solvers, Oxford University Press, 2005.