advanced computational mechanics · chapter 1 ordinary diﬀerential equations in many engineering...

Advanced Computational Mechanics

A. Ooi

October 14, 2005

Contents

1 Ordinary Differential Equations 51.1 Euler’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Backward (Implicit) Euler method . . . . . . . . . . . . . . . . . . . 91.3 Trapezoidal or Crank-Nicolson method . . . . . . . . . . . . . . . . . 101.4 Linearization of Crank-Nicolson method . . . . . . . . . . . . . . . . 111.5 Runge-Kutta methods . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.5.1 Second Order Runge-Kutta Method . . . . . . . . . . . . . . . 141.5.2 4th Order Runge-Kutta Scheme (RK-4) . . . . . . . . . . . . 16

1.6 Stability and error analysis . . . . . . . . . . . . . . . . . . . . . . . . 171.7 Systems of Ordinary Differential Equations . . . . . . . . . . . . . . . 241.8 Runge-Kutta-Fehlberg method: Runge-Kutta with error control . . . 281.9 Boundary Value problem . . . . . . . . . . . . . . . . . . . . . . . . . 32

2 Fourier series 372.1 Some properties of the Fourier coefficients . . . . . . . . . . . . . . . 402.2 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3 Finite Difference 473.1 General Finite Difference Schemes . . . . . . . . . . . . . . . . . . . . 47

3.1.1 First Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 473.1.2 Some popular differencing schemes . . . . . . . . . . . . . . . 483.1.3 Higher order derivatives . . . . . . . . . . . . . . . . . . . . . 513.1.4 Summary of finite difference formula . . . . . . . . . . . . . . 52

3.2 Centred Difference Schemes . . . . . . . . . . . . . . . . . . . . . . . 533.2.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.3 Solving PDEs using finite difference schemes . . . . . . . . . . . . . . 563.4 Fourier Analysis of error . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.4.1 Fourier analysis of central differencing scheme . . . . . . . . . 653.5 Stability analysis using the modified wavenumber . . . . . . . . . . . 683.6 Dispersion-Relation-Preserving Scheme . . . . . . . . . . . . . . . . . 713.7 General Finite Difference Schemes For The Second Derivative . . . . 72

3.7.1 Some popular differencing schemes . . . . . . . . . . . . . . . 733.8 Multidimensional problems . . . . . . . . . . . . . . . . . . . . . . . . 75

3

4 CONTENTS

3.8.1 Steady problem . . . . . . . . . . . . . . . . . . . . . . . . . . 753.8.2 Unsteady problem . . . . . . . . . . . . . . . . . . . . . . . . 773.8.3 Modified wavenumber stability analysis . . . . . . . . . . . . . 81

3.9 Test Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 833.10 Euler equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4 Differentiation: Unequally spaced data 874.1 Approximation of the 2nd Derivative . . . . . . . . . . . . . . . . . . 894.2 Application of Finite Difference Formulas . . . . . . . . . . . . . . . . 92

5 Galerkin Method 955.1 Convection equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 955.2 Burgers Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.3 Aliasing error in the calculation of the nonlinear term . . . . . . . . . 98

6 Collocation Method 1016.1 Matrix operator for Fourier spectral numerical differentiation . . . . . 101

7 Some numerical examples 1037.1 Heating of an oil droplet in air . . . . . . . . . . . . . . . . . . . . . 1037.2 Blasius solution:

Contributed by Mr. M. Giacobello . . . . . . . . . . . . . . . . . . . 109

Chapter 1

Ordinary Differential Equations

In many engineering problems, you will need to solve differential equations that looksomething like

dx

dt= f(t, x) (1.1)

in the domain

a ≤ t ≤ b

with the initial condition

x(t = a) = α

1.1 Euler’s method

Euler’s method is probably the simplest method used to solve Eq. (1.1). ConsiderTaylor’s theorem

x(tn+1) = x(tn) + (tn+1 − tn)dx

dt(tn) +

(tn+1 − tn)2

2

d2x

dt2(ξn) (1.2)

where ξ is somewhere in between tn and tn+1.

Exercise 1.1: Find the numerical value of ξ if x(t) = sin(πt/2), tn = 0, tn+1 = 1,hence h = 1. Note that according to the Eq. (1.2), the value of ξ must lie between0 and 1.

If we let tn+1 − tn = h and substitute Eq. (1.1) into Eq. (1.2), we get

x(tn+1) = x(tn) + hf(tn, xn) +h2

2

d2x

dt2(ξn). (1.3)

If we assume that h is small, then we can neglect the second order term in theequation above. Thus, we get the formula for Euler’s method

5

6 CHAPTER 1. ORDINARY DIFFERENTIAL EQUATIONS

xn+1 = xn + hf(tn, xn) (1.4)

where xn is the numerical approximation of the exact solution x(tn). Equation(1.4) is sometimes also called the explicit Euler formula because there is an implicitversion of the Euler’s method will be be discussed later.

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(a)

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(b)

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(c)

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(d)

Figure 1.1: Figure showing the solution to Exercise 1.2. h = 2.0 (a), h = 1.0 (b),h = 0.5 (c), h = 0.1 (d). exact solution, computed solution.

1.1. EULER’S METHOD 7

Exercise 1.2: Using Euler’s method, solve

dx

dt= 1− x

for 0 ≤ t ≤ 10. Use x(t = 0) = 0 and

(a) h = 2.0

(b) h = 1.0

(c) h = 0.5

(d) h = 0.1

Compare the numerical solution with the analytical (exact) solution which is x(t) =1− e−t.

The solution to Exercise 1.2 is shown in Figure 1.1. Note that the numerical solutiongets closer to the exact solution for smaller values of h. By comparing, Eqs. (1.3)and (1.4), one can conclude that if the exact solution is known at time t = tn, thethe numerical solution at time t = tn+1 will have an error which is O(h2). One cansay that the Euler’s method has a local truncation error of order 2. This is becauseif we reduce h by 2, then it can be expected that the error will be approximatelyreduced by 4. However, it can be shown that (see [3] and [1] and Exercise 1.3 ) theerrors actually accumulate over time. This is illustrated in Fig. 1.2 which showsthe error associated with the Euler’s method when the solution is computed usingh = 0.2, 0.1 and 0.05. Figure 1.2 (a) plots the error

Eh(tn) = |x(tn)− xn| (1.5)

for various values of h. The gaps between all three curves are the same whichindicates that the error is proportional to h. This fact is confirmed in Fig. 1.2(b)which shows the ratio of the errors, Eh=0.1/Eh=0.2 error Eh=0.05/Eh=0.2. From thegraphse, it is clear that the error is approximately halved when the h is halved.Thus, the global error is shown to be O(h) i.e. Euler’s method is only first orderaccurate. In summary, Euler’s method is the easiest method to implement but it isnot very accurate.


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 510!5

10!4

10!3

10!2

10!1

100

t

Erro

r

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.2

0.4

0.6

0.8

1

t

Erro

r rat

io

(a)

(b)

Figure 1.2: Figure showing the error associated to the solution to Exercise 1.2. Theerror is shown in (a) with h = 0.2 , h = 0.1 ,h = 0.05 . (b) showsthe ratio between the error, Eh=0.1/Eh=0.2, Eh=0.05/Eh=0.2

Exercise 1.3: Subtract Eq. (1.4) from Eq. (1.3) and show that

εn+1 = εn + h(f(tn, x(tn))− f(tn, xn)) +h2

2x′′(ξn). (1.6)

where εn = x(tn) − xn, the error at time t = tn. For the case where f(t, x) = 2t,use the above equation and show that the error at time, tn = nh is O(h).

Note: Even though you have only proven that the error is O(h) for the specificcase where f(t, x) = 2t, this result is more general and applies to all cases wheref(t, x) is a smooth function.

1.2. BACKWARD (IMPLICIT) EULER METHOD 9

1.2 Backward (Implicit) Euler method

In many applications (see later) the Euler method described in section 1.1 is veryunstable. A more stable method is the backward Euler scheme given by the followingformula:

xn+1 = xn + hf(tn+1, xn+1) (1.7)

The Backward Euler method is an implicit method because it has xn+1 on bothsides of the equation. For simple problems, there is generally no real differencebetween the implicit Euler’s method and the more conventional explicit Euler’smethod because it is possible to obtain an explicit expression for xn+1 from Eq.(1.7). As an example, for the problem discussed in Exercise 1.2, one can use Eq.(1.7) to show that

xn+1 =h+ xn

1 + h(1.8)

Using the Eq. (1.8), one can generate the results shown in Fig. 1.3. ComparingFigs. 1.3(a) and 1.1(a), it is clear that the Backward Euler method gives a bettersolution h = 2 in this situation. This is one property of the implicit Eulers method.For a given value of h, it usually gives a better solution than the explicit Euler’smethod.

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(a)

0 1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

t

x(t)

(b)

Figure 1.3: Figure showing the solution to Exercise 1.2 using the Backward Eulermethod. h = 2.0 (a), h = 1.0 (b). exact solution, computed solution.

However, in generally, it is more difficult to obtain a solution using backward Eulersmethod than the (explicit) Eulers method. This commonly occurs when you have aslightly more complicated expression for f(t, x) which is illustrated in Example 1.1


Example 1.1: Consider the nonlinear first order ordinary differential equation

dx

dt− cos(x) = 0 (1.9)

with the initial condition x(0) = 0.0. The exact solution to Eq. (1.9) is

x(t) = arctan

(e2t − 1

2et

)(1.10)

We will now show how the backward Euler method can be used to obtain anapproximate solution to Eq. (1.9). Applying Eq. (1.7) to Eq. (1.9) gives

xn+1 − h cos(xn+1)− xn = 0. (1.11)

xn is known from the previous time step. One then needs to find xn+1. It is noteasy to find an explicit expression for xn+1. Root finding methods (see 2nd yearcomputational mechanics lecture notes) must be used to find xn+1 numerically. Ifwe used the Newton-Raphson method, the iterative scheme could be written as

x(k+1)n+1 = x

(k)n+1 −

x(k+1)n+1 − h cos(x

(k+1)n+1 )− xn

1 + h sin(x(k+1)n+1

where x(k)n+1 is the k’th guess for the value of xn+1 that satisfies Eq. (1.11). While

one may use any value for x(k)n+1, it would be logical to use x

(0)n+1 = xn.

Figure 1.4 shows the numerical solution to Eq. (1.9) obtained using both theimplicit and the explicit Euler’s method. For large h = 2.0, the solution obtainedusing explicit Euler is oscillatory even though the exact solution does not oscillate.The backward Euler gives a solution that is in better agreement with the exactsolution. For h = 0.5 both implicit and explicit Euler solution converges to theexact analytical solution.

1.3 Trapezoidal or Crank-Nicolson method

The solution to Eq. (1.1) can also be obtain by integration

x(t) = x(tn) +

∫ tn+1

tn

f(t, x(t))

The integration can be approximated using the trapezoidal rule (see 2nd year, Com-putational mechanics lecture notes)∫ tn+1

tn

f(t, x(t)) =h

2(f(tn, x(tn)) + f(tn+1, x(tn+1))) +O(h3) (1.12)

1.4. LINEARIZATION OF CRANK-NICOLSON METHOD 11

0 1 2 3 4 5 6 7 8 9 100

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

t

x(t)

(a)

0 1 2 3 4 5 6 7 8 9 100

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

t

x(t)

(b)

Figure 1.4: Figure showing the solution to Example 1.1 for h = 2.0 (a) h = 0.5 (b).exact solution, explicit Euler and backward Euler solution.

where h = tn+1 − tn = h. Ignoring the error term, one will obtain the followingformula for the Crank-Nicolson (trapezoidal) method

xn+1 = xn +h

2(f(tn, xn) + f(tn+1, xn+1)) (1.13)

By comparing Eqs. (1.12) with (1.13) one can conclude that the error associatedwith the Crank-Nicolson method is O(h3). However, similar to the Euler’s method,the error accumulate over time. Thus, the global error of the Crank-Nicolson methodis O(h2).

Exercise 1.4: Solvedx

dt+ 2x = 0

for 0 < t < 3 with x(t = 0) = 1 using

(a) Euler method

(b) Backward Euler method

(c) Crank Nicolson method

1.4 Linearization of Crank-Nicolson method

The main problem with implicit method is that they are difficult to solve, i.e. itis difficult to obtain xn+1 from the resulting discretized equations. Sometimes, it ispossible to “linearized” implicit methods. Consider the Crank-Nicolson scheme

xn+1 = xn +h

2(f(tn, xn) + f(tn+1, xn+1)) +O(h3) (1.14)


The difficulty comes from the term f(tn+1, xn+1). Let’s consider a Taylor seriesexpansion about of f(tn+1, xn+1) about xn

f(tn+1, xn+1) = f(tn+1, xn)+(xn+1−xn)∂f

∂x(xn, tn+1)+(xn+1−xn)2∂

2f

∂x2(xn, tn+1)+HOT

(1.15)But we know that

xn+1 − xn = O(h)

Substituting into Eq. (1.15) gives

f(tn+1, xn+1) = f(tn+1, xn) + (xn+1 − xn)∂f

∂x(xn, tn+1) +O(h2) (1.16)

Substituting Eq. (1.16) into (1.14) gives

xn+1 = xn +h

2

(f(tn+1, xn) + (xn+1 − xn)

∂f

∂x(xn, tn+1) + f(tn, xn)

)(1.17)

It is now possible to obtain an explicit expression for xn+1 to be

xn+1 = xn +

[h2(f(xn, tn+1) + f(xn, tn))

1− h2

∂f∂x

(xn, tn+1)

](1.18)

This method has got good stability characteristics (see later). The main problemwith this scheme is that you have to find the derivative of f with respect to x. Thisis not always possible.

Example 1.2: Consider again the example

dx

dt= cos(x)

Applying the Crank Nicolson method will give an implicit expression for xn+1

xn+1 −h

2cos(xn+1)− xn −

h

2cos(xn) = 0 (1.19)

Since xn is known from the previous time step, it is possible to use root findingmethods to find a solution to Eq. (1.19). On the other hand, applying the linearizedCrank-Nicolson method (Eq. (1.18)) gives an explicit expression for xn+1

xn+1 = xn +h cos(xn)

1 + (h/2) sin(xn)(1.20)

1.5. RUNGE-KUTTA METHODS 13

As you can probably imagine, writing a computer program to implement Eq. (1.19)is a lot simpler than writing a computer program to implement Eq. (1.20). Theresult for both the implicit and linearized Crank-Nicolson methods are shown in Fig.1.5. As you can see, both methods give very similar results.

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

t

x(t)

Figure 1.5: Comparing the linearized and implicit Crank-Nicolson methods for h =1. linearized Crank-Nicolson and implicit Crank-Nicolson method.

1.5 Runge-Kutta methods

These methods are probably the most popular methods in solving initial value prob-lems. However, many variations of the Runge-Kutta methods exist. We will deriveone set below and see why the various formulae are not unique. Let’s start with theproblem we would like to solve

dx

dt= f(t, x)

In general, the Runge-Kutta schemes can be written as

xn+1 = xn + φ(xn, tn, h)h (1.21)

where h is the interval size, i.e. h = ∆t = tn+1 − tn. φ is known as the incrementalfunction and it can be interpreted as the slope which is used to predict the newvalue of x. In general, φ can be written as


φ = a1k1 + a2k2 + a3k3 + a4k4 + · · · aNkN (1.22)

where

k1 = f(tn, xn)

k2 = f(tn + p1h, xn + q11k1h)

k3 = f(tn + p2h, xn + q21k1h+ q22k2h)

k4 = f(tn + p3h, xn + q31k1h+ q32k2h+ q33k3h)...

......

......

......

......

kN = f(tn + pN−1h, xi + qN−1,1k1h+ qN−1,2k2h+ · · ·+ qN−1,N−1kN−1h)

For N = 1, we get the first order Runge-Kutta scheme. This is just the same as theEuler integration scheme presented earlier.

1.5.1 Second Order Runge-Kutta Method

If we put N = 2 into Eq. (1.22), we get

φ = a1k1 + a2k2

Substituting the above into Eq. (1.21) gives

xn+1 = xn + (a1k1 + a2k2)h

xn+1 = xn + a1f(tn, xn)h+ a2f(tn + p1h, xn + q11k1h)h (1.23)

Thus in order to find a numerical scheme, we need to find values for the followingconstants, a1, a2, p1 and q11. The last term in the above equation is a 2 variablefunction and the Taylor series expansion (to the lineariazed approximation) for atwo variable function is given by

f(t+ h, x+ ∆) = f(t, x) + h∂f

∂t+ ∆

∂f

∂x+ HOT

Using this relationship and ignoring the higher order terms (HOT),

f(tn + p1h, xn + q11k1h) = f(tn, xn) + p1h∂f

∂t(tn, xn) + q11k1h

∂f

∂x(tn, xn) (1.24)

We know that

1.5. RUNGE-KUTTA METHODS 15

k1 = f(tn, xn)

Hence Eq. (1.24) can be written as

f(tn + p1h, xn + q11k1h) = f(tn, xn) + p1h∂f

∂t+ q11f(tn, xn)h

∂f

∂x(1.25)

Substituting Eq. (1.25) into Eq. (1.23) gives

xn+1 = xn + (a1 + a2)f(tn, xn)h+ a2p1∂f

∂th2 + a2q11f(tn, xn)

∂f

∂xh2 (1.26)

We can also write a Taylor series expansion for x in terms of t as

x(tn+1) = x(tn) +dx

dt(tn)h+

d2x

dt2(tn)

h2

2!+HOT

where HOT stands for higher order terms. Let’s assume that they are small. So theabove equation becomes

xn+1 = xn +dx

dth+

d2x

dt2h2

2!(1.27)

The problem we are trying to solve has

dx

dt= f(t, x)

Substituting the above into Eq. (1.27) gives

xn+1 = xn + f(tn, xn)h+df(tn, xn)

dt

h2

2

df =∂f

∂tdt+

∂f

∂xdx

df

dt=

∂f

∂t+∂f

∂x

dx

dt(1.28)

Hence

xn+1 = xn + f(tn, xn)h+1

2

∂f

∂th2 +

1

2

∂f

∂xf(tn, xn)h2 (1.29)

Comparing Eqs. (1.29) with Eq. (1.26) will give you the following three equations


a1 + a2 = 1

a2p1 =1

2

a2q11 =1

2(1.30)

We have four unknowns (a1, a2, p1 and q11) but only the above three equations.So there cannot be a unique solution. You have an infinite number of solutions. Onepossible solution is to set

a2 =1

2

then

a1 =1

2p1 = 1

q11 = 1

(1.31)

Hence, one possible second order Runge-Kutta time stepping scheme is

xn+1 = xn +

(1

2k1 +

1

2k2

)h (1.32)

where

k1 = f(tn, xn)

k2 = f(tn + h, xn + hk1)

(1.33)

1.5.2 4th Order Runge-Kutta Scheme (RK-4)

By far, the most popular numerical method for solving ODE is the 4th order Runge-Kutta scheme

xn+1 = xn +

(1

6k1 +

1

3(k2 + k3) +

1

6k4

)h (1.34)

where

1.6. STABILITY AND ERROR ANALYSIS 17

k1 = f(tn, xn)

k2 = f

(tn +

h

2, xn +

h

2k1

)k3 = f

(tn +

h

2, xn +

h

2k2

)k4 = f (tn + h, xn + hk3)

This method is very accurate and has good stability properties.

1.6 Stability and error analysis

When discussing stability of the numerical methods, one usually considers the modelproblem

dx

dt= λx (1.35)

λ is a constant which can be a complex number. In most engineering problems, thereal part of λ is usually negative. This means that the solution that you are afterwill typically decay with time. Applying the Euler method (with timestep ∆t = h)to Eq. (1.35) gives

xn+1 = xn + λhxn (1.36)

where xn is the numerical solution of x(tn) where tn is the time tn = nh.To determine the region of stability, simplify Eq. (1.36) to give

xn+1 = (1 + λh)xn

Thus the error at any time step n can be written as

xn = x0 (1 + λh)n

= x0 (1 + (λR + IλI)h)n

= x0σn

(1.37)

where λR and λI are the real and imaginary parts of λ = λR + IλI .

σ = (1 + hλR + IhλI)


is usually called the amplification factor. If |σ| < 1 then the error will decay withtime. The opposite is true if |σ| > 1. Hence, in order to ensure the stability of thenumerical method, |σ| < 1.

|σ|2 = (1 + hλR)2 + (hλI)2 < 1

This is just a circle of radius 1 centred on (-1,0). This plot is called the stabilitydiagram and is shown in Fig. 1.6

2

λ h

λ h

I

R

Figure 1.6: The stability diagram of Euler method.

If λ is real and negative, then in order for the numerical method to be stable,

h ≤ 2

|λ|(1.38)

Exercise 1.5: Solve

dx

dt+ 2x = 0 (1.39)

with

x(t = 0) = 1

Use h = 1.2, 0.8 and 0.3 in the domain 0 ≤ t ≤ 30

Consider now the case where λ is purely imaginary i.e.

λ = Iω


where I is the imaginary unit and ω is a real number. The model problem forstability, Eq. (1.35) becomes

dx

dt= Iωx

For illustrative purposes, let’s use x(t = 0) = 1. The exact solution to this problemis

x(t) = x0eIωt (1.40)

where x0 = x(t = 0). If the Euler method is used to solve this equation, we knowthat the amplitude will grow with time as the value of λ is not within the stabilityregion. Let’s now analyse the phase error. The amplification factor, σ for thisproblem can be written as

σ = 1 + Iωh

= |σ|eIθ (1.41)

whereθ = arctan(ωh)

Let’s now compare the exact solution at time t = nh

x(t = nh) = x0eIωnh

with the approximated solution given by the Euler’s equation.

xn = σnx0 = AneInθx0

Dividing the two equations and a little bit of algebra will give you

xn = x(nh)AneIn(θ−ωh)

So the approximated solution that you will get will be amplified by An and itsphase will be shifted by n(θ−ωh). Thus A is usually called the amplitude error and(θ − ωh) is called the phase error.

At any time step, h, the error associated with the phase is ωh− θ

ωh− θ = ωh− arctan(ωh)

= ωh−(ωh− (ωh)3

3+

(ωh)5

5− (ωh)7

7+ ......

)=

(ωh)3

3− (ωh)5

5+

(ωh)7

7+ ......

(1.42)


So the phase error has a leading order term of (ωh)3 which is very small. In contrast,the amplitude error grows like An.

We can also apply the stability analysis to the model equation using the implicitEuler method

xn+1 = xn + λxn+1h

Rearranging the above equation gives

xn+1 =xn

1− λh

hence

xn = σnx0

where

σ =1

1− λh

Exercise 1.6: Show that the amplification factor, σ for the implicit Euler schemecan be written as

σ = Aeiθ

where

A =1√

(1− λRh)2 + (λIh)2

and

θ = arctanλIh

1− λRh

Since we are only interested in problems which have λR < 0, A is always smaller thanunity. Thus the implicit Euler scheme is always stable (as long as λR < 0), no matterwhat the value of h. This means that the implicit Euler scheme is unconditionallystable, which is a property of most implicit numerical scheme.


Exercise 1.7: Compare the implicit and explicit Euler scheme and show that forthe model problem

dx

dt= Iωx,

the two schemes have the different amplitude error but the phase error is thesame. Show that the magnitude of the solution given by the explicit Euler schemeis growing and the magnitude of the solution given by the implicit Euler schemedecreases with time. The solution to this exercise with h = 0.1 is shown in Fig.1.7.

0 2 4 6 8 10 12 14 16 18 20!5

!4

!3

!2

!1

0

1

2

3

4

5

t

x(t)

Figure 1.7: Figure showing the solution to Exercise 1.7. exact solution,backward Euler method, Euler method.


Exercise 1.8:

(a) For the model problem

dx

dt= λx,

perform a stability analysis on the Crank-Nicolson method and show that itis unconditionally stable, i.e. the method is stable for all values of h.

(b) Show that the linearized Crank-Nicolson and the implicit Crank-Nicolsonmethods have the same stability characteristics.

(c) Also, show that when using the Crank-Nicolson to solve the model problem

dx

dt= Iωx,

there is no amplitude error associated with the numerical method. Show thatthe phase error has a leading order term that looks like

PE =(ωh)3

12+ . . .

Exercise 1.9: Perform stability analysis for the second order Runge-Kuttamethod on the model equation

dx

dt= λx

and show that the region of stability can be obtained by solving the followingequation

1 + λh+λ2h2

2− eiθ = 0 (1.43)

Show also that for the 4th order Runge-Kutta method, the stability region isobtained by solving

λh+λ2h2

2+λ3h3

6+λ4h4

24+ 1− eiθ = 0 (1.44)

The solutions to Eqs. (1.43) and (1.44) are shown in Fig. 1.8.

Example 1.3:For the model problem


!3 !2 !1 0 1!3

!2

!1

0

1

2

3

!R h

! I h

Figure 1.8: Figure showing the stability region of the 2nd ( ) and 4th order( ) Runge-Kutta methods. For comparison, the stability region of the Eulersscheme is shown as .

dx

dt= Iω,

The solution given by the numerical schemes can be written as

xn+1 = σxn

where

|σEuler| =√

1 + (ωh)2 (1.45)

|σRK2| =√

1 +(ωh)4

4(1.46)

|σRK4| =√

1− (ωh)6

72+

(ωh)8

576(1.47)


for the Eulers, second order Runge-Kutta (RK2) and fourth order Runge-Kutta(RK4) schemes respectively. A plot of the magnitude of σ for the various schemesis shown in Fig. 1.9. It is seen that the magnitude of σ is very close to 1 for boththe RK4 and RK2 schemes while it diverges away from 1 very quickly for the Eulerscheme. Thus, the magnitude error for the RK4 and RK2 schemes (especially theRK4) is very small.

To get an idea how small the error is, if we take ωh = 0.5, then

σEuler =√

1 + 0.52 = 1.118

σRK2 =

√1 +

0.54

4= 1.0078

σRK4 =

√1− 0.56

72+

0.58

576= 0.9998948784

These numbers might seem small but consider this. If we carry out simulations withωh=0.5, then after 100 time steps, the RK4 scheme produce a numerical solutionwhose magnitude is 98.9% of the true solution, the RK2 scheme will give you asolution that is 217% of the true solution, and the Euler’s method will be out by7000000% (yes 7 million percent) !

It is interesting to point out here that since σRK4 is given by Eq. (1.47), thestability curve for the 4th order Runge-Kutta scheme (shown in Fig. 1.8) interceptsthe (λIh axis when

λIh = ωh

=

√576

72

=√

8

= 2.8284

1.7 Systems of Ordinary Differential Equations

In many engineering problems, it is essential to solve, not just one, but a set ofordinary differential equations. This problem can be expressed in matrix-vectornotation as

d

dtx = [A] x (1.48)

1.7. SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS 25

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.95

0.96

0.97

0.98

0.99

1

1.01

1.02

1.03

1.04

1.05

! h

"

Figure 1.9: Figure showing the magnitude of σ for Euler ( ), RK2 ( ) andRK4 ( )

where the curly brackets () denotes a vector and the square brackets [] representsa matrix. x is the vector of dependent variables and [A] is usually a matrix ofconstant coefficients. Equation (1.48) can be solved using the methods describedabove. Applying the explicit Euler method

xn+1 − xn

h= [A] xn

xn+1 = xn + h [A] xn

At every time step, one would need to perform a matrix-vector multiplication inorder to obtain the values of the vector x at the next time step.

If an implicit scheme is required to solve the problem, then one would needto solve an system of linear algebraic equations at every time step. For example,applying the trapezoidal/Crank Nicolson scheme to Eq. (1.48) gives


xn+1 − xn

h=

1

2[A] xn +

1

2[A] xn+1

xn+1 − h

2[A] xn+1 = xn +

h

2[A] xn[

I − h

2[A]

]xn+1 =

[I +

h

2[A]

]xn (1.49)

Equation (1.49) can now be solved using methods for solving a system of differ-ential (e.g. LU decomposition, Gauss-Seidel etc.)

Exercise 1.10: Solve the following second order ordinary differential equation

d2x

dt2+ ω2x = 0 (1.50)

with the initial conditions

x (t = 0) = p

dx

dt(t = 0) = 0.

Outline the numerical algorithm for the Eulers and Trapezoidal method.

Multi-dimensional RK-4

In many situations, you might be required to solve a multi-dimensional system usingRK-4. The RK-4 can easily be extended for a multi-dimensional system. An exampleof using RK-4 to solve a system with 4 dependent variables is given below.

In the example below, it will be assumed that the system you are interested inis autonomous, i.e. the right hand side is not a function of time. Suppose you areasked to solve a system of ordinary differential equations that looks like

da1

dt= f1(a1, a2, a3, a4)

da2

dt= f2(a1, a2, a3, a4)

da3

dt= f3(a1, a2, a3, a4)

da4

dt= f4(a1, a2, a3, a4)

Equation (1.34) can be written in multi-dimensional form as

1.7. SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS 27

an+11 = an

1 +

(1

6k11 +

1

3(k21 + k31) +

1

6k41

)h (1.51)

an+12 = an

2 +

(1

6k12 +

1

3(k22 + k32) +

1

6k42

)h (1.52)

an+13 = an

3 +

(1

6k13 +

1

3(k23 + k33) +

1

6k43

)h (1.53)

an+14 = an

4 +

(1

6k14 +

1

3(k24 + k34) +

1

6k44

)h (1.54)

where the approximated solution to ai at time n+ 1 is denoted as an+1i . Note that

there has been a change in notation. The subscript of the variable a correspond tothe number of dependent variables. The time level is now denoted as a superscript.In order to calculate ai at various times, kij need to be calculated as follows

k11 = f1(a1, a2, a3, a4)

k12 = f2(a1, a2, a3, a4)

k13 = f3(a1, a2, a3, a4)

k14 = f4(a1, a2, a3, a4)

k21 = f1

(a1 +

h

2k11, a2 +

h

2k12, a3 +

h

2k13, a4 +

h

2k14

)k22 = f2

(a1 +

h

2k11, a2 +

h

2k12, a3 +

h

2k13, a4 +

h

2k14

)k23 = f3

(a1 +

h

2k11, a2 +

h

2k12, a3 +

h

2k13, a4 +

h

2k14

)k24 = f4

(a1 +

h

2k11, a2 +

h

2k12, a3 +

h

2k13, a4 +

h

2k14

)


k31 = f1

(a1 +

h

2k21, a2 +

h

2k22, a3 +

h

2k23, a4 +

h

2k24

)k32 = f2

(a1 +

h

2k21, a2 +

h

2k22, a3 +

h

2k23, a4 +

h

2k24

)k33 = f3

(a1 +

h

2k21, a2 +

h

2k22, a3 +

h

2k23, a4 +

h

2k24

)k34 = f4

(a1 +

h

2k21, a2 +

h

2k22, a3 +

h

2k23, a4 +

h

2k24

)

and

k41 = f1 (a1 + hk31, a2 + hk32, a3 + hk33, a4 + hk34)

k42 = f2 (a1 + hk31, a2 + hk32, a3 + hk33, a4 + hk34)

k43 = f3 (a1 + hk31, a2 + hk32, a3 + hk33, a4 + hk34)

k44 = f4 (a1 + hk31, a2 + hk32, a3 + hk33, a4 + hk34)

Generalization of RK-4 to a system of N dependent variables should be straightforward.

1.8 Runge-Kutta-Fehlberg method: Runge-Kutta

with error control

In this method, the step size, h is adjusted every time level in order to keep theerror of the numerical scheme under control. In order to see how this can be done,recall that the N ’th order Runge-Kutta scheme can be written as,

xn+1 = xn + hφ(tn, xn, h) (1.55)

the Taylor series can be written as

x(tn+1) = x(tn) + hψ(tn, x(tn)) +O(hN+1) (1.56)

where φ is a series consisting of your k values and ψ is a series consisting of derivativesof x(t). Recall that in Eq. (1.55), the expression for φ is derived by requiring thatthey are identical to ψ in Eq. (1.56). Thus, locally, at time level tn, the numericalvalue of φ and ψ must be the same. A higher order Runge-Kutta scheme can bewritten as

1.8. RUNGE-KUTTA-FEHLBERG METHOD: RUNGE-KUTTA WITH ERROR CONTROL29

xn+1 = xn + hφ(tn, xn, h) (1.57)

and it is obtained from a higher order Taylor series

x(tn+1) = x(tn) + hψ(tn, x(tn)) +O(hN+2) (1.58)

Let’s assume that at time level, tn, there is hardly any error in the numerical solu-tions, i.e. xn ≈ xn ≈ x(tn). Rearranging Eq. (1.56), one obtains an expression forthe local truncation error,

τn+1(h) =x(tn+1)− x(tn)

h− ψ(tn, x(tn))

=x(tn+1)− xn

h− ψ(tn, x(tn))

=x(tn+1)− (xn + hψ(tn, x(tn)))

h.

Note that τn+1(h) is O(hN). Since the Runge-Kutta scheme requires that the nu-merical value of ψ = φ, we can continue the above as

τn+1(h) =x(tn+1)− (xn + hφ(tn, xn))

h

=1

h(x(tn+1)− xn+1) (1.59)

Similarly, starting from Eq. (1.58), one can obtain an expression for the local trun-cation

τn+1(h) =1

h(x(tn+1)− xn+1) (1.60)

which is O(hN+1). Going back to Eq. (1.59) and using Eq. (1.60)

τn+1(h) =1

h(x(tn+1)− xn+1)

=1

h([x(tn+1)− xn+1] + [xn+1 − xn+1])

= τn+1(h) +1

h[xn+1 − xn+1]

Recall that τn+1(h) is O(hN) and τn+1(h) is O(hN+1). Thus, if h is small, τn+1(h)can be simply approximated as

τn+1(h) ≈1

h(xn+1 − xn+1) (1.61)


Let’s now see how this information can be used to control the local truncation error.Since τn+1(h) is O(hN), one can write

τn+1(h) = KhN (1.62)

If we increase or decrease the time step h by a factor of r, then the local truncationerror would be τn+1(rh) = K(rh)N = rNKhN = rNτn+1(h) ≈ rN

h(xn+1 − xn+1),

using Eq. (1.61). Thus if we want to bound the local truncation error to a smallvalue ε, then

r ≤(

εh

xn+1 − xn+1

)1/N

(1.63)

In practice, one usually sets

r =

( ε2h

xn+1 − xn+1

)1/N

(1.64)

One popular method to implement the above algorithm is called the Runge-Kutta-Fehlberg method. In this method xn+1 and xn+1 are approximated as

xn+1 = xn + h

(25

216k1 +

1408

2565k3 +

2197

4104k4 −

1

5k5

)(1.65)

xn+1 = xn + h

(16

135k1 +

6656

12825k3 +

28561

56430k4 −

9

50k5 +

2

55k6

)(1.66)

where

k1 = f(tn, xn)

k2 = f

(tn +

h

4, xn +

h

4k1

)k3 = f

(tn +

3h

8, xn +

3h

32k1 +

9h

32k2

)k4 = f

(tn +

12h

13, xn +

1932h

2197k1 −

7200h

2197k2 +

7296h

2197k3

)k5 = f

(tn + h, xn +

439h

216k1 − 8hk2 +

3680h

513k3 −

845h

4104k4

)k6 = f

(tn +

h

2, xn −

8h

27k1 + 2hk2 −

3544h

2565k3 +

1859h

4104k4 −

11h

40k5

)

It can be shown that the global error associated with xn+1 is O(h4) and the globalerror associated with xn+1 is O(h5). So N = 4 and r is calculated as

1.8. RUNGE-KUTTA-FEHLBERG METHOD: RUNGE-KUTTA WITH ERROR CONTROL31

r = 0.84

(εh

xn+1 − xn+1

)1/4

(1.67)

Recall that the error at time level n+1 is approximated as |xn+1−xn+1| and assumingthat there is no error at time level n, i.e. x(tn) ≈ xn ≈ xn. So using Eqs. (1.65)and (1.66), the error at time level n+ 1 is approximated as

ε = |xn+1 − xn+1|

=

∣∣∣∣ 1

360k1 −

128

4275k3 −

2197

75240k4 +

1

50k5 +

2

55k6

∣∣∣∣The main MATLAB code to implement the Runge-Kutta-Fehlberg method wouldlook something like that shown in Matlab Program 1.1

Matlab Program 1.1:

%

% Main matlab loop to implement the Runge-Kutta Fehlberg method

%

while FLAG==1

K1=h*TestFunction(t(n),x(n));

K2=h*TestFunction(t(n)+(1/4)*h,x(n)+(1/4)*K1);

K3=h*TestFunction(t(n)+(3/8)*h,x(n)+(3/32)*K1+(9/32)*K2);

K4=h*TestFunction(t(n)+(12/13)*h,x(n)+(1932/2197)*K1

-(7200/2197)*K2+(7296/2197)*K3);

K5=h*TestFunction(t(n)+h,x(n)+(439/216)*K1

-8*K2+(3680/513)*K3-(845/4104)*K4);

K6=h*TestFunction(t(n)+(1/2)*h,x(n)-(8/27)*K1

+2*K2-(3544/2565)*K3+(1859/4104)*K4-(11/40)*K5);

R=(1/h)*abs((1/360)*K1-(128/4275)*K3

-(2197/75240)*K4+(1/50)*K5+(2/55)*K6);

if R<TOL

t(n+1)=t(n)+h;

x(n+1)=x(n)+(25/216)*K1+(1408/2565)*K3+(2197/4104)*K4-(1/5)*K5;

n=n+1;

end

delta=0.84*(TOL/R)^(0.25);


if delta <0.1

h=0.1*h

elseif delta >4.0

h=4.0*h;

else

h=delta*h;

end

if h > hmax

h=hmax

end

if t(n)>=tmax

FLAG=0;

elseif t(n)+h>tmax

h=tmax-t(n);

elseif h<hmin

%time step is too small. exiting program....

FLAG=0;

end

end

1.9 Boundary Value problem

The methods introduced above can only be used to solve initial value problems.Meaning that all the information you are given is at time t = 0, say, and you areasked to predict the solution at a later point in time, say t = tf . What if you aregiven some information at t = tf? This kind of problems are called boundary valueproblems. Can the methods above be used to solve boundary value problems? Itturns out that it is possible to use the methods introduced in the preceding chaptersto solve boundary value problems. These family of methods are called shootingmethods and in general they guess the information at t = 0 in order to give therequired conditions at t = tf . The method will be introduced by considering asystem of 3 ODEs. Generalization to a system of N ODEs is straightforward.

Consider the system of 3 ordinary differential equations.

1.9. BOUNDARY VALUE PROBLEM 33

da

dt= fa(a, b, c)

db

dt= fb(a, b, c)

dc

dt= fc(a, b, c) (1.68)

You are given the following conditions

a(0) = 0, b(0) = 0 and c(tf ) = P (1.69)

You are asked to find a(t), b(t) and c(t). It is important to note that c(0) is notdefined and for the purpose of the following discussion, let c(0) = α. The idea behindthe shooting methods is that because c(0) is not defined, we are free to choose anyvalue for α that will give us c(tf ) = P . But we do not know before hand whatvalue of α will give you c(tf ) = P . So we need iterate through different values of αuntil the computed value of c(tf ) = P . As you can imagine, this is a very inefficientmethod. There are more intelligent methods in choosing the value of α that willgive you c(tf ) = P . This is described below.

Using any numerical scheme for ODEs, Eq. (1.68) can be solved with the fol-lowing initial conditions

a(0) = 0, b(0) = 0, c(0) = α (1.70)

Let’s say we take N steps of step size h to approach t = tf , then the approximatevalue of c(tf ) can be denoted as cN . The value of cN is dependent on the value ofα. Since we are only guessing the value of α, it is very likely that cN − P 6= 0. Forthe following analysis, it will be convenient to define a function

g(α) = cN(α)− P. (1.71)

In order for the numerical solution to satisfy the original boundary conditions definedin Eq. (1.69), we must ensure that

g(α) = 0.

Thus this becomes a root finding problem, i.e we have to iteratively find the valueof α such that g(α) = 0. Each value α will give you a numerical solution. Onlythe numerical solution computed with the value of α that ensures that g(α) = 0 isthe correct solution to the original problem. Since the problem has been recast as aroot finding problem, the Secant formula is usually used to provide a better guessvalue of α,

αi+1 = αi −(αi − αi−1)g(αi)

g(αi)− g(αi−1)(1.72)


that will make g(αi) = 0.

Example 1.4: You are given the following ordinary differential equation

d2x

dt2= −4 cos(x) (1.73)

and the boundary conditions x(0) = 0 and x(1) = 5. You are asked to find x(t)using numerical methods. Firstly, rewrite Eq. (1.73) as a set of 2 ODEs. Set

a = x

b =dx

dt

da

dt= b

db

dt= −4 cos(a) (1.74)

We are given that a(0) = x(0) = 0. However we do not know the value of dx/dt(0) =b(0). Instead of defining a condition for b(0) = dx/dt(t = 0) we are given thecondition at t = 1, i.e. a(1) = 5. So we are free to choose b(0) = α. Numericalsolution computed using different values of α will give you different values of aN

(for simplicity, I have just used Euler’s method to compute the numerical solution.But you can use any method you like). We would like to pick only the numericalsolution that will give us aN = 5. Let’s define a function

g(α) = aN(α)− 5.

So our task now is to find the value of α such that g(α) = 0.Let’s just guess α0 = 0 and α1 = 1. For these values of α, numerical solution for

Eq. (1.74) could be computed and g(α0) = −6.7733 and g(α1) = −5.9639. One canthen use Eq. (1.72) to compute α2. This process is then repeated until g(αi) ≈ 0.The table below shows numerical values of αi and g(αi) for each iteration.

i αi g(αi)

0 0 -6.77331 1 -5.96392 8.3682 3.25753 5.7654 0.66664 5.0957 -0.14245 5.2136 0.00856 5.2069 0.0001

The numerical solution computed using different values of αi is shown in Fig. 1.10.Note that the αi are chosen according to Eq. (1.72) and it forces the numerical

1.9. BOUNDARY VALUE PROBLEM 35

solution of x at t = 1 to approach 5. The numerical solution computed using α5

and α6 are not distinguishable on the scale of the diagram.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1!2

0

2

4

6

8

10

t

x(t)

!0

!1

!2

!3

!4

Figure 1.10: Figure showing the solution to Example 1.4. The numerical solutioncomputed using α0, α1, α2, α3, α4 are indicated on the diagram.

Chapter 2

Fourier series

Any periodic function can be, f(x) can expressed as a complex Fourier series withcoefficients ck. To prove this, recall that in Engineering Analysis, any periodicfunction can be expressed as a sin and cosine series

f (x) = a0 +∞∑l=1

[al cos (lk0x) + bl sin (lk0x)], (2.1)

where

k0 =2π

L,

L is the length of the domain and k0 is the fundamental wavenumber. The term

lk0 = kl

is the fundamental harmonics. The coefficients, al and bl in Eq. (2.1) can be foundby evaluating the following integrals

a0 =1

L

∫ L

0

f (x) dx

al =2

L

∫ L

0

f (x) cos (lk0x) dx

bl =2

L

∫ L

0

f (x) sin (lk0x) dx

Note that

a−l = al

b−l = −bl(2.2)

37

38 CHAPTER 2. FOURIER SERIES

Recall that sin and cos functions can be expressed in terms of complex exponentials

sin ρ =eIρ − e−Iρ

2i(2.3)

cos ρ =eIρ + e−Iρ

2(2.4)

Substituting Eqs. (2.3) and (2.4) into Eq. (2.1) gives

f (x) = a0 +∞∑l=1

al

(eIlk0x + e−Ilk0x

2

)+ bl

(eIlk0x − e−Ilk0x

2I

)= a0 +

∞∑l=1

eIlk0x

(al

2+bl2I

)+ e−Ilk0x

(al

2− bl

2I

)= a0 +

∞∑l=1

(al − Ibl

2

)eIlk0x +

(al + Ibl

2

)e−Ilk0x

Define cl to be

c0 = a0

cl =al − Ibl

2

c−l =a−l − Ib−l

2=al + Ibl

2

Substituting the definition for c into the expression for f(x) gives

f (x) = c0 +∞∑l=1

cleIlk0x +

∞∑l=1

c−le−Ilk0x

=∞∑l=0

cleIlk0x +

∞∑l=1

c−leI(−l)k0x

Observe that

∞∑l=1

c−leI(−l)k0x =

−∞∑l=−1

cleIlk0x

Thus

39

f (x) =∞∑l=0

cleIlk0x +

−∞∑l=−1

cleIlk0x

=∞∑

l=−∞

cleIlk0x (2.5)

(2.6)

f(x) =∞∑

l=−∞

cleIlk0x (2.7)

Exercise 2.1: Use the definition of al and bl above and show that

cl =al − Ibl

2

=1

L

∫ L

0

f (x) e−Ilk0xdx (2.8)

Solution to Exercise 2.1:

cl =al − Ibl

2

=1

2

[2

L

∫ L

0

f (x) cos (lk0x) dx− 2I

L

∫ L

0

f (x) sin (lk0x) dx

]=

1

L

∫ L

0

f (x) cos (lk0x) dx− I

L

∫ L

0

f (x) sin (lk0x) dx

=1

L

∫ L

0

f (x)

(eIlk0x + e−Ilk0x

2

)dx

− i

L

∫ L

0

f (x)

(eIlk0x − e−Ilk0x

2I

)dx

cl =1

L

∫ L

0

f (x) e−Ilk0xdx

We usually cannot sum to infinity - problem with Eq. (2.7) In general you cannotdo the integral specified in (2.8) exactly. The Nyquist critical wavenumber i.e. thelargest wavenumber that is resolvable by an equally sampled data in physical spaceis

N

2k0 (2.9)


Partial solution to these problems is to introduce the discrete Fourier Transform(DFT). In engineering, functions are often represented by finite set of discrete equallyspaced values. Additionally, data is often collected in or converted to such a discreteformat. The discrete analog to Eq. (2.7) is given by

f (xj) =1

N

N2∑

l=−N2

+1

cle+Ilk0xj (2.10)

where xj = jL/N and L is the length of the domain.The above equation can be used to find cl since it can easily be noted that Eq.

(2.10) is a linear equation which consist of N unknowns, c−N/2+1, .., cN/2. We canfind all the cl’s by solving the linear system of equations. However, it is much moreefficient to use the orthogonality property of a Fourier series to find the cl’s.

Exercise 2.2:Show the following orthogonality relationship

N−1∑j=0

eIk0(l−m)xj =

N if l −m = pN where p = 0,±1,±2, ...0 otherwise

(2.11)

If we multiply both sides of Eq. (2.10) by e−Ik0mxj and summing from j = 0 toN − 1 gives

cl =N−1∑j=0

f (xj) e−Ilk0xj (2.12)

2.1 Some properties of the Fourier coefficients

• If f(x) is a real (no imaginary part) function, then cl = c∗−l. The superscript∗ denote complex conjugation.

Proof:

cl =N−1∑j=0

f (xj) cos (klxj) + If (klxj) sin (klxj)

c−l =N−1∑j=0

f (xj) cos (klxj)− If (klxj) sin (klxj)

• If f(x) is a real function, then both c0 and cN/2 are real numbers

• If f(x) is a real function, then cN−l = c−l.

2.1. SOME PROPERTIES OF THE FOURIER COEFFICIENTS 41

Proof:

cl =N−1∑j=0

f (xi) e−Ilk0xj ,

c−l =N−1∑j=0

f (xi) eIlk0xj

cN−l =N−1∑j=0

f (xi) e−I(N−l)k0xj

=N−1∑j=0

f (xi) e−INk0xjeIlk0xj

where

xj = j∆

= jL

N(2.13)

e−iNk0xj = e−IN( 2πL )(Lj

N )

= e−I2πj

= cos (2πj)︸︷︷︸=1

+ I sin (2πj)︸︷︷︸=0

= 1

Hence,

cN−l = c−l (2.14)

Exercise 2.3: Show that

f (xj) =1

N

N2∑

l=−N2

+1

cle+Ilk0xj

=1

N

N−1∑l=0

cleIlk0xj


Solution to Exercise 2.3:

f (xj) =1

N

N2∑

l=−N2

+1

cle+Ik0lxj

=1

N

N2∑

l=0

cleIlk0xj +

−1∑l=−N

2+1

cleIlk0xj

=

1

N

N2∑

l=0

cleIlk0xj +

N2−1∑

l=1

c−le−Ilk0xj

But we know that c−l = cN−l. So

f (xj) =1

N

N2∑

l=0

cleIlk0xj +

N2−1∑

l=1

cN−leI(N−l)k0xj

f (xj) =

1

N

N−1∑l=0

cleIlk0xj

Note that

eI(N−l)k0xj = eINk0xje−Ilk0xj

= eINk0(Lj/N)e−Ilk0xj

= e−Ilk0xj

(2.15)

Thus the discrete Fourier transform pair can be written as

f (xj) =1

N

N−1∑l=0

cleIlk0xj (2.16)

cl =N−1∑j=0

f (xj) e−Ilk0xj (2.17)

• If f(x) is an even function, cl’s are real numbers

• If f(x) is an odd function, cl’s are imaginary numbers

• If the Fourier transform if f(x) is cl, then the Fourier transform of f(x+ t) iscle

Iklt

2.1. SOME PROPERTIES OF THE FOURIER COEFFICIENTS 43

Proof:

f (x) =1

N

N/2∑l=−N/2+1

cleIklx

f (x+ t) =1

N

N/2∑l=−N/2+1

cleIkl(x+t)

=1

N

N/2∑l=−N/2+1

cleIklteIklx

=1

N

N−1∑k=0

bleIklx

where bl = cleIklt is the Fourier image of f(x+ t).

Exercise 2.4:Show that if f(x) is a real function, then

f (xj) =

N2∑

l=−N2

+1

cle+Iklxj

= p0 +2

N

N2∑

l=0

(pl cos(klxj)− ql sin(klxj))

where kl = lk0 and pl and ql are the real and imaginary parts of cl respectively.

If we only consider situations where f is defined in the domain

0 < x < 2π

i.e. L = 2π,

k0 = 2π/L

= 2π/2π

= 1

(2.18)

With this assumption, the discrete Fourier series


f (xj) =1

N

N2∑

l=−N2

+1

cle+Ilxj (2.19)

cl =N−1∑j=0

f (xj) e−Ilxj (2.20)

Exercise 2.5: Express the solution of the convection diffusion equation

∂f

∂t= −U ∂f

∂x+ µ

∂2f

∂x2(2.21)

in terms of a complex Fourier series and show that the coefficients of the Fourierseries evolves as

cl(t) = cl(t = 0)eIUkle−µk2l t (2.22)

where kl = lk0.If we set U = 1, the solutions at time t = 0 and t = 20 are shown in Fig.

2.2 Aliasing

For any sample interval ∆, there is a special wave number kc called the Nyquistcritical wave number which is defined to be

kc =2π

2∆

=π

∆

For a domain L = 2π, ∆ = 2π/N . Thus

kc =π

2π/N

=N

2

kc is important for the following reasons. Any energy in wave number that existbeyond kc will be reflected to k < kc with kc as the point of reflection.

2.2. ALIASING 45

−10 −5 0 5 10 15 20 25 30−0.2

0

0.2

0.4

0.6

0.8

1

x

f(x,

t)

Figure 2.1: initial condition at time t = 0. Solution at t = 20 for µ = 0, solution at t = 20 for µ = 0.01, solution at t = 20 for µ = 0.1.

Chapter 3

Finite Difference

3.1 General Finite Difference Schemes

In many applications, one needs to find the derivatives of a function. This sectionof the notes describes a procedure that one could use to find the spatial derivativesof a function.

3.1.1 First Derivatives

We would like to express the derivative of a function, f(x), in terms the functionvalues at neighbouring points. The general explicit formula to for doing this can bewritten as

f′

i =1

∆

(M∑

j=1

a−jfi−j + a0fi +M∑

j=1

ajfi+j

)(3.1)

where f′i is the approximation to the exact derivative df/dx(x = xi) at x = xi. We

will assume for the moment that all data is equally spaced i.e.

∆ = xi+1 − xi = constant.

In Eq. (3.1)

fi±j = f(xi ± j∆)

So performing the Taylor series

f(xi±j∆) = f(xi)±(j∆)f ′(xi)+1

2!(j∆)2f ′′(xi)±

1

3!(j∆)3f ′′′(xi)+

1

4!(j∆)4f iv(xi)±HOT.

Substituting into the RHS of Eq. (3.1) gives

47

48 CHAPTER 3. FINITE DIFFERENCE

f′

i =1

∆(a−M + a−M+1 + ...+ a−1 + a0 + a1 + ...aM−1 + aM)f(xi)

+1

∆(a−M(−M) + ...+ a−1(−1) + a1 + ...+ aMM))∆

df

dx(xi)

+1

∆

(a−M

(−M)2

2!+ ...+ a−1

(−1)2

2!+ a1

12

2!+ ...+ aM

M2

2!

)∆2d

2f

dx2(xi)

+1

∆

(a−M

(−M)3

3!+ ...+ a−1

(−1)3

3!+ a1

13

3!+ ...+ aM

M3

3!

)∆3d

2f

dx2(xi)

+ O(∆4)

So if we want f′i to approximate df/dx(xi) as closely as possible, we would need to

satisfy the following equations

a−M + a−M+1 + ...+ a−1 + a0 + a1 + ...aM−1 + aM = 0

a−M(−M) + ...+ a−1(−1) + a1 + ...+ aMM = 1

a−M(−M)2

2!+ ...+ a−1

(−1)2

2!+ a1

12

2!+ ...+ aM

M2

2!= 0

a−M(−M)3

3!+ ...+ a−1

(−1)3

3!+ a1

13

3!+ ...+ aM

M3

3!= 0

... =...

The more equations we choose to satisfy, the higher the accuracy of the approxi-mation. For example, if we only satisfy the first two equations, then the accuracyof the approximation is O(∆). If we satisfy the first three equations, then theapproximation is O(∆2).

3.1.2 Some popular differencing schemes

Let’s now see what happens when we choose M = 1. The finite diffrencing can bewritten as

f′

i =1

∆(a−1f−1 + a0f0 + a1f1)

The equations that we need to satisfy in order for f′i to approximate df/dx(xi) can

be written as

a−1 + a0 + a1 = 0 (3.2)

−a−1 + a1 = 1 (3.3)

a−11

2+ a1

1

2!= 0 (3.4)

−a−11

3!+ a1

1

3!= 0 (3.5)

3.1. GENERAL FINITE DIFFERENCE SCHEMES 49

We have 3 unknowns, a−1, a0 and a1, thus we can at satisfy 3 equations at most.If we only choose to satisfy the first two equations, then we will have a numericalscheme that is first order accurate. We will also have one free variable. Let’s seta−1 = 0, then one possible solution would be

a−1 = 0

a0 = −1

a1 = 1

With this set of coefficients, the derivative of f can be approximated as

f′

i =1

∆(fi+1 − fi) (3.6)

This is called the 1st order forward difference scheme.

If we choose to set a1 = 0, then yet another possible solution would be

a−1 = 0

a0 = 1

a1 = −1


f′

i =1

∆(fi − fi−1) (3.7)

This is called the 1st order backward difference scheme.

If we now choose to satisfy Eqs. (3.2) to (3.4), then we will not have a freeparameter. The formula we get will be second order accurate and

a−1 = −1

2a0 = 0

a1 =1

2

with the corresponding formula being the approximation for f ,

f′

i =1

2∆(fi+1 − fi−1) . (3.8)

This scheme is called the 2nd order centred difference scheme.


Exercise 3.1: A five-point stencil finite difference formula can be obtained bysetting M = 2 in Eq. (3.1) to obtain

f′

i =1

∆(a−2fi−2 + a−1fi−1 + a0fi + a1fi+1 + a2fi+2)

Show that in order to obtain a fourth order accurate finite difference formula, onemust satisfy the following 5 equations

a−2 + a−1 + a0 + a1 + a2 = 0

−2a−2 − a−1 + a1 + 2a2 = 1

2a−2 +1

2a−1 +

1

2a1 + 2a2 = 0

−4

3a−2 −

1

6a−1 +

1

6a1 +

4

3a2 = 0

2

3a−2 +

1

24a−1 +

1

24a1 +

2

3a2 = 0

Solve the 5 equations above and show that

a0 = 0

a1 = −a−1 =2

3

a2 = −a−2 = − 1

12

3.1. GENERAL FINITE DIFFERENCE SCHEMES 51

Exercise 3.2: A three-point stencil, one-sided finite difference formula can bewritten as

f′

i =1

∆(a0fi + a1fi+1 + a2fi+2)

Show that in order to obtain a second order accurate finite difference formula, onemust satisfy the following 3 equations

a0 + a1 + a2 = 0

a1 + 2a2 = 11

2a1 + 2a2 = 0

Solve the 3 equations above and show that

a0 = −3

2a1 = 2

a2 = −1

2

3.1.3 Higher order derivatives

Generally, the finite difference formula for the second derivative, f′′i ≈ d2f/dx2(xi)

is expressed as

f′′

i =1

∆2

(M∑

j=1


j=1

ajfi+j

). (3.9)

Expanding the right hand side in terms of Taylor series gives

a−M + a−M+1 + ...+ a−1 + a0 + a1 + ...aM−1 + aM = 0

a−M(−M) + ...+ a−1(−1) + a1 + ...+ aMM = 0

a−M(−M)2

2!+ ...+ a−1

(−1)2

2!+ a1

12

2!+ ...+ aM

M2

2!= 1

a−M(−M)3

3!+ ...+ a−1

(−1)3

3!+ a1

13

3!+ ...+ aM

M3

3!= 0

... =...

As for the case as the first derivative, the more equations you satisfy, the moreaccurate your finite difference approximation will be.


In order to obtain a difference formula for the second derivative, consider asituation where M = 1. The equations that we need to satisfy are

a−1 + a0 + a1 = 0

−a−1 + a1 = 0

a−11

2+ a1

1

2!= 1

Solving the above equations will give you

a−1 = a1 = 1

a0 = −2

Thus a finite difference for the second derivative is given by

f′′

i =1

∆2(fi−1 − 2fi + fi+1) (3.10)

In order to find finite difference formula for higher order derivatives of order n,one would need to try the general formula

fni =

1

∆n

(M∑

j=1


j=1

ajfi+j

). (3.11)

3.1.4 Summary of finite difference formula

In the interior points, it is common to use central difference scheme. This is becausefor a given stencil, the central difference scheme always gives a higher order accuracy.Some commonly used central difference scheme

Central difference scheme Error

f′

i =1

2∆(−fi−1 + fi−1) O(∆2)

f′

i =1

12∆(fi−2 − 8fi−1 + 8fi+1 − fi+2) O(∆)4

At the end points, it is common to use one-sided difference scheme. Forward differ-ence scheme is

3.2. CENTRED DIFFERENCE SCHEMES 53

Forward difference scheme Error

f′

i =1

∆(−fi + fi+1) O(∆)

f′

i =1

2∆(−3fi + 4fi+1 − fi+2) O(∆2)

f′

i =1

6∆x(−11fi + 18fi+1 − 9fi+2 + 2fi+4) O(∆3)

One can also come up with a partially one sided scheme, for example

f′

i =1

6∆x(−2fi−1 − 3fi + 6fi+1 − fi+2) (3.12)

Backward difference scheme is

Backward difference scheme Error

f′

i =1

∆(−fi − fi−1) O(∆)

f′

i =1

2∆(3fi − 4fi−1 + fi+2) O(∆2)

f′

i =1

6∆x(−2fi−3 + 9fi−2 − 18fi−1 + 11fi) O(∆3)

One can also come up with a partially one sided scheme, for example

f′

i =1

6∆x(fi−2 − 6fi−1 + 3fi + 2fi+1) (3.13)

3.2 Centred Difference Schemes

As you have seen in the exercises above, the most accurate scheme arise when thecoefficients of the finite difference scheme (the aj s and a−js) are antisymmetrical(i.e. a−j = −aj). These finite difference schemes are called the centred differencescheme and one would start with the following formula

df

dx(xi) ≈ f

′

i =1

∆

M∑j=1

aj (fi+j − fi−j) (3.14)

where fi+j is the numerical approximation to f(xi + j∆). It is known that one canexpand f(x+ j∆) as a Taylor series


f(xi +m∆) = f(xi)

+1

1!f′(xi)(m∆)

+1

2!f′′(xi)(m∆)2

+1

3!f′′′(xi)(m∆)3

+1

4!f iv(xi)(m∆)4

+1

5!f v(xi)(m∆)5

+1

6!f vi(xi)(m∆)6

+ . . .

f(xi −m∆x) = f(xi)

− 1

1!f′(xi)(m∆)

+1

2!f′′(xi)(m∆)2

− 1

3!f′′′(xi)(m∆)3

+1

4!f iv(xi)(m∆)4

− 1

5!f v(xi)(m∆)5

+1

6!f vi(xi)(m∆)6

− . . .

Substituting into Eq (3.14) gives

3.2. CENTRED DIFFERENCE SCHEMES 55

df

dx(xi) =

2

1!f′(xi)(1a1 + 2a2 + 3a3 + . . .+MaM)(∆)

+2

2!f′′(xi)(0 + 0 + 0 + . . .+ 0)(∆)2

+2

3!f′′′(xi)(1a1 + 8a3 + 27a3 + . . .+M3aM)(∆)3

+2

4!f iv(xi)(0 + 0 + 0 + . . .+ 0)(∆)4

+2

5!f v(xi)(1a1 + 32a2 + 243a3 + . . .+M5aM)(∆)5

+2

6!f vi(xi)(0 + 0 + 0 + . . .+ 0)(∆)6

+ . . .

Thus, if we want a formula to calculate the derivative at xi, then all we have to dois to solve the following system of equations

1a1 + 2a2 + 3a3 + . . .+MaM = 1

1a1 + 8a2 + 27a3 + . . .+M3aM = 0

1a1 + 32a2 + 243a3 + . . .+M5aM = 0... =

...

The number of equations we need to use will depend on the number of coefficientsyou are using in your formula. The higher the value of M , the more accurate theanswer. The error will be of the order of (∆)2M

3.2.1 Example

Let’s suppose that we want to express

df

dx(xi) = a1(f(xi + ∆)− f(xi −∆))

Taking the steps outlined above, we would require that

2a1 = 1

thus

a1 =1

2So the formula to calculate the derivative would be

df

dx(xi) =

1

2∆(f(xi + ∆)− f(xi −∆))


3.3 Solving PDEs using finite difference schemes

Let’s now consider using the finite difference technique to solve to solve the convec-tion equation

∂f

∂t= −U ∂f

∂x(3.15)

with initial condition f(t = 0, x) = 0.5e−(ln 2)(x/3)2 . The domain is x ∈ [−20, 600].You are also given that f(t, x = −20) = 0.0. Using analytical techniques, it isstraightforward to show that the exact solution to this problem is

f(t, x) = 0.5e−(ln 2)((x−Ut)/3)2

Let’s first discretize the domain into N + 1 data points. The grid is your seriesof x values, x0, x1 ..... xN , and the corresponding value of f is denoted as f0, f1, ...., fN . For this example, we will use 2nd order central spatial discretization at theinterior nodes. So at the interior points,

df1

dt= −U f2 − f0

2∆df2

dt= −U f3 − f1

2∆df3

dt= −U f4 − f2

2∆... =

...dfN−1

dt= −U fN − fN−2

2∆

Since, the function f is only defined inside the domain, we do not have the value ofthe function at x = x−1 and x = xN+1. Thus we cannot use the central differenceformula at the end points. It is common to use a lower order scheme at the endpoints. For this example, we will use 1st order one-sided scheme at the end points toapproximate the spatial derivatives. Note that we only need to find the x derivativeat x = xN since we have been given that f0 = 0 as the boundary condition to ourproblem.

dfN

dt= −U fN − fN−1

∆

The above set of N ordinary differential equations can be put into the form

d

dtx = [A] x+ c .

For example, if N = 5, the above system will look like

3.3. SOLVING PDES USING FINITE DIFFERENCE SCHEMES 57

d

dt

f1

f2

f3

f4

f5

= −U∆

0 1/2 0 0 0

−1/2 0 1/2 0 00 −1/2 0 1/2 00 0 −1/2 0 1/20 0 0 −1 1

f1

f2

f3

f4

f5

+

−f0/2

0000

From previous lectures, we have already seen that the eigenvalues of [A] will de-termine the stability of the time-integration scheme. The eigenvalues for the aboveexample is

λ1,2 = −U∆

(0.0712± 0.8331I)

λ3,4 = −U∆

(0.2500± 0.4330I)

λ5 = −U∆

(0.3576)

where I is the imaginary unit. Remember that from our stability analysis, thestability of a system is determined by plotting λh where h is the temporal step size.Thus the stability of our system is dependent on the parameter

CFL =Uh

∆(3.16)

This parameter is called the CFL number. For stability, we would like the CFL tobe as small as possible. Note that the error of our spatial discretization is dependenton ∆. However, if we make ∆ small, then CFD becomes big and the system becomesunstable !

The stability plots for N = 5 is shown in Fig. 3.1. The eigenvalues are plottedwith CFL=1.0 on both the Euler and Runge-Kutta stability diagrams. It is easilyseen that the Euler scheme is unstable and the 4th order Runge-Kutta scheme wouldbe stable.

The solution for the convection equation with ∆ = 1, U = 1 at time, t = 400is shown in Fig. 3.2. The solution computed for h = 1 and h = 0.001 are plotted.It is clear that there are numerical oscillations in the computed solution. Theseoscillations do not go away if we decrease h. With such small value of h = 0.001,the error in the Runge-Kutta scheme is negligible. It is clear that the oscillationscome from the propogation of the error in the spatial discretisation scheme. Toconfirm this statement, a simulation was carried out with an even larger value of


−2 −1.8 −1.6 −1.4 −1.2 −1 −0.8 −0.6 −0.4 −0.2 0−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

λR

h

λ I h

−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1−3

−2

−1

0

1

2

3

λR

hλ I h

Figure 3.1: Convection equation stability plots. Euler scheme (left) and Runge-Kutta scheme (right)

x

f(x)

0 100 200 300 400 500 600-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

(a)

x

f(x)

0 100 200 300 400 500 600-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

(b)

Figure 3.2: Solution of the convection equation at time t = 400. (a) computed withh = 1.0 and (b) computed with h = 0.001. The exact solution is shown as a solidline and the computed solution is shown as a dashed line.

3.3. SOLVING PDES USING FINITE DIFFERENCE SCHEMES 59

x

f(x)

0 100 200 300 400 500 600-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

Figure 3.3: Numerical solution at t = 400 computed with ∆ = 0.1, h = 0.01. Thenumerical solution is plotted with dashed line and the exact solution is shown as asolid line. They lie on top of each other.


380 385 390 395 400 405 410 415 420−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

x

f(x)

(a)

380 385 390 395 400 405 410 415 420−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

x

f(x)

(b)

Figure 3.4: Exact solution, numerical solution. (a) is for ∆ = 1.0 and (b) iscomputed with ∆ = 0.5.

h = 0.01 and small value of ∆ = 0.1. The result at t = 400 is shown in Fig. 3.3together with the exact solution. One cannot distinguish between the two plots.

If we use the 4th order central scheme at the interior points and Kennedy andCarpenters one-sided scheme at the end points with the classical 4th order RungeKutta time step scheme to solve the convection equation given above, with h = 0.1,the results are shown in Fig 3.4. The wiggles present in Fig 3.4 (a) is due todispersion error, waves with different wave numbers travel at different speeds. Someget left behind. If we reduce ∆, then we get the exact solution.

3.4 Fourier Analysis of error

The above analysis only gives you an idea how accurate the calculation of derivativeis with respect to the grid size. This does not give all information regarding thecharacteristics of the numerical scheme. Many physical processes exhibit wave-likemotions. Hence, a Fourier analysis would provide additional information about theresolution characteristics of a particular numerical scheme.

Consider a domain, length L. Let’s assume that the depedent variable, f(x) isperiodic. Thus, f(x) can be represented as a Fourier series

f(x) =

N/2∑l=−N/2+1

cleIklx

where

3.4. FOURIER ANALYSIS OF ERROR 61

kl = k0l

=2π

Ll

kl ∈ [0, π]. L is the length of the domain of interest. Direct differentiation of f(x)gives

df

dx(x) =

N/2∑l=−N/2+1

clIkleiklx (3.17)

Note that the Fourier coefficients of the derivative of a periodic function is just theFourier coefficients of the function multiplied by Ikl.

In order to analyse the error introduced by the discretisation scheme, note that

f(x+ j∆) =N∑

l=−N+1

cleIkl(x+j∆)

f(x− j∆) =N∑

l=−N+1

cleIkl(x−j∆)

Substituting into Eq. (3.1) gives

f′

n =1

∆

M∑j=1

a−j

N/2∑l=−N/2+1

cleIkl(xn−j∆) + a0

N/2∑l=−N/2+1

cleIklxn +

M∑j=1

aj

N/2∑l=−N/2+1

cleIkl(xn+j∆)

=

1

∆

N/2∑l=−N/2+1

cl

(M∑

j=1

a−je−Iklj∆ + a0 +

M∑j=1

ajeIklj∆

)eIklxn

f′

n =

N/2∑l=−N/2+1

clIk∗l e

Iklxn (3.18)

We have defined a modified wavenumber, k∗l such that

I∆k∗l =M∑

j=1

a−je−Iklj∆ + a0 +

M∑j=1

ajeIklj∆

=M∑

j=1

(aj + a−j) cos(klj∆) + a0 + I

M∑j=1

(aj − a−j) sin(klj∆)


∆k∗l =M∑

j=1

(aj − a−j) sin(klj∆)− I

(M∑

j=1

(aj + a−j) cos(klj∆) + a0

)

If we split k∗l into its real and imaginary parts, α∗l and -β∗l respectively, then

∆α∗l =M∑

j=1

(aj − a−j) sin(klj∆)

∆β∗l =

(M∑

j=1

(aj + a−j) cos(klj∆) + a0

)

Example 3.1:Recall that the modified wavenumber can be written as k∗l = α∗l − Iβ∗l For a

second order scheme, one will have a−1 = −1/2, a0 = 0 and a1 = 1/2.

α∗l ∆ = sin(kl∆)

β∗l ∆ = 0

For a fourth order accurate scheme, a1 = −a−1 = 8/12 and a2 = −a−2 = −1/12.Thus the components of the modified wavenumber are given by

α∗l ∆ =4

3sin(kl∆)− 1

6sin(2kl∆)

β∗l ∆ = 0

Applying the finite-difference schemes to solve the convection equation

∂f

∂t= −U ∂f

∂x

Subtituting Eq. (3.18) into the right hand side of the above equation gives

∂

∂t

N/2∑l=−N/2+1

cleIklxn = −U

N/2∑l=−N/2+1

clIk∗l e

Iklxn

Comparing the coefficient of eIklxn on both sides of the equations will give

dcldt

= −UIk∗l cl

If you solve this equation using analytical techniques


cl = cl(0)e−IUα∗l te−Uβ∗l t (3.19)

If you now consider the convection diffusion equation

∂f

∂t= −U ∂f

∂x+ µ

∂2f

∂x2

Substituting the Fourier series for f gives

∂

∂t

N/2∑l=−N/2+1

cleIklxn = −U

N/2∑l=−N/2+1

clIkleIklxn − µ

N/2∑l=−N/2+1

clk2l e

Iklxn

=

N/2∑l=−N/2+1

cl(−IUkl − µk2

l

)eIklxn

Comparing the coefficient of eIklxn on both sides of the equations will give

dcldt

= −(UIkl + µk2

l

)cl

Solving the above equation using analytical techniques will give you

cl = cl(0)e−IUklte−µk2

l t (3.20)

Comparing Eqs. (3.19) with (3.20) will give you the numerical dissipation

µ∗ = Uβ∗lk2

l

(3.21)

One can also define a numerical phase speed

U∗ = Uα∗lkl

(3.22)

Example 3.2:Let’s look at the case where M = 1. Thus the finite difference scheme can be

writen as

f′

i =1

∆(a−1fi−1 + a0fi + a1fi+1)

For the central difference scheme which is 2nd order accurate, then

a−1 = −1

2a0 = 0

a1 =1

2


For these values of ai’s, the values of ∆α∗l and ∆β∗l

∆α∗l = sin(kl∆)

∆β∗l = 0.0

For the forward difference scheme which is 1st order accurate, then

a−1 = 0

a0 = −1

a1 = 1

thus


∆β∗l = cos(kl∆)− 1 (3.23)

For the backward difference scheme which is 1st order accurate, then

a−1 = −1

a0 = 1

a1 = 0

thus


∆β∗l = 1− cos(kl∆) (3.24)

Plots of ∆α∗l and ∆β∗l for the central, forward and backward differencing schemesare shown in Fig. 3.5. Note that in Fig. 3.5 (b), it is shown that the ∆β∗l for theforward differencing scheme is always negative. This indicates that if you solve

∂f

∂t= −U ∂f

∂x,

using the forward differencing scheme, then it will blow up. However, if you solve theabove equation using the backward differencing scheme, then ∆β∗l is always positiveand looking at Eqs. (3.21) and (3.19) would suggest that the solution will decaywith time. Figure 3.6 shows the numerical solution together with the exact solution.It is clear that the magnitude of the numerical solution is less than than the exactsolution which indicates a diffusion in the numerical scheme.


In a domain with L = 2π, kl = l. If at time t = 0,

f(x, t = 0) = sin(2x)

then the only nonzero value of kl is 2. The numerical phase speed for the wave is

U∗ = Usin(2∆)

2∆

where ∆ = L/(N − 1) where N is the number of grid points. The values of U∗

corresponding to N is shown in the table below

N U∗/U

11 0.756827

51 0.989506

Since U∗/U is always less than 1, then the numerical wave speed will always beslower than the true wave speed. When we use more number of grid points, thenU∗/U will approach 1 and the dispersion error will decrease to zero.

3.4.1 Fourier analysis of central differencing scheme

Substituting into the central differencing formula gives

df

dx(xi) =

1

∆

M∑j=1

aj

(N∑

l=−N+1

cl[eIkl(xi+j∆) − eIkl(xi−j∆)

])

=1

∆

M∑j=1

aj

(N∑

l=−N+1

cleIklxi

[eIkl(j∆) − e−Ikl(j∆)

])

=1

∆

M∑j=1

aj

(N∑

l=−N+1

cleIklxi2I sin(kl∆)

)

=1

∆

N∑l=−N+1

clI

(M∑

j=1

aj2 sin(kl∆)

)eIklxi

df

dx(xi) =

1

∆

N∑l=−N+1

cli

(M∑

j=1

aj2 sin(kl∆)

)eiklxi (3.25)

By comparing Eq. (3.17) with Eq. (3.25), we can see that the coefficients of theFourier expansion of have now been modified. It should be clear that


! !"# $ $"# % %"# &!

!"#

$

$"#

%

%"#

&

'()!

*+,-'()!.

-/.

! !"# $ $"# % %"# &!%

!$"#

!$

!!"#

!

'()!

01*-'()!.!$

-2.

! !"# $ $"# % %"# &!

!"#

$

$"#

%

'()!

$!01*-'()!.

-0.

Figure 3.5: (a) Plot of ∆α∗l for point the central (2nd order), forward and backward(1st order) differencing schemes. (b) Plot of ∆β∗l for the forward differencing scheme.(b) Plot of ∆β∗l for the backward differencing scheme.


0 1 2 3 4 5 6

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

x

f(x)

Figure 3.6: Simulation carried out using the upwind differencing scheme at timet = 2 with U = 1, N = 101 and the initial condition f(x, t = 0) = sin(2x).

exact solution numerical solution.


k∗l =1

∆

(M∑

j=1

aj2 sin(kl∆)

)

is effectively the wave number of the finite difference scheme. k∗l ∆ is a function ofkl∆. To understand how well the finite difference scheme approximate the exactderivative, it is informative to plot k∗l ∆ as a function of kl∆.

Exercise 3.3: Show that the real part of the non-dimensional effective wavenum-ber for both the 1st order forward scheme and the 2nd order central differencescheme is given by

α∗l ∆ = sin(kl∆). (3.26)

For the 1st backward difference scheme, the imaginary part of the non-dimensionaleffective wavenumber is

β∗l ∆ = 1− cos(kl∆). (3.27)

Also show that for the 4th order central difference scheme, the nondimensionaleffective wavenumber is

k∗l ∆ =4

3sin(kl∆)− 1

6sin(2kl∆) (3.28)

3.5 Stability analysis using the modified wavenum-

ber

In general, when the finite difference approximation is used to solve a linear partialdifferential equation, the equation of motion can be written as a set of ordinarydifferential equation

d

dtf = [A] f . (3.29)

Strictly speaking, the stability of the above system can be determined by justfinding the eigenvalues of [A]. The linear operator [A] usually contains informationregarding the boundary nodes. However, this is not usually convenient to find theeigenvalues of [A]. From past experience, it is more more likely for the interiornodes to go unstable thus there is really no need to worry about what is happeningclose to the boundary points. Thus we will confine the stability analysis assuming aperiodic solution and thus confining ourselves to the operator at the interior points.Consider the periodic function

3.5. STABILITY ANALYSIS USING THE MODIFIED WAVENUMBER 69

fj =

N/2∑l=−N/2+1

cleIklxj

The exact derivative can be written as

f′

j =

N/2∑l=−N/2+1

clIkleIklxj

The finite difference approximation to f′j can be written as

f′

j =1

2∆(fj+1 − fj−1)

If we express fj−1 and fj+1 in terms of a Fourier series, the above equation can bewritten as

f′

j =1

2∆

(∑l

cleIkl(xj+∆) −

∑l

cleIkl(xj−∆)

)

=∑

l

cl

(1

2∆

(eIkl∆ − e−Ikl∆

))eIklxj

=∑

l

clI

(1

∆sin (kl∆)

)eIklxj

=∑

l

clIk∗l e

Iklxj

where

k∗l =1

∆sin (kl∆) . (3.30)

If we now use the 2nd order finite difference scheme to solve the convection equation,

∂f

∂t= −U ∂f

∂xDiscretising using finite difference scheme gives∑

l

dcldteIklxj =

∑l

−UclIk∗l eIklxj

Comparing all the terms containing eIklxj gives

dcldt

= −UclIk∗ldcldt

= −UclI(

1

∆sin (kl∆)

)


Referring back to the model problem considered when analysing the stability ofthe ordinary differential equations (dy/dt = λy), the stability is determined by themaximum magnitude of sin(kl∆). Since 0 ≤ kl∆ ≤ π the maximum value of theright hand side of the above equation is λmax = IU/∆. The stability is determinedby λmaxh = IUh/∆. One can see that λ is purely imaginary so if we use the explicitEuler time stepping scheme, the system will always be unstable.

Exercise 3.4: Using similar analysis as above, show that

dcldt

= −UclI1

∆

(4

3sin(kl∆)− 1

6sin(2kl∆)

)(3.31)

if you use a higher order difference formula, say the 4th order central scheme

f ′j =1

12∆(fj−2 − 8fj−1 + 8fj+1 − fj+2)

to approximate the derivative (see section 3.1.4).

Equation (3.31) shows that the stability of your time stepping scheme is depen-dent on

IU1

∆

(4

3sin(kl∆)− 1

6sin(2kl∆)

).

Since the maximum value of(4

3sin(kl∆)− 1

6sin(2kl∆)

)occurs when kl∆ = 1.7974 and the value of

(43sin(1.7974)− 1

6sin(2× 1.7974)

)=

1.3722. Thus the value of λmaxh for the 4th order central scheme is

IUh

∆1.3722.

In the discussion above, it was shown that the maximum value of λmaxh for the 2ndorder central scheme is

IUh

∆.

Note that for both cases the value of λmaxh is purely imaginary. In Example 1.3 itwas shown that if λmaxh is purely imaginary, then in order to keep the 4th orderRunge-Kutta scheme stable,

λmaxh < 2.8284I

Thus if we use the discretise the spatial domain using the 2nd order central scheme,then

3.6. DISPERSION-RELATION-PRESERVING SCHEME 71

h < 2.8284∆

U.

Whereas if you discretise the spatial domain using the 4th order central scheme,then

h <2.8284

1.3722

∆

U

< 2.0612∆

U.

So we have to take a slightly smaller time step when you use a higher spatial dis-cretisation scheme. In general, this is usually the case.

3.6 Dispersion-Relation-Preserving Scheme

The Dispersion-Relation-Preserving (DRP) scheme is a numerical scheme introducedby [4] to minimize the dispersion error in the numerical scheme. For a 7 stencilscheme, 4th order accuracy, the central differencing scheme requires that

2a1 + 4a2 + 6a3 = 11

3a1 +

8

3a2 + 9a3 = 0

The above eqns can be solved to obtain in terms of a1.

a2 = −4

5a1 +

9

20(3.32)

a3 =1

5a1 −

2

15(3.33)

(3.34)

In order to ensure that the derivatives have better dispersion characteristics, a1 willbe chosen so as to minimize the integrated error, E defined as

E =

∫ π/2

−π/2

(iκ− i2a1 sin(κ)− i2a2 sin(2κ)− i2a3 sin(3κ))2 dκ

=

∫ π/2

−π/2

(iκ− i2a1 sin(κ)− i

(−4

5a1 +

9

20

)sin(2κ)− i2

(1

5a1 −

2

15

)sin(3κ)

)2

dκ


Note that E is only a function of a1. In order to minimize a1, it is required that

dE

da1

= 0 (3.35)

this will give you

−3584

375a1 −

992

1125+

2

75π +

84

25a1 π = 0

a1 = − 1

42

−496 + 15 π

45π − 128a1 = 0.799266426974

Putting this value of a1 into Eqs. (3.32) and (3.33) will give you

a2 = −0.18941314

a3 = 0.02651995

3.7 General Finite Difference Schemes For The

Second Derivative

The general explicit formula to calculate derivatives of a function can be written as

f′′

i =1

∆2

(M∑

j=1

b−jfi−j + b0fi +M∑

j=1

bjfi+j

)(3.36)

where f′′i is the approximation to the exact derivative d2f/dx2(x = xi) at x = xi.

Let’s do a Taylor series expansion on Eq. (3.1)

∆2f′′

i = (b−M + b−M+1 + ...+ b−1 + b0 + b1 + ...bM−1 + bM) f(xi)

+ (b−M(−M) + ...+ b−1(−1) + b1 + ...+ bMM))∆df

dx(xi)

+

(b−M

(−M)2

2!+ ...+ b−1

(−1)2

2!+ b1

12

2!+ ...+ bM

M2

2!

)∆2d

2f

dx2(xi)

+

(b−M

(−M)3

3!+ ...+ b−1

(−1)3

3!+ b1

13

3!+ ...+ bM

M3

3!

)∆3d

2f

dx2(xi)

+ O(∆4)

So if we want f′′i to approximate d2f/dx2(xi) as closely as possible, we would need

to satisfy the following equations

3.7. GENERAL FINITE DIFFERENCE SCHEMES FOR THE SECOND DERIVATIVE73

b−M + b−M+1 + ...+ b−1 + b0 + b1 + ...bM−1 + bM = 0

b−M(−M) + ...+ b−1(−1) + b1 + ...+ bMM = 0

b−M(−M)2

2!+ ...+ b−1

(−1)2

2!+ b1

12

2!+ ...+ aM

M2

2!− 1 = 0

b−M(−M)3

3!+ ...+ b−1

(−1)3

3!+ b1

13

3!+ ...+ aM

M3

3!= 0

... =...

The more equations we choose to satisfy, the higher the accuracy of the approxi-mation. For example, if we only satisfy the first two equations, then the accuracyof the approximation is O(∆). If we satisfy the first three equations, then theapproximation is O(∆2).

3.7.1 Some popular differencing schemes

Let’s now see what happens when we choose M = 2. The finite differencing schemfor the second derivative can be written as

f′′

i =1

∆2(b−2f−2 + b−1f−1 + b0f0 + b1f1 + b2f2)

The equations that we need to satisfy in order for f′′i to approximate d2f/dx2(xi)

can be written as

b−2 + b−1 + b0 + b1 + b2 = 0 (3.37)

−2b−2 − b−1 + b1 + 2b2 = 0 (3.38)

2b−2 +1

2b−1 +

1

2b1 + 2b2 − 1 = 0 (3.39)

−4

3b−2 −

1

6b−1 +

1

6b1 +

4

3b2 = 0 (3.40)

2

3b−2 +

1

24b−1 +

1

24b1 +

2

3b2 = 0 (3.41)

We have 5 unknowns, b−2, b−1, b0, b1 and b2. If we only choose to satisfy the firstthree equations, then we will have a numerical scheme that is first order accurate.We will also have two free variables. Let’s set b−2 = 0 and b−1 = 0, then one possibleset of solutions would be


b−2 = 0

b−1 = 0

b0 = 1

b1 = −2

b2 = 1


f′

i =1

∆2(fi − 2fi+1 + fi+2) (3.42)

This is called the 1st order forward difference scheme for the second derivative.If we choose to set b2 = 0 and b1 = 0, then yet another possible solution would

be

b−2 = 1

b−1 = −2

b0 = 1

b1 = 0

b2 = 0


f′′

i =1

∆(fi − 2fi−1 + fi−2) (3.43)

This is called the 1st order backward difference scheme for the second derivative.If we now choose to satisfy Eqs. (3.37) to (3.41), then we will not have a free

parameter. The formula we get will be second order accurate and

b−2 = − 1

12

b−1 =4

3

b0 = −5

2

b1 =4

3

b2 = − 1

12

3.8. MULTIDIMENSIONAL PROBLEMS 75

with the corresponding formula being the approximation for f ,

f′

i =1

12∆2(−fi−1 + 16fi−1 − 30fi + 16fi+1 − fi−1) . (3.44)

This scheme is called the 4th order centered difference scheme for the second deriva-tive.

Exercise 3.5:

1. Show that the second order centered difference scheme for the second deriva-tive is given by

f′′

j =1

∆2(fj−1 − 2fj + fj+1) (3.45)

where ∆ is the spacing between the grid points.

2. Show also that the modified wave number for this operator is given by

∆2k∗2l = 2 (1− cos(kl∆)) (3.46)

where kl is the wavenumber and k∗l is the modified wavenumber.

3. Using the result above, obtain the condition of stability if one were to use Eq.(3.45) along with the explicit Euler scheme to solve the diffusion equation

∂f

∂t= µ

∂2f

∂x2. (3.47)

3.8 Multidimensional problems

3.8.1 Steady problem

Let’s now see how we can use differentiation schemes to solve the Poisson’s equation

k∂2φ

∂x2+∂2φ

∂y2= Q(x, y) (3.48)

The discretize form of this equation is

φi−1,j + φi+1,j + φi,j−1 + φi,j+1 − 4φi,j =∆2Q(xi, yj)

k

If we assume that Nx = 4 and Ny = 4 number of intervals in the x and y directions,the system of equation that you will be required to solve is


i = 1 j = 1 : φ0,1 + φ2,1 + φ1,0 + φ1,2 − 4φ1,1 =∆2Q(x1, y1)

k

i = 2 j = 1 : φ1,1 + φ3,1 + φ2,0 + φ2,2 − 4φ2,1 =∆2Q(x2, y1)

k

i = 3 j = 1 : φ2,1 + φ4,1 + φ3,0 + φ3,2 − 4φ3,1 =∆2Q(x3, y1)

k

i = 1 j = 2 : φ0,2 + φ2,2 + φ1,1 + φ1,3 − 4φ1,2 =∆2Q(x1, y2)

k

i = 2 j = 2 : φ1,2 + φ3,2 + φ2,1 + φ2,3 − 4φ2,2 =∆2Q(x2, y2)

k

i = 3 j = 2 : φ2,2 + φ4,2 + φ3,1 + φ3,3 − 4φ3,2 =∆2Q(x3, y2)

k

i = 1 j = 3 : φ0,3 + φ2,3 + φ1,2 + φ1,4 − 4φ1,3 =∆2Q(x1, y3)

k

i = 2 j = 3 : φ1,3 + φ3,3 + φ2,2 + φ2,4 − 4φ2,3 =∆2Q(x2, y3)

k

i = 3 j = 3 : φ2,3 + φ4,3 + φ3,2 + φ3,4 − 4φ3,3 =∆2Q(x3, y3)

k

In matrix form, the above equations become

−4 1 0 1 0 0 0 0 01 −4 1 0 1 0 0 0 00 1 −4 1 0 1 0 0 01 0 0 −4 1 0 1 0 00 1 0 1 −4 1 0 1 00 0 1 0 1 −4 1 0 10 0 0 1 0 0 −4 1 00 0 0 0 1 0 1 −4 10 0 0 0 0 1 0 1 −4

φ1,1

φ2,1

φ3,1

φ1,2

φ2,2

φ3,2

φ1,3

φ2,3

φ3,3

=


∆2Q(x1, y1)/k − φ1,0 − φ0,1

∆2Q(x2, y1)/k − φ2,0

∆2Q(x3, y1)/k − φ3,0

∆2Q(x1, y2)/k − φ0,2

∆2Q(x2, y2)/k∆2Q(x3, y2)/k

∆2Q(x1, y3)/k − φ1,4 − φ0,3

∆2Q(x2, y3)/k − φ2,4

∆2Q(x3, y3)/k − φ3,4

(3.49)

One should then be able to solve Eq. (3.49) using solvers for linear systems. Becausethere are so many zeros on the left hand side of Eq. (3.49), it is more common touse iterative solvers to solve Eq. (3.49).

3.8.2 Unsteady problem

Consider the two-dimensional heat equation

∂φ

∂t= Γ

(∂2φ

∂x2+∂2φ

∂y2

)(3.50)

The solution at t = 20 at Γ = 0.1/1.2 and Nx = 41, Ny = 41 is shown in Fig.3.7. The boundary condition is

φ(x, y = 1, t) = 300

φ(x = 0, y, t) = 20(1− y) + 300

∂φ

∂y(x, y = 0, t) = 0

∂φ

∂x(x = 1, y, t) = 0

If we use 2nd order finite difference scheme to represent the spatial derivatives, thendiscretise form of Eq. (3.50) is

d

dtφi,j = Γ

(φi+1,j − 2φi,j + φi−1,j

∆2x

+φi,j+1 − 2φi,j + φi,j−1

∆2y

)(3.51)

where φi,j is the approximated value of φ at point (xi, yj).At the points close to the boundaries, we have to implement the boundary con-

ditions. Close to the east boundary,

φNx−1,j = φNx,j (3.52)


d

dtφNx−1,j = Γ

(−φNx−1,j + φi−1,j

∆2x

+φi,j+1 − 2φi,j + φi,j−1

∆2y

)(3.53)

The above is just a system of ODE’s that one will have to solve in order to obtaina time-dependent solution. One can use Euler, Runge-Kutta etc. to get a solutionto the set of equations. In order to carry out stability analysis, one has to put itinto matrix-vector form. Taking into account of the boundary conditions, the abovesystem of equations could be written into matrix-vector form (assuming Nx = 5and Ny = 4 number of intervals in the x and y directions respectively) as follows.

d

dt

φ1,1

φ2,1

φ3,1

φ4,1

φ1,2

φ2,2

φ3,2

φ4,2

φ1,3

φ2,3

φ3,3

φ4,3

= Γ

b α 0 0 γ 0 0 0 0 0 0 0α b α 0 0 γ 0 0 0 0 0 00 α b α 0 0 γ 0 0 0 0 00 0 α a 0 0 0 γ 0 0 0 0γ 0 0 0 β α 0 0 γ 0 0 00 γ 0 0 α β α 0 0 γ 0 00 0 γ 0 0 α β α 0 0 γ 00 0 0 γ 0 0 α a 0 0 0 γ0 0 0 0 γ 0 0 0 β α 0 00 0 0 0 0 γ 0 0 α β α 00 0 0 0 0 0 γ 0 0 α β α0 0 0 0 0 0 0 γ 0 0 α a

φ1,1

φ2,1

φ3,1

φ4,1

φ1,2

φ2,2

φ3,2

φ4,2

φ1,3

φ2,3

φ3,3

φ3,4

+Γ

φ0,1/∆2x

000

φ0,2/∆2x

000

φ0,3/∆2x + φ1,4/∆

2y

φ2,4/∆2y

φ3,4/∆2y

φ4,4/∆2y

where

α =1

∆2x

β = − 2

∆2x

− 2

∆2y

γ =1

∆2y

a = − 1

∆2x

− 2

∆2y

b = − 2

∆2x

− 1

∆2y

One can then use can be the above system to perform stability analysis.

If you want to use implicit time-stepping scheme to compute the solution to Eq.(3.51), things get a bit complicated. Let’s use the Crank-Nicholson time-steppingscheme


φ(n+1)i,j − φ

(n)i,j

h=

Γ

2

(φ

(n+1)i+1,j − 2φ

(n+1)i,j + φ

(n+1)i−1,j

∆2x

+φ

(n+1)i,j+1 − 2φ

(n+1)i,j + φ

(n+1)i,j−1

∆2y

)

+Γ

2

(φ

(n)i+1,j − 2φ

(n)i,j + φ

(n)i−1,j

∆2x

+φ

(n)i,j+1 − 2φ

(n)i,j + φ

(n)i,j−1

∆2y

)

Rearranging and putting on the unknowns (φ(n+1)i,j ) on the left hand side and all the

terms that are known (φ(n)i,j ) on the right hand side gives

−(

Γh

2∆2y

)φ

(n+1)i,j−1 −

(Γh

2∆2x

)φ

(n+1)i−1,j +

(1− Γh

∆2x

− Γh

∆2y

)φ

(n+1)i,j −

(Γh

2∆2x

)φ

(n+1)i+1,j −

(Γh

2∆2y

)φ

(n+1)i,j+1

=

(Γh

2∆2y

)φ

(n)i,j−1 +

(Γh

2∆2x

)φ

(n)i−1,j +

(1 +

Γh

∆2x

+Γh

∆2y

)φ

(n)i,j +

(Γh

2∆2x

)φ

(n)i+1,j

(Γh

2∆2y

)φ

(n)i,j+1

One can write the above system in matrix-vector form as

[A]φ(n+1)

= C

where the matrix [A] is a sparse matrix consisting of lots of zeros. The vector Cis a linear function of φ

(n)i,j and the boundary values of φ

(n+1)i,j which we know. For

Nx = 5 and Ny = 4, the matrix [A] would look similar to

b α 0 0 γ 0 0 0 0 0 0 0α b α 0 0 γ 0 0 0 0 0 00 α b α 0 0 γ 0 0 0 0 00 0 α a 0 0 0 γ 0 0 0 0γ 0 0 0 β α 0 0 γ 0 0 00 γ 0 0 α β α 0 0 γ 0 00 0 γ 0 0 α β α 0 0 γ 00 0 0 γ 0 0 α a 0 0 0 γ0 0 0 0 γ 0 0 0 β α 0 00 0 0 0 0 γ 0 0 α β α 00 0 0 0 0 0 γ 0 0 α β α0 0 0 0 0 0 0 γ 0 0 α a


where

α = − Γh

2∆2x

β = 1− Γh

∆2x

− Γh

∆2y

γ = − Γh

2∆2y

a = 1− 3Γh

2∆2x

− Γh

2∆2y

b = 1− Γh

2∆2x

− 3Γh

2∆2y

In order to solve the system of equations, one can use point Jacobi iteration. Wecan re-write

(1− Γh

∆2x

− Γh

∆2y

)φ

(n+1)i,j =

(Γh

2∆2y

)φ

(n+1)i,j−1

+

(Γh

2∆2x

)φ

(n+1)i−1,j

+

(Γh

2∆2x

)φ

(n+1)i+1,j

+

(Γh

2∆2y

)φ

(n+1)i,j+1

+

(Γh

2∆2y

)φ

(n)i,j−1

+

(Γh

2∆2x

)φ

(n)i−1,j

+

(1 +

Γh

∆2x

+Γh

∆2y

)φ

(n)i,j

+

(Γh

2∆2x

)φ

(n)i+1,j

+

(Γh

2∆2y

)φ

(n)i,j+1

The first four terms on the right hand side (i.e. terms containing φ(n+1)i−1,j , φ

(n+1)i+1,j ,

φ(n+1)i,j−1 , φ

(n+1)i,j+1 ) are unknowns, so we just have to guess their values in order to solve

for φ(n+1)i,j+1 . The iterative formula one would use to iteratively solve for φ

(i,jn+ 1) is


(1− Γh

∆2x

− Γh

∆2y

)φ

(n+1),r+1i,j =

(Γh

2∆2y

)φ

(n+1),ri,j−1

+

(Γh

2∆2x

)φ

(n+1),ri−1,j

+

(Γh

2∆2x

)φ

(n+1),ri+1,j

+

(Γh

2∆2y

)φ

(n+1),ri,j+1

+

(Γh

2∆2y

)φ

(n)i,j−1

+

(Γh

2∆2x

)φ

(n)i−1,j

+

(1 +

Γh

∆2x

+Γh

∆2y

)φ

(n)i,j

+

(Γh

2∆2x

)φ

(n)i+1,j

+

(Γh

2∆2y

)φ

(n)i,j+1

This scheme is called the point Jacobi scheme. One would use the formula above toiteratively obtain the value for φ

(n+1)i,j for the (r + 1)th iteration using the values of

φ(n+1)i−1,j , φ

(n+1)i+1,j , φ

(n+1)i,j−1 , φ

(n+1)i,j+1 ) at the rth iteration.

If you want to use the Gauss-Seidel iteration scheme, one would use the latestvalue of φ

(n),ri,j instead of the previous value φ

(n),r−1i,j .

3.8.3 Modified wavenumber stability analysis

Using eigenvalue techniques to analyse the stability of a numerical algorithm canbe tedious and not very informative. It is much quicker if you use the modifiedwavenumber in order to determine the stability of a numerical scheme.

Let’s express φi,j in terms of a two-dimensional Fourier series

φi,j =∑

l

∑m

clm(t)eIklxieIkmyj

where I is the imaginary unit. Therefore


314

310

316

307

305

302

301301

X

Y

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

Figure 3.7: Solution at time t = 20 for the 2d heat equation.

3.9. TEST PROBLEMS 83

φi−1,j =∑

l

∑m

clm(t)eIkl(xi−∆x)eIkmyj

φi+1,j =∑

l

∑m

clm(t)eIkl(xi+∆x)eIkmyj

φi,j−1 =∑

l

∑m

clm(t)eIklxieIkm(yj−∆y)

φi,j+1 =∑

l

∑m

clm(t)eIklxieIkm(yj+∆y)

Substituting the above into Eq. (3.51) and compare coefficients of eIklxieIkmyj gives

d

dtclm(t) = −2Γ

(1− cos (kl∆x)

∆2x

+1− cos (kl∆y)

∆2y

)clm(t)

= λclm(t)

In order for stability, we would like λ to fall within the stability diagram of theparticular numerical time-stepping scheme that you are using. If you are usingEuler time-stepping, then

−2 > λh > 0

The worst case scenario is when cos(kl∆x) = cos(km∆y) = −1. So if you time-stepusing the explicit Euler scheme, then

h ≤ 1

Γ

(2

∆2x

+2

∆2y

).

Exercise 3.6: If one were to use 2nd order central difference (for spatial discreti-sation) and the Euler time stepping schemes to solve the diffusion equation, whatis the maximum value of h that one can use. Use, ∆x = ∆y = ∆ = 0.01 andΓ = 0.1. Compare your 1-d and 2-d results.

3.9 Test Problems

3.10 Euler equations

Using second order central difference with 4th order Runge-Kutta time steppingwith the following parameters dt = 0.1, ∆x = ∆y = 1, Mach number, M = 0.8 andpressure forcing function


x

y

−100 −80 −60 −40 −20 0 20 40 60 80 100−100

−80

−60

−40

−20

0

20

40

60

80

100

Figure 3.8: Contour plot of the solution to the Euler Equations at time t = 400.The solution is computed using 2nd order central difference scheme and Runge-Kutta time stepping. Contour levels are at p = ±0.1,±0.05,±0.01,±0.005,±0.001.Note the dispersion error and also the reflected wave. The numerical solution wasobtained with dt = 0.1, ∆x = ∆y = 1.

sin(Ωt)e((x+20)2+y2)/9

where Ω = 0.03π. The solution at time t = 400 is shown in Fig. 3.8. Note the highwavenumber dispersion waves and the reflected waves.The dispersion error could be eliminated by conducting the simulation with moregrid points. Figure 3.9 shows the results from the same simulation but carried outwith a better spatial resolution of ∆x = ∆y = 0.5 with dt = 0.02. The contour plotshown is at t = 600. Note that no visible high wavenumber dispersive waves. Onlythe reflected waves are the source of error. The effects the reflected waves from theexternal boundaries could be lessen by computing the solution using a bigger domain.Solution computed with ∆x = ∆y = 0.5, dt = 0.01 but with −300 ≤ x ≤ 300 and−300 ≤ y ≤ 300 is shown in Fig. 3.10.

3.10. EULER EQUATIONS 85

x

y

−100 −80 −60 −40 −20 0 20 40 60 80 100−100

−80

−60

−40

−20

0

20

40

60

80

100

Figure 3.9: Contour plot of the solution to the Euler Equations at time t = 600.The solution is computed using 2nd order central difference scheme and Runge-Kutta time stepping. Contour levels equally spaced with 50 contour levels betweenthe maximum and minimum p values. The numerical solution was obtained withdt = 0.02, ∆x = ∆y = 0.5.


!300 !200 !100 0 100 200 300!300

!200

!100

0

100

200

300

Figure 3.10: Contour plot of p for the solution to the Euler Equations at timet = 400. The solution is computed using 2nd order central difference scheme andRunge-Kutta time stepping. The numerical solution was obtained with dt = 0.01,∆x = ∆y = 0.5. The solution was computed on a bigger domain, −300 ≤ x ≤ 300and −300 ≤ y ≤ 300

Chapter 4

Differentiation: Unequally spaceddata

In order to find a formula to differentiate a function, consider the Taylor series

f(xi+1) = f(xi) + f′(xi) (xi+1 − xi) +

1

2!f′′(xi) (xi+1 − xi)

2 +1

3!f′′′(ξ) (xi+1 − xi)

3

(4.1)where

xi+1 = xi + ∆i

f′(x) =

df

dx

and ξ is somewhere in between xi and xi+1. Equation (4.1) can be re-arranged toobtain

f′(xi) =

f(xi+1)− f(xi)

(xi+1 − xi)− 1

2!f′′(xi) (xi+1 − xi) +

1

3!f′′′(ξ) (xi+1 − xi)

2 (4.2)

Hence f′(x) can be approximated as

f′(xi) =

f(xi+1)− f(xi)

(xi+1 − xi)(4.3)

Equation (4.3) is called the Forward Difference Scheme (FDS). The other termsthat have been neglected in Eq. (4.2) gives an approximation of the error in ap-proximating f

′(xi) with Eq. (4.3). The leading term in the truncation error for the

FDS approximation is

EFDS =1

2!f′′(xi) (xi+1 − xi)

87

88 CHAPTER 4. DIFFERENTIATION: UNEQUALLY SPACED DATA

Thus, in order to reduce the error, one would like to make (xi+1 − xi) as small aspossible.

The Taylor series expansion Eq. (4.1) could also be used to obtain an expressionfor f(xi−1).

f(xi−1) = f(xi) + f′(xi) (xi−1 − xi)

+1

2!f′′(xi) (xi−1 − xi)

2 +1

3!f′′′(ξ2) (xi−1 − xi)

3

= f(xi)− f′(xi) (xi − xi−1)

+1

2!f′′(xi) (xi − xi−1)

2 − 1

3!f′′′(ξ2) (xi − xi−1)

3 (4.4)

where xi−1 < ξ2 < xi. Equation (4.4) can be re-arranged to obtain an anotherapproximation for f

′(xi)

f′(xi) =

f(xi)− f(xi−1)

(xi − xi−1)(4.5)

Equation (4.5) is called the Backward Difference Scheme (BDS) approximationof the first derivative. If we assume that xi−1 is very close to xi, then the leadingterm in the truncation error for the FDS approximation is

EBDS =1

2!f′′(xi) (xi − xi−1)

Exercise 4.1: Subtract Eq. (4.4) from Eq. (4.1) and show that, after somealgebra, that the first derivative can be written as

f′(xi) = f(xi+1)−f(xi−1)

xi+1−xi−1− f

′′(xi)

(xi+1−xi)2−(xi−xi−1)2

2!(xi+1−xi−1)

− f′′′

(ξ)3!

(xi+1−xi)3

(xi+1−xi−1)− f

′′′(ξ2)3!

(xi−xi−1)3

(xi+1−xi−1)

Following Ex. (4.1), yet another formula for computing f′(xi) is

f′(xi) =

f(xi+1)− f(xi−1)

xi+1 − xi−1

(4.6)

This formula is called the Central Difference Scheme (CDS) and its leading ordererror is given by

ECDS = f′′(xi)

(xi+1 − xi)2 − (xi − xi−1)

2

2! (xi+1 − xi−1)

4.1. APPROXIMATION OF THE 2ND DERIVATIVE 89

Exercise 4.2:For the case where all the xi’s are equally spaced, i.e. xi+1−xi = ∆ = const, showthat Eqs. (4.3), (4.5), and (4.6) simplifies to the following expressions for the FDS,BDS and CDS

f′(xi) =

f(xi+1)− f(xi)

∆

f′(xi) =

f(xi)− f(xi−1)

∆

f′(xi) =

f(xi+1)− f(xi−1)

2∆

Also show that the leading error term in the FDS and BDS is O(h) and the leadingerror term in the CDS is O(h2).

Example 4.1: Suppose we have a function

f(x) = e−x2

sin(10x)

This function is shown in Fig. 4.1. Note that all the ’action’ of this function isconfined to the region 0 ≤ x ≤ 3. Thus, if we want to use the difference schemes(CDS, FDS or BDS) to calculate the derivatives, we would like to to confine mostof the grid points in the region 0 ≤ x ≤ 3 and not so many grid points in the regionx > 3. One method would be to describe place the grid points where

xi+1 = xi + ∆i

∆i = r∆i−1 and r > 1. Figures 4.2 and 4.3 shows the error in calculating thederivatives using the CDS in the domain 0 ≤ x ≤ 5 using 30 grid points. The datain fig. 4.2 was computed using r = 1.0 (i.e. equally spaced grid points) and thedata in fig. 4.3 was computed using r = 1.1. As you can clearly see, you can getmuch better results by simply just making a small change in the location of the gridpoints. Numerically, the mean error for r = 1.0 using the CDS scheme is 0.5411 andfor r = 1.1 is 0.2729.

4.1 Approximation of the 2nd Derivative

A formula for the 2nd derivative can be obtained by evaluating f′(x) at points

halfway between xi+1 and xi and between xi and xi−1 (see Fig. 4.4).

d2f

dx2(xi) =

dfdx

(xi+1/2)− dfdx

(xi−1/2)(xi + 1

2(xi+1 − xi)

)−(xi−1 + 1

2(xi − xi−1)

) (4.7)

Using Eq. (4.6) to approximate the first derivative, Eq. (4.7) can be written as


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

x

f(x)

Figure 4.1: Figure showing f(x) = e−x2sin(10x).

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−1

−0.5

0

0.5

1

x

f(x)

(a)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−10

−5

0

5

10

x

df(x)/dx

(b)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

10−5

100

x

Error

(c)

Figure 4.2: Analysis of the error in the calculation for the derivatives for r = 1.0.(a) shows the function ( ) and the data points, xi ( ). The analytical derivativeof the function ( ) and the calculated derivative using CDS is shown using thesymbols in (b). (c) shows the error, indicated by the symbols, calculated at every

grid point.

d2f

dx2(xi) =

f(xi+1)−f(xi)xi+1−xi

− f(xi)−f(xi−1)xi−xi−1

12(xi+1 − xi−1)

(4.8)

4.1. APPROXIMATION OF THE 2ND DERIVATIVE 91

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−1

−0.5

0

0.5

1

x

f(x)

(a)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−10

−5

0

5

10

x

df(x)/dx

(b)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

10−5

100

x

Error

(c)

Figure 4.3: Analysis of the error in the calculation for the derivatives for r = 1.1.(a) shows the function ( ) and the data points, xi ( ). The analytical derivativeof the function ( ) and the calculated derivative using CDS is shown using thesymbols in (b). (c) shows the error, indicated by the symbols, calculated at every

grid point.

xi+1xixi-1

xi-1/2xi+1/2

l l m m

Figure 4.4: Figure showing the data points used to calculate the second derivative

Exercise 4.3: Show that Eq. (4.8) simplifies to

d2f

dx2(xi) =

f(xi+1) (xi − xi−1)− f(xi) (xi+1 − xi−1) + f(xi−1) (xi+1 − xi)12(xi+1 − xi−1) (xi+1 − xi) (xi − xi−1)

In addition, if one assumes that xi+1 − xi = xi − xi−1 = ∆, then

d2f

dx2(xi) =

f(xi+1)− 2f(xi) + f(xi−1)

∆2

Results from the above exercise illustrates that the most accurate scheme is theCDS because the error in the CDS approximation is O(∆2). This mean that if wehalve the grid, the error will reduce to a quarter its original value. For the FDS and


BDS approximations, halving h will only halve the error.In most practical engineering calculations, it is inefficient to have equally spaced

grid. Take for example, in the situation below,

EFDS =1

2f ′′(xi)∆i

EBDS =1

2f ′′(xi)∆i−1

ECDS =1

2f ′′(xi)

∆i−1

2

(∆i

∆i−1

− 1

)where ∆i = xi+1 − xi

4.2 Application of Finite Difference Formulas

In many engineering problems, you will have to solve ordinary differential equationsthat look something like

d2y

dx2+ p(x)

dy

dx+ q(x)y = r(x) (4.9)

where

a ≤ x ≤ b

We are also usually given the conditions at the boundaries

y(a) = αy(b) = β

Exercise 4.4: Using CDS approximation for all the derivatives, show that Eq.(4.9) can be approximated as(

− pi

xi+1−xi−1+ 2

(xi+1−xi−1)(xi−xi−1)

)yi−1

+(− 2

(xi+1−xi)(xi−xi−1)+ qi

)yi

+(

pi

xi+1−xi−1+ 2

(xi+1−xi−1)(xi+1−xi)

)yi+1 = ri

(4.10)

whereyi = y(xi)pi = p(xi)qi = q(xi)ri = r(xi)

4.2. APPLICATION OF FINITE DIFFERENCE FORMULAS 93

From the boundary conditions, we know that

y0 = αyN = β

(4.11)

Eq. (4.10) together with Eq. (4.11) represent a linear system that can be solved.They can be represented by

[A]X = C

where

[A] =

1 0 0 0 · · · 0χ1 ε1 η1 0 · · · 0

0 χ2 ε2 η2. . . 0

0 0. . . . . . . . .

...0 0 · · · χN−2 εN−2 ηN−1

0 0 0 0 · · · 1

X =

y0

y1.........yN

C =

αr1r2...

rN−1

β

and

χi =

(− pi

xi+1 − xi−1

+2

(xi+1 − xi−1) (xi − xi−1)

)εi =

(− 2

(xi+1 − xi) (xi − xi−1)+ qi

)ηi =

(pi

xi+1 − xi−1

+2

(xi+1 − xi−1) (xi+1 − xi)

)

Chapter 5

Galerkin Method

In this chapter, we will use Fourier series to solve a few classical partial differentialequations

5.1 Convection equation

Let’s use Fourier series to solve the wave equation

∂u

∂t− ∂u

∂x= 0 (5.1)

with initial conditions given by

u(x, 0) = sin(π cos(x))

We will also assume that the domain of interest is x ∈ [0, 2π]. Approximate thesolution as a Fourier series

u(x, t) =1

N

N/2∑k=−N/2+1

ck (t) eikx. (5.2)

Substitute Eq. (5.2) into Eq. (5.1) gives

1

N

∑k

dck (t)

dteikx − 1

N

∑k

ck(t) (ik) eikx = 0.

Use Galerkin method and multiply the equation above by (1/2π) e−ilx and inte-grate over the domain 0 < x < 2π to give

N/2∑k=−N/1+1

∫ 2π

0

dckdtei(k−l)xdx =

N/2∑k=−N/1+1

ckik

∫ 2π

0

ei(k−l)x︸︷︷︸=2πδkl

95

96 CHAPTER 5. GALERKIN METHOD

Using the orthogonality condition will give

dcl(t)

dt= ilcl(t) (5.3)

The equation above is on ordinary differential equation with t as the indepen-dent variable. The initial condition is obtained by Fourier transforming the initialcondition. The solution to Eq. (5.3) can be formally written as

ck(t) = ck(t = 0)eikt. (5.4)

Alternatively, one can use Rungge-Kutta or Euler integration scheme to numericallyapproximate ck(t). Using Euler integration scheme,

ck(t+ h)− ck(t)

h= ikck(t)

ck(t+ h) = ck(t) + hikck(t)

(5.5)

5.2 Burgers Equation

Burger’s equation is a model equation for the Navier-Stokes equation. It can bewritten as

∂u

∂t+ u

∂u

∂x= ν

∂2u

∂x2(5.6)

To find a solution to Eq. (5.6), let

u(x, t) =1

N

N/2∑k=−N/2+1

ak (t) eikx (5.7)

All terms in Eq. (5.6), must be evaluated before one can solve it. Look at thediffusion term on the right hand side first

∂u

∂x=

1

N

∑k

ak (ik) eikx

∂2u

∂x2=

1

N

N∑k

ak

(−k2

)e−ikx (5.8)

Calculation of u∂u/∂x is a problem (will elaborate later), but for the moment,it will be treated as just another periodic function and can be represented as

5.2. BURGERS EQUATION 97

u∂u

∂x=

1

N

N∑k

bk(t)eikx (5.9)

Substituting Eqs. (5.9), (5.8) and (5.7) into Eq. (5.6) gives

1

N

∑k

dak

dteikx +

1

N

∑k

bkeikx = ν

1

N

∑k

(−k2

)ake

ikx

Use Galerkin Formulation and multiply the above equation by (1/2π)e−imx gives

N−1∑k=0

dak

dt

∫ 2π

0

e−i(m−k)xdx+N−1∑k=0

bk

∫ 2π

0

e−i(m−k)xdx = νN−1∑k=0

−k2ak

∫ 2π

0

e−i(m−k)xdx

Using the orthogonality condition will give the Burgers equation in Fourier space

dam

dt+ bm = −νk2am (5.10)

Equation (5.10) is just an ordinary differential equation and one can use Rungge-Kutta and other standard ode solvers to time step it forward with time. There isanother neater way of doing things. One can rearrange Eq. (5.10) as

dam

dt+ νk2am = −bm

Multiplying both sides by an integrating factor

eR

νk2dt = eνk2t

gives

eνk2t dam

dt+ eνk2tνk2am = −bmeνk2t

d

dt

(ame

νk2t)

= −bmeνk2t

(5.11)

Using Euler integration scheme gives

am(t+ h)eνm2(t+h) − am(t)eνm2t

h= −bm(t)eνm2t

Re-arranging this equation will give us

am(t+ h) = (am(t)− bm(t)h) e−k2ν∆t (5.12)

So to use Fourier Spectral Galerkin method to solve Burger’s equation, do the fol-lowing:


(1) Fourier transform u(x, t = 0) = sin(x) to get the Fourier coefficients, ak(t = 0).

(2) Calculate Fourier coefficients for ∂u/∂x⇒ ck = −ikak

(3) Inverse transform ck to get ∂u/∂x(xi) in physical space.

(4) Calculate u(xi)∂u/∂x(xi) in physical space.

(5) Obtain bk by taking the Fourier transform of u(xi)∂u/∂x(xi). This method iscommonly known as the pseudo spectral method.

(6) Use Eq. (5.12) to get am(t+ h)

(7) Repeat steps (2) - (6) to step forward the solution in time.

5.3 Aliasing error in the calculation of the non-

linear term

Let’s see what happen when we use pseudo spectral method to calculate the Fouriercoefficients of the product of two functions, u(x) × v(x). Assume that both u(x)and v(x) are periodic functions

u (xj) =1

N

N2∑

p=−N2

+1

u (p) eipxj

v (xj) =1

N

N2∑

q=−N2

+1

ν (q) eiqxj

(5.13)

The product of u and v can then be written as

u (xj) v (xj) =1

N2

∑p

∑q

u (p) v (q) ei(p+q)xj

Taking the Fourier transform of the above gives

uv (k) =1

N2

∑j

∑p

∑q

u (p) v (q) ei(p+q−k)xj (5.14)

Using orthogonality condition

1

N

N−1∑j=0

eirxj =

1 if r = Nm, where m = 0,±1,±2, .....0 otherwise

5.3. ALIASING ERROR IN THE CALCULATION OF THE NONLINEAR TERM99

Eq. (5.14) can be written as

uv (k) =1

N

∑k=p+q

u (p) ν (q) +1

N

∑k=p+q±N

u (p) ν (q)︸︷︷︸Errorterm

The ”exact” result should be

uvExact

(k) =1

N

∑k=p+q

u (p) ν (q)

ExampleLet

u(x) = sin(3x)

v(x) = sin(5x)

Thus

u(x)v(x) = sin(3x) sin(5x) =1

2cos(2x)− 1

2cos(8x)

If N is big, this is what we will get

u (k) =

(

N2

)i if k = 3

−(

N2

)i if k = −3

0 otherwise

v (k) =

(

N2

)i if k = 5

−(

N2

)i if k = −5

0 otherwise

uv (k) =

(

N4

)i if k = ±2

−(

N4

)i if k = ±8

0 otherwise

if N = 14

u (k) =

7i if k = 3−7i if k = −30 otherwise

v (k) =

7i if k = 5−7i if k = −50 otherwise


uv (k) =

3.5 if k = ±23.5 if k = ±60 otherwise

The mode at k = ±6 is the aliasing error term of

1

N

∑k=p+q±N

u (p) v (q)

The error mainly occurs because the FFT cannot differentiate between

uv(x) =1

2cos(2x)− 1

2cos(8x)

and

uv(x) =1

2cos(2x)− 1

2cos(6x)

at the data points (see handout)

Chapter 6

Collocation Method

In the collocation method, the approximated solution to the differential equation isrequired to satisfy the differential equation at discrete collocation points. Thus it isrequired to find the derivative at the collocation points.

6.1 Matrix operator for Fourier spectral numeri-

cal differentiation

Here, we recall that the Fourier transform pair is given by

ck =N−1∑j=0

f(xj)e−ikxj (6.1)

f(xj) =1

N

N/2∑k=−N/2+1

ckeikxj (6.2)

It was earlier shown that the derivative at the grid points, xj can be written as

(Du)l =1

N

N/2∑k=−N/2+1

ckikeikxl . (6.3)

Substituting Eq. (6.1) into (6.3) gives

101

102 CHAPTER 6. COLLOCATION METHOD

(Du)l =1

N

N/2∑k=−N/2+1

N−1∑j=0

f(xj)e−ikxj ikeikxl

=1

N

N−1∑j=0

N/2∑k=−N/2+1

e−ikxj ikeikxlf(xj)

=1

N

N−1∑j=0

N/2∑k=−N/2+1

ike−ik(xl−xj)f(xj)

(6.4)

remembering that

xj =2πj

Ngives

(Du)l =1

N

N−1∑j=0

N/2∑k=−N/2+1

ike−2πik

N(l−j)f(xj). (6.5)

If we define

dl,j =1

N

N/2∑k=−N/2+1

ike−2πik

N(l−j) (6.6)

then the derivative at every grid point can be written as

(Du)l =N−1∑j=0

dljf(xj). (6.7)

Note that it is possible to show that

dl,j =

(1/2) (−1)l−j cot

[π(l−j)

N

]if l 6= j

0 if l = j

Note that Eq. (6.7) can be written in vector-matrix form as

dudx

(x = x0)dudx

(x = x0)......

dudx

(x = xN−1)

=

d0,0 d0,1 · · · · · · d0,N−1...

. . ....

.... . .

......

. . ....

dN−1,0 · · · · · · · · · dN−1,N−1

u(x = x0)u(x = x1)

...

...u(x = xN−1)

Chapter 7

Some numerical examples

In this chapter, some examples of problems and solutions written by past and presentPhD. students are presented. It is intended to give the reader the vast majority ofproblems that can be solved using the numerical methods presented in this set ofnotes.

7.1 Heating of an oil droplet in air

In this section, we will look at the problem where an oil droplet is suspended inair. This same problem has been solved by Dombrovsky and Sazhin [2] so compar-ison could be made wit their data. The partial differential equation governing thebehaviour of temperature inside the liquid, φl, is

ρlcp,l∂φl

∂t= kl

1

r2

∂

∂r

(r2∂φl

∂r

). (7.1)

where kl = 0.14W/(mK) is the thermal conductivity of oil, cp,l = 2.83 kJ/(kg K)is the specific heat capacity of the liquid and ρl = 600 kg/m3 is the density of theoil. We will assume that the oil have constant properties. Let’s also assume thatthe droplet size is 20µm so Eq. (7.1) is only valid for 0 < r < 2× 10−6. The initialtemperature of the droplet is set to be 300 K. Because of symmetry, the boundarycondition at r = 0 is

∂φl(0)

∂r= 0 (7.2)

On the gas side, the governing equation for the temperature, φg, is

ρgcp,l∂φg

∂t= kg

1

r2

∂

∂r

(r2∂φg

∂r

). (7.3)

where kg = 0.061W/(mK) is the thermal conductivity of oil, cp,g = 1.12 kJ/(kg K)is the specific heat capacity of the liquid and ρg = 23.8 kg/m3 is the density of the

103

104 CHAPTER 7. SOME NUMERICAL EXAMPLES

oil. Again, we will assume all properties are constant. Let’s set the boundary ofthe calculation to be 20 times the radius of the droplet so Eq. (7.1) is only valid for2× 10−6 < r < 4× 10−5. The initial temperature of the air is set to be 880K.

At the interface of the droplet, energy conservation requires that

kl∂φl

∂r= kg

∂φg

∂r(7.4)

Using one sided scheme to approximate the derivatives for Eq. (7.4) gives

klφs − φl,Nl−1

∆l

= kgφg,1 − φs

∆g

φs =φl,Nl−1 + kg

kl

∆l

∆gφg,1

1 + kg

kl

∆l

∆g

(7.5)

where it is assumed that the temperature on the liquid side is represented by thearray [φl,0, φl,1, ....., φl,Nl−1, φl,Nl

] and the temperature on the gas side is representedas [φg,0, φg,1, ....., φg,Ng−1, φl,Ng ]. Ng and Nl are the number of grid points (includingboundary points) in the gas and liquid side respectively. ∆l and ∆g are the grid sizein the gas and liquid side respectively. φs = φl,Nl

= φg,0 is the temperature on thesurface of the droplet.

The solution can be obtained by solving Eqs. (7.1) and (7.3) along with theboundary conditions given by Eq. (7.5) and (7.2). φg,i (i = 1..Ng) is set at 880 K.The solution can be computed using Runge-Kutta time stepping scheme using thefollowing numerical parameters ∆g = 7.7551× 10−6, ∆l = 2.2222× 10−6, h = 10−5.The solution at various times are shown in Fig. 7.1. The corresponding matlab codeis shown in Matlab Program 7.1 and 7.2

7.1. HEATING OF AN OIL DROPLET IN AIR 105

! " #

$%"!!&

'!!

&!!

(!!

)!!

*!!

+!!

,!!

-.!/!!!!!!%0

1

!

0 1 2x 10!4

300

400

500

600

700

800

900

t=0.001000 s

r

! (K

)

0 1 2x 10!4

300

400

500

600

700

800

900

t=0.002000 s

r

! (K

)

0 1 2x 10!4

300

400

500

600

700

800

900

t=0.003000 s

r

! (K

)

0 1 2x 10!4

300

400

500

600

700

800

900

t=0.004000 s

r

! (K

)

0 1 2x 10!4

300

400

500

600

700

800

900

t=0.005000 s

r

! (K

)

Figure 7.1: Numerical solution for droplet heating problem. , temperature insidethe droplet, , air temperature and • , temperature on the interface of the droplet.


Matlab Program 7.1:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% File to solve heat equation using

% central differencing scheme. The problem

% we are looking at is a droplet of fuel in air.

% The parameters are chosen so that results

% could be compared with the data published in

%

% "A parabolic temperature profile model for

% heating of droplets"

% L.A. Dombrovsky and S.S. Sazhin

% J. Heat Transfer, June 2003, vol 125, 535-537.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

clear all;

close all;

Nliquid=10;

Ngas=50;

h=1.0e-5;

nstep=500;

rhol=600 %Density of liquid in kg/m^3

rhog=23.8 %Density of gas in kg/m^3

kl=0.14 %Thermal conductivity of liquid in W/(mK)

kg=0.061 %Thermal conductivity of gas in W/(mK)

cpl=2830 %specific heat capacity of liquid in J/(kg K)

cpg=1120 %specific heat capacity of gas in J/(kg K)

nuliquid=kl/(rhol*cpl); %Thermal diffusivity

nugas=kg/(rhog*cpg); %Thermal diffusivity

L=20e-6; % Initial size of droplet in m

rliquid=linspace(0,L,Nliquid);

rgas=linspace(L,20*L,Ngas);

rliquid=rliquid’;

rgas=rgas’;

Deltaliquid=rliquid(2)-rliquid(1);

Deltagas=rgas(2)-rgas(1);

Delta=Deltaliquid;

7.1. HEATING OF AN OIL DROPLET IN AIR 107

ratio=(kg/kl)*(Deltaliquid/Deltagas);

fliquid=300*ones(size(rliquid));

fgas=880*ones(size(rgas));

fgas(1)=(fliquid(Nliquid-1)+ratio*fgas(2))/(1+ratio);

fliquid(Nliquid)=fgas(1);

%

%Plotting initial solution

%

t=0;

gg=plot(rliquid(1:Nliquid-1),fliquid(1:Nliquid-1),’ko’, rgas(2:Ngas),fgas(2:Ngas),’ks’,rgas(1),fgas(1),’ko’);

set(gg(3),’markerfacecolor’,[0 0 0],’markersize’,10);

axis([0 20e-5 250 900]);

crap=sprintf(’t=%f s’,t);

text(1.5e-4,500,crap,’fontsize’,12);

xlabel(’r’);

ylabel(’\phi’)

%

% Starting time iteration

%

for j=1:nstep

foldgas=fgas;

k1=CalcRHS(nugas,Deltagas,rgas,foldgas);

temp=foldgas+k1*(h/2);

k2=CalcRHS(nugas,Deltagas,rgas,temp);

temp=foldgas+k2*(h/2);


temp=foldgas+k3*h;


fgas=foldgas+((1./6.)*k1+(1./3.)*(k2+k3)+(1./6.)*k4)*h;

foldliquid=fliquid;

k1=CalcRHS(nuliquid,Deltaliquid,rliquid,foldliquid);


temp=foldliquid+k1*(h/2);

k2=CalcRHS(nuliquid,Deltaliquid,rliquid,temp);

temp=foldliquid+k2*(h/2);


temp=foldliquid+k3*h;


fliquid=foldliquid+((1./6.)*k1+(1./3.)*(k2+k3)+(1./6.)*k4)*h;

%Heat flux at r=L

fgas(1)=(fliquid(Nliquid-1)+ratio*fgas(2))/(1+ratio);

fliquid(Nliquid)=fgas(1);

%Symmetry condition at r=0;

fliquid(1)=fliquid(2);

t=t+h

%

% Plotting solution

%

figure(1)

gg=plot(rliquid(1:Nliquid-1),fliquid(1:Nliquid-1),’ko’, rgas(2:Ngas),fgas(2:Ngas),’ks’,rgas(1),fgas(1),’ko’);

set(gg(3),’markerfacecolor’,[0 0 0],’markersize’,10);

axis([0 20e-5 250 900]);

crap=sprintf(’t=%f s’,t);

text(1.5e-4,500,crap,’fontsize’,12);

xlabel(’r’);

ylabel(’\phi (K)’)

figure(2)

plot(t,fgas(1),’ko’);

xlabel(’t(s)’);

ylabel(’\phi_s’);

hold on

pause(0.001);

end

7.2. BLASIUS SOLUTION: CONTRIBUTED BY MR. M. GIACOBELLO 109

Matlab Program 7.2:

function d2f=CalcRHS(mu,Delta,r,f)

dr=Delta;

N=length(f);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%Initializing matrix and allocating memory

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

d2f=zeros(N,1);

d2f(1)=0;

for i=2:N-1

d2f(i)=mu*((f(i+1)-2*f(i)+f(i-1))/dr^2+(2/r(i))*(f(i+1)-f(i-1))/(2*dr));

end

d2f(N)=0;

7.2 Blasius solution:

Contributed by Mr. M. Giacobello

To derived the Blasius boundary layer equation, the starting point is the steadyincompressible Navier–Stokes and continuity equations. For a freestream flow thatis aligned with the x-axis, these are given as

u∂u

∂x+ v

∂u

∂y= −1

ρ

∂p

∂x+ ν

(∂2u

∂x2+∂2u

∂y2

), (7.6)

u∂v

∂x+ v

∂v

∂y= −1

ρ

∂p

∂y+ ν

(∂2v

∂x2+∂2v

∂y2

), (7.7)

∂u

∂x+∂v

∂y= 0. (7.8)

For a boundary layer developing on a semi-infinite flat plate, these equations aresubject to the boundary conditions

u = v = 0 for x ≥ 0, y = 0. (7.9)

The Navier-Stokes equations can be simplified by noting that within the bound-ary layer u changes from zero at y = 0 to a value close to Uo at y = δ(x). Here δ(x)is a measure of the boundary layer thickness and is defined as the distance from


the wall, to the point where u reaches 99% of the freestream velocity, Uo. In thewall-parallel direction, the rate of change of u is much more gradual. By consideringthe order of magnitude of each term in the Navier-Stokes and continuity equationsand retaining only the dominant terms, it can be shown that equation (7.6) and(7.8) reduce to

u∂u

∂x+ v

∂u

∂y= ν

∂2u

∂y2(7.10)

∂u

∂x+∂v

∂y= 0. (7.11)

These equations must be solved within the boundary layer subject to the boundaryconditions

u = v = 0 for x ≥ 0, y = 0. (7.12)

and the asymptotic conditions

u→ Uo, v → 0 as y →∞. (7.13)

To simplify the solution further, the stream function, ψ, is introduced, such that

u =∂ψ

∂y, (7.14)

v = −∂ψ∂x

. (7.15)

A dimensionless coordinate η is defined as

η = y

(Uo

νx

)1/2

, (7.16)

and a dimensionless stream function f(η) is also defined by writing

ψ = (νUox)1/2 f(η). (7.17)

In terms of f(η), equation (7.10) may be re-written as

∂3f

∂η3+

1

2f∂2f

∂η2= 0. (7.18)

The boundary conditions (7.12) become

f(0) =∂f

∂η(0) = 0, (7.19)

and the asymptotic condition (7.13) becomes

∂f

∂η(∞) = 1. (7.20)


Equation (7.18) to (7.20) comprise a boundary value problem for f(η). Due to itsnon-linearity, equation (7.18) cannot be solved in closed form and Blasius resortedto a series solution. Today it is much easier to solve this equation using a computer.Here a fourth order Runge–Kutta method is used. Runge–Kutta integration meth-ods are best suited to initial value problems. To solve this boundary value problem,it is first recast as the initial value problem

∂3f

∂η3+

1

2f∂2f

∂η2= 0, (7.21)

f(0) =∂f

∂η(0) = 0, (7.22)

∂2f

∂η2(∞) = tk. (7.23)

The value tk must then be chosen to ensure, that

lim

k →∞∂f

∂η(∞, tk) = 1. (7.24)

Here the correct value of tk is converged-on iteratively using the secant method.An initial guess of tk is made and equation (7.21) must be integrated to“infinity”.At this point the computed value of ∂f/∂η is compared to the specified boundaryvalue. If the agreement is not satisfactory another guess is made. This is known asa shooting method.

In practice it is not necessary to integrate to “infinity”. Integrating from η = 0to 20 was found to be sufficient for an accurate solution. The numerical solution forf , ∂f/∂η and ∂2f/∂η2 versus η is presented in figure 7.2. Once f(η) is obtained,u(η) and v(η) may be calculated directly from their relationship to the dimensionlessstream function. That is

u(η) = Uo∂f

∂η, (7.25)

v(η) =1

2

(νUo

x

)1/2 [∂f

∂ηη − f

]. (7.26)

u(η)/Uo and v(η)/Uo are plotted versus y/δ(x) in figure 7.3 for Rex = Uox/ν = 1.


0 1 2 3 4 5 60

0.5

1

1.5

2

2.5

!

fdf/d!d2f /d!2

Figure 7.2: Numerical solution for f , ∂f/∂η and ∂2f/∂η2 versus η.


0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

U/Uo

y/!(x)

u/Uov/Uo

Figure 7.3: Numerical solution for u(η)/Uo and v(η)/Uo versus y/δ(x) for Rex = 1.


Matlab Program 7.3:

%-------------------------------------------------------------%

% Nane: Matteo Giacobello. %

% Date: 01/07/05. %

% Solution to the Blasius Boundary Layer Profile. %

% A shooting method is used to solve the BVP as an IVP. %

% Ths secant method is used to accelerate towards the %

% best guess of f’’(0). %

%-------------------------------------------------------------%

% Note: w1 == f(y), w2 == dfdy, w3 == d2fdy2. %

%-------------------------------------------------------------%

% As Chong would say, "it works like a charm". %

%-------------------------------------------------------------%

% Check that what I’m solving is correct. %

function main

close all;

clear all;

clc;

error_tol = 1e-5;

nu = 1.0;

x = 1.0;

Uo = 1.0;

output_tag = 1; % 1 == output, 0 == no output.

N = 800; % number of grid points is N+1.

kk = 1;

M = 100; % maximum number of iterations.

a = 0.0; % domain boundries.

b = 20.0;

ya = 0.0; % boundary conditions. f(0) = 0.

dya = 0.0; % f’(0) = 0.

dyb = 1.0; % f’(infty) = 1.0

h = (b-a)/N; % grid spacing.

tkm2 = (dyb-dya)/(b-a); % first guess of d2f/dx2.


for ii = 1:N+1

eta(ii) = a + (ii-1)*h;

end

w1(1) = ya;

w2(1) = dya;

w3(1) = tkm2;

[w1,w2,w3] = RKutta(w1(1),w2(1),w3(1),a,h,N);

ykm2 = w2(N+1);

tkm1 = tkm2 +(dyb-ykm2)/(b-a); % second guess of d2f/dx2.

while (kk <= M)

w1(1) = ya;

w2(1) = dya;

w3(1) = tkm1;

[w1,w2,w3] = RKutta(w1(1),w2(1),w3(1),a,h,N);

ykm1 = w2(N+1);

tk = tkm1-(ykm1-dyb)*(tkm1-tkm2)/(ykm1-ykm2);

tkm2 = tkm1;

ykm2 = ykm1;

tkm1 = tk;

kk = kk + 1;

if(abs(w2(N+1)-dyb)<error_tol) % if error is small plot to screen.

figure;

plot(eta,w1,’-x’,eta,w2,’-o’,eta,w3,’-+’)

axis([0.0 6.0 0.0 2.5])

xlabel(’eta’);

%title(’Blasius ZPG Boundary Layer Profile’);

grid on;

legend(’f’,’df/deta’,’d^2f /deta^2’);

string = sprintf(’Convergence after %d iterations using secant method’,kk);

disp(string);

break;

elseif(kk==M)

string = sprintf(’Solution did not converge....’);


disp(string);

end

%-------------------------------------------------------------%

% return to physical space. %

% Note: u = dPsi/dx is calculated using 2nd order FD. %

% The finite difference has been checked for %

% a known Psi. %

% v = 0.5*sqrt(nuUo/x)*(eta*f’-f). %

%-------------------------------------------------------------%

for ii = 1: N+1

y(ii) = sqrt(nu*x/Uo)*eta(ii);

Psi(ii) = sqrt(nu*Uo*x)*w1(ii); % The streamfunction.

%Psi(ii) = y(ii)^3;

end

%ii = 1; % forward difference.

%u(ii) = (-3*Psi(ii) + 4.0*Psi(ii+1) - Psi(ii+2))/(2*(y(ii+1)-y(ii)));

%for ii = 2: N % central difference.

%u(ii) = (Psi(ii+1)-Psi(ii-1))/(2*(y(ii+1)-y(ii)));

%end

%ii = N+1; % backward difference.

%u(ii) = (Psi(ii-2)-4.0*Psi(ii-1)+3*Psi(ii))/(2*(y(ii)-y(ii-1)));

end

for ii = 1: N+1

u(ii) = Uo*w2(ii);

v(ii) = 0.5*sqrt(nu*Uo/x)*(eta(ii)*w2(ii)-w1(ii));

end

figure;

plot(u,y,’-x’, v,y,’-o’)

legend(’u’,’v’);

xlabel(’U’);

ylabel(’y’);

axis([0.0 1.1 0.0 10.0])

%title(’Blasius ZPG Boundary Layer Profile in Physical Coordinates’);

grid on;

%-------------------------------------------------------------%


% Finally nornalise by U_99. %

%-------------------------------------------------------------%

for ii = 1: N+1

if(u(ii)>=0.99*Uo)

y99 = y(ii)

u99 = u(ii);

break;

end

end

figure;

plot(u/Uo,y/y99,’-x’, v/Uo, y/y99,’-o’)

legend(’u/Uo’,’v/Uo’);

%title(’Normalised Blasius ZPG Boundary Layer Profile’);

xlabel(’U/Uo’);

ylabel(’y/delta(x)’);

axis([0.0 1.1 0.0 1.0])

grid on;

if (output_tag == 1)

out1 = sprintf(’f_vs_eta.plt’);

out2 = sprintf(’u_v_vs_y.plt’);

fid1 = fopen(out1,’a’);

fid2 = fopen(out2,’a’);

fprintf(fid1,’variables="eta","f", "df", "d2f"’);

fprintf(fid2,’variables="y","u", "v"’);

for ii = 1: N+1

fprintf(fid1,’%10.8g %10.8g %10.8g %10.8gn’, eta(ii), w1(ii), w2(ii), w3(ii));

fprintf(fid2,’%10.8g %10.8g %10.8g n’, y(ii)/y99, u(ii), v(ii));

end

fclose(fid1);

fclose(fid2);

end

string = sprintf(’The boundary layer thickness is approximately Delta = %g ’,y99);

disp(string);

function [w1,w2,w3] = RKutta(w1,w2,w3,a,h,N)


for ii = 1:1:N

eta(ii) = a + (ii-1)*h;

K11 = h*rhs1(eta(ii), w1(ii), w2(ii), w3(ii));



K21 = h*rhs1(eta(ii)+h/2,w1(ii)+K11/2,w2(ii)+K12/2,w3(ii)+K13/2);






K41 = h*rhs1(eta(ii)+h,w1(ii)+K31,w2(ii)+K32,w3(ii)+K33);



w1(ii+1) = w1(ii) +(K11+2*(K21+K31)+K41)/6;

w2(ii+1) = w2(ii) +(K12+2*(K22+K32)+K42)/6;

w3(ii+1) = w3(ii) +(K13+2*(K23+K33)+K43)/6;

end

function rhs1 = rhs1(x, w1, w2, w3)

rhs1 = w2;


rhs2 = w3;


rhs3 = -0.5*w1*w3;

%-------------------------------------------------------------%

Bibliography

[1] K. Atkinson. Elementary Numerical Analysis. John Wiley and Sons, 1985.

[2] L. Dombrovsky and S. Sazhin. A parabolic temperature profile model for heatingof droplets. J. Heat Transfer, 125:535–537, 2003.

[3] P. Moin. Fundamentals of Engineering Numerical Analysis. Cambridge Univer-sity Press, 2001.

[4] C. Tam, J. C. Webb, and Z. Dong. A study of the short wave components incomputational acoustics. J. Comp. Acoustics., 1:1–30, 1993.

119

advanced computational mechanics · chapter 1 ordinary diﬀerential equations in many engineering...

Documents