euler’s and heun’s methods douglas wilhelm harder, m.math. lel department of electrical and...

Euler’s and Heun’s Methods

Douglas Wilhelm Harder, M.Math. LELDepartment of Electrical and Computer Engineering

University of Waterloo

Waterloo, Ontario, Canada

ece.uwaterloo.ca

[email protected]

© 2012 by Douglas Wilhelm Harder. Some rights reserved.

2

Outline

This topic discusses numerical differentiation:– Initial-value problems– Euler’s method– Heun’s method– Multi-step methods

Euler's and Heun's Methods

3

Outcomes Based Learning Objectives

By the end of this laboratory, you will:– Understand how to approximate a solution to a 1st-order IVP

using Euler’s method– Understand the limitations of Euler’s method– Be able to apply the same ideas from the trapezoidal rule to

improve Euler’s method, i.e., Heun’s method


4

Initial-value Problems

Given the initial value problem

Invariably, initial-value problems deal with time:– We know the state y0 of a system at time t0

– We understand how the system evolves (through the ODE)– We want to approximate the state in the future

(1)

0 0

,y t f t y t

y t y


5

Ordinary Differential Equations

Your first question should be:

Can we always write a 1st-order ODE in the form:

?

For example, the ODE could be implicitly defined as:

Fortunately, the implicit function theorem says that, in almost all cases, “yes”– We may end up using a truncated approximation similar to

Taylor series

(1) ,y t f t y t

3(1) (1), , 1 sin 0F t y t y t t y t y t


6


What does the formula

mean?

Given any point (t*, y*), if a solution y(t) to the ODE passes through that point, the derivative of the solution must be:

(1) ,y t f t y t

(1) * * *,y t f t y


7


For example, the ODE

suggests, for example, at the point (1, 2), the slope is approximately

We could pick a few hundred points, determine the slopes at each of these lines, and plot that slope

(1) cosy t t y t y t t y t

1 2 2 1 cos 2 1.416146836


8


Doing this with the ODE

yields



9


The following are three solutions that satisfy these initial conditions

0 1

0 0

0 1

y

y

y


10


The ODE

was chosen because there is no explicit solution

The next example does have explicit solutions



11


Consider the ODE

This has the following field plot:

2 2(1) 1 1y t y t t


12


This clearly has y(t) = 1 as one solution; however, another solution is

2 2(1) 1 1y t y t t

3 2

3 2

3 3

3 3 3

t t ty t

t t t


13


This clearly has y(t) = 1 as one solution; however, another solution is

We can confirm this by substitution

3 2

3 2

3 3

3 3 3

t t ty t

t t t

2 2(1) 1 1y t y t t


14


Calculating the derivative:

Substituting the function into the equation

Everything cancels in the numerator except the one

23 2

23 2 3 2

9 13 3

3 3 3 3 3 3

td t t t

dt t t t t t t

23 22

3 2

2 23 2 3 2 3 2 3 22

23 2

3 31 1

3 3 3

3 3 2 3 3 3 3 3 3 3 31

3 3 3

t t tt

t t t

t t t t t t t t t t t tt

t t t

23 9

2 21 1y t t


15


Now, we see that y(1) = 1:

The slope at this point should be:

If we evaluate the calculated derivative at t = 1, we get:

3 2

3 2

1

2 6 6 11 2 6 6 11 31 1

2 6 6 17 2 6 6 17 3t

t t ty

t t t

2 2(1) 1 1,1 1 1 1 1 16y f

2

2

36 1 1 36 416

92 6 6 17


16

Euler’s Method

Now, suppose we have an initial condition:

y(t0) = y0

We want to approximate the solution at t0 + h; therefore, we can look at the Taylor series:

where

1 2 20 0 0

1

2y t h y t y t h y h

0 0,t t h


17

Euler’s Method

We can replace the initial condition

y(t0) = y0

into the Taylor series

Next, we also know what the derivative is from the ODE:

Thus,

1 2 20 0 0

1

2y t h y y t h y h

1 ,y t f t y t

2 20 0 0 0

1,

2y t h y f t y h y h


18

Euler’s Method

Thus, we have a formula for approximating the next point

together with an error term .

0 0 0 0,y t h y h f t y

2 21

2y h


19

Euler’s Method

Using our example:

we can implement both the right-hand side of the ODE and the solution:

2 2(1) 1 1

(0) 0

y t y t t

y

function [dy] = f2a(t, y) dy = (y - 1).^2 .* (t - 1).^2;end

function [y] = y2a( t ) y = (t.^3 - 3*t.^2 + 3*t)./(t.^3 - 3*t.^2 + 3*t + 3);end


20

Euler’s Method

Using our example:

we can therefore approximate y(0.1):>> approx = 0 + 0.1*f2a(0,0) actual = 0.100000000000000

>> actual = y2a(0.1) actual = 0.082849281565271

>> abs( actual - approx ) ans = 0.017150718434729

2 2(1) 1 1

(0) 0

y t y t t

y


21

Euler’s Method

Now, if we halve h, the error should drop by a factor of 4

We will therefore approximate y(0.05):>> approx = 0 + 0.05*f2a(0,0) approx = 0.050000000000000

>> actual = y2a(0.05) actual = 0.045384034047969

>> abs( actual - approx ) ans = 0.004615965952031

Previous error when h = 0.1: 0.017150718434729


22

Euler’s Method

Lets consider what we are doing:– The actual solution is in red– The two approximations are shown as circles

• We are following the same slope out from (0, 0)


23

Euler’s Method

The problem is, the second approximation does not approximate y(0.1)—it approximates the solution at the closer point t = 0.05– How can we proceed to approximate y(0.1)?


24

Euler’s Method

How about finding the slope at (0.05, 0.05) and following that out for another h = 0.05?


25

Euler’s Method

How about finding the slope at (0.05, 0.05) and following that out for another h = 0.05?>> 0.05 + 0.05*f2a(0.05, 0.05)ans = 0.090725312500000


26

Euler’s Method

We could repeat this process again, and approximate the solution at t = 0.15?>> 0.090725312500000 + 0.05*f2a( 0.1, 0.090725312500000 )ans = 0.124209921021793


27

Euler’s Method

As you can see, the three points are shadowing the actual solution


28

Euler’s Method

Note that we require more work if we reduce h:– Dividing h by 2 requires twice the work, and– Dividing h by 10 requires ten times the work

to approximate the same final point


29

Euler’s Method

In addition, we are using an approximation to approximate the next approximation, and so on…– The error for approximating one point is O(h2)– In the laboratory, you will attempt to determine how this affects

the error


30

Euler’s Method

Thus, given an IVP

and suppose we want toapproximate y(tfinal)

We could simply use

h = tfinal – t0

and find y0 + h f(t0, y0)

Problem: we have no controlover the accuracy

(1)

0 0

,y t f t y t

y t y


31

Euler’s Method

Thus, given an IVP

and suppose we want toapproximate y(tfinal)

Instead, divide the interval [t0, tfinal] into n points and now repeat Euler’s method n – 1 times

(1)

0 0

,y t f t y t

y t y


32

Euler’s Method

For example, if we chose n = 11, we would find approximations at0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 ? ? ? ? ? ? ? ? ? ?

where y(0) = 0 and we want to approximate y(1)


33

Euler’s Method

Use the initial points to approximate y(0.1):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 ? ? ? ? ? ? ? ? ?


34

Euler’s Method

Use the next two points to approximate y(0.2):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 ? ? ? ? ? ? ? ?


35

Use the next two points to approximate y(0.3):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 ? ? ? ? ? ? ?

Euler’s Method


36

Use these two points to approximate y(0.4):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 ? ? ? ? ? ?

Euler’s Method


37

Use these two points to approximate y(0.5):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 ? ? ? ? ?

Euler’s Method


38

Use these two points to approximate y(0.6):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 ? ? ? ?

Euler’s Method


39

Use these two points to approximate y(0.7):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 ? ? ?

Euler’s Method


40

Use these two points to approximate y(0.8):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 ? ?

Euler’s Method


41

Use these two points to approximate y(0.9):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902 ?

Euler’s Method


42

Finally, use these two to approximate y(1.0):0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.00000 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902 0.2907

Our approximation is y(1.0) ≈ 0.290681404577720

Euler’s Method


43

Euler’s Method

You will implement Euler’s method:function [t_out, y_out] = euler( f, t_rng, y0, n )

wheref a function handle to the bivariate function f(t, y)

t_rng a row vector of two values [t0, tfinal]

y0 the initial condition

n the number of points that we will break the interval

[t0, tfinal] into

You will return two vectors:t_out a row vector of n equally spaced values from t0 to tfinal

y_out a row vector of n values where

y_out(1) equals y0

y_out(k) approximates y(t) at t_out(k) for k from 2 to n


44

Euler’s Method

This function will:1. Determine

2. Assign toa. tout a vector of n equally spaced points going from t0 to tfinal, and

b. yout a vector of n zeros where yout, 1 is assigned the initial value y0,

3. For k going from 1 to n – 1, repeat the following:a. Using f, calculate the slope K1 at the point tout,k and yout,k, and

b. Set .

final 0

1

t th

n

out, 1 out, 1k ky y h K


45

Euler’s Method

For example, consider our initial-value problem

Approximating the solution on [0, 1] with n = 11 points yields:

>> [t2a, y2a] = euler( @f2a, [0, 1], 0, 11 ) t2a = 0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000

1.0000 y2a = 0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902

0.2907

2 2(1) 1 1

0 0

y t ty t

y


46

Euler’s Method

The function ode45 is Matlab’s built-in ODE solver:[t2a, y2a] = euler( @f2a, [0, 1], 0, 11 );plot( t2a, y2a, 'or' ); hold on[t2a, y2a] = ode45( @f2a, [0, 1], 0 );plot( t2a, y2a, 'b' )


47

Euler’s Method

The function ode45 is Matlab’s built-in ODE solver:[t2a, y2a] = euler( @f2a, [0, 1], 0, 21 );plot( t2a, y2a, 'or' ); hold on[t2a, y2a] = ode45( @f2a, [0, 1], 0 );plot( t2a, y2a, 'b' )


48

Euler’s Method



>> [t2b, y2b] = euler( @f2b, [0, 1], 1, 11 ) t2b = 0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000

1.0000 y2b = 1 1.0460 1.1000 1.1626 1.2343 1.3154 1.4059 1.5057 1.6144 1.7310

1.8543

(1) cos

0 1

y t t y t y t t y t

y


49

Euler’s Method

In this case, Euler’s method does not fare so well:hold on[t2b, y2b] = euler( @f2b, [0, 1], 1, 11 );plot( t2b, y2b, 'or' )[t2b, y2b] = ode45( @f2b, [0, 1], 1 );plot( t2b, y2b, 'b' )


50

Euler’s Method

We can increase the number of points by a factor of 10:hold on[t2b, y2b] = euler( @f2b, [0, 1], 1, 101 );plot( t2b, y2b, '.r' )[t2b, y2b] = ode45( @f2b, [0, 1], 1 );plot( t2b, y2b, 'b' )


51

Error Analysis

Now, we saw the error for Euler’s method was O(h2)– However, except with the first point, we are using an

approximation to find an approximation

– Thus, repeatedly applying Eulerresults in an error of O(h)


1

2 2

1

1

2

n

kk

E y h

1,k k kt t

1

2

12

n

kk

hy h

final

0

12 2

1

tn

kk t

y h y d

final

0

2

2

t

t

hy d

52

Improving on Euler’s Method

In the lab, you will find that, for Euler’s method:– Reducing the error by half requires twice as much effort and

memory– Reducing the error by a factor of 10 requires ten times the time

and memory

This is exceptionally inefficient and we will therefore take this lab and the next lab to see how we can improve on Euler’s method


53


Suppose you are approximating the integral of a function over an interval:

b

a

g x dx


54


One of the worst approximations would be to simply use the value of the function at one end-point:

b

a

g x dx g a b a


55


At the very least, it would be better to approximate the integral by taking the average of the two end-points:

This is the trapezoidal rule of integration

2

b

a

g a g bg x dx b a


56


When we are essentially integrating using information only at the initial value:

1 ,y t f t y t


0 0

0 0

1 ,t h t h

t t

y t dt f t y t dt

0

0

0 0 ,t h

t

y t h y t f t y t dt

0

0

0 0 ,t h

t

y t h y t f t y t dt

0 0 0,y t h f t y

57


The problem is, we would have to know the slope at t0 + h in order to approximate mimic the trapezoidal rule

Note, however, that Euler’s method gives usan approximationof y(t0 + h)

y(t0 + h) ≈ y0 + hK1

Therefore, we can approximate thethe slope at t0 + h with

1 0 0,K f t y

2 0 0 1,K f t h y h K


58


Thus, we have one slope and one approximation of a slope:

Applying the same principle as thetrapezoidal rule, we would thenapproximate

1 0 0

2 0 0 1

,

,

K f t y

K f t h y h K

1 20 0 2

K Ky t h y h


59

Heun’s Method

Graphically, Euler’s method follows the initial slope out a distance h– We calculate only one slope:

K1

1 0 0,K f t y


60

Heun’s Method

Heun’s method states that we determine the slope at the second point, too

K1

K2

2 0 0 1,K f t h y h K


61

Heun’s Method

Take the average of the two slopes and follow that new slope out a distance h:

K1 1 2

2

K K

K2

1 20 2

K Ky h


62

Heun’s Method

Thus, you will write a second function, heun(), that has the same signature as euler(), where you will1. Determine

2. Assign toa. tout a vector of n equally spaced points going from t0 to tfinal, and

b. yout a vector of n zeros where yout, 1 is assigned the initial value y0,

3. For k going from 1 to n – 1, repeat the following:a. Using f, calculate the slope K1 at the point tout,k and yout,k,

b. Use K1 to find K2, and

c. Set .

final 0

1

t th

n

1 2out, 1 out, 2k k

K Ky y h


63

Heun’s Method



[t2a, y2a] = heun( @f2a, [0, 1], 0, 11 ) t2a = 0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000

1.0000 y2a = 0 0.0828 0.1399 0.1798 0.2074 0.2262 0.2382 0.2454 0.2491 0.2505

0.2508

2 2(1) 1 1

0 0

y t ty t

y


64

Heun’s Method

The function ode45 is Matlab’s built-in ODE solver:[t2a, y2a] = heun( @f2a, [0, 1], 0, 11 );plot( t2a, y2a, 'or' ); hold on[t2a, y2a] = ode45( @f2a, [0, 1], 0 );plot( t2a, y2a, 'b' )


65

Heun’s Method



>> [t2b, y2b] = heun( @f2b, [0, 1], 1, 11 ) t2b = 0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000

1.0000 y2b = 1 1.0500 1.1091 1.1779 1.2569 1.3463 1.4462 1.5562 1.6756 1.8029

1.9362

(1) cos

0 1

y t t y t y t t y t

y


66

Heun’s Method

Heun’s method is significant better than Euler:[t2b, y2b] = heun( @f2b, [0, 1], 1, 11 );plot( t2b, y2b, 'or' ); hold on[t2b, y2b] = ode45( @f2b, [0, 1], 1 );plot( t2b, y2b, 'b' )

Euler’s Method


67

Heun’s Method

Comparing the accuracy of– Euler’s method (11 and 41 points in magenta) , and– Heun’s method (11 points in red)

We see that Heun is significantly better


68

Heun’s Method

The absolute errors are also revealing:– A reduction by a factor of three

0.02390.00705


69

0.02390.00705

Heun’s Method

To be fair, we should count function evaluations:– Euler’s method with n points has n – 1 function evaluations– Heun’s method with n points has 2(n – 1) function evaluations

Still, Heun’s method comes out ahead...


70

Error Analysis

Without proof, the error for Heun’s method is O(h3)– However, again, except with the first point, we are using an

approximation to find an approximation– As with Euler’s method, repeatedly applying Heun’s method will

results in an error of O(h2)


71

Summary

We have looked at Euler’s and Heun’s methods for approximating1st-order IVPs:– Euler’s method is a direct application of Taylor’s series– Heun’s method uses the ideas from the trapezoidal rule to

improve on Euler’s method– Heun’s method requires twice as many function evaluations as

does Euler’s method and yet it is significantly more accurate


72

References

[1] Glyn James, Modern Engineering Mathematics, 4th Ed., Prentice Hall, 2007.

[2] Glyn James, Advanced Modern Engineering Mathematics, 4th Ed., Prentice Hall, 2011.


euler’s and heun’s methods douglas wilhelm harder, m.math. lel department of electrical and...

Documents

limitations of eulers

actual solution

solution yt

order ode

y2a t y

approximate y0

end function y

state y