chapter 6 nonlinear equations - iitkhome.iitk.ac.in/~pranab/eso208/rajesh/03-04/nonlinear.pdfthe...
TRANSCRIPT
Chapter 6
Nonlinear Equations
6.1 The Problem of Nonlinear Root-finding
In this module we consider the problem of using numerical techniques to find the roots of nonlinear
equations,f(x) = 0. Initially we examine the case where the nonlinear equations are a scalar
function of a single independent variable,x. Later, we shall consider the more difficult problem
where we have a system ofn nonlinear equations inn independent variables,f(x) = 0.
Since the roots of a nonlinear equation,f(x) = 0, cannot in general be expressed in closed
form, we must use approximate methods to solve the problem. Usually, iterative techniques are
used to start from an initial approximation to a root, and produce a sequence of approximations
�x(0); x(1); x(2); : : :
;
which converge toward a root. Convergence to a root is usually possible provided that the function
f(x) is sufficiently smooth and the initial approximation is close enough to the root.
6.2 Rate of Convergence
Let
�x(0); x(1); x(2); : : :
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
70 NONLINEAR EQUATIONS
be a sequence which converges to a root�, and let�(k) = x(k) � �. If there exists a numberp and
a non-zero constantc such that
limk!1
���(k+1)��j�(k)jp = c; (6.1)
thenp is called theorder of convergenceof the sequence. Forp = 1; 2; 3 the order is said to be
linear, quadraticandcubicrespectively.
6.3 Termination Criteria
Since numerical computations are performed using floating point arithmetic, there will always be
a finite precision to which a function can be evaluated. Let� denote the limiting accuracy of
the nonlinear function near a root. This limiting accuracy of the function imposes a limit on the
accuracy,�, of the root. The accuracy to which the root may be found is given by:
� =�
jf 0(�)j : (6.2)
This is the best error bound foranyroot finding method. Note that� is large when the first derivative
of the function at the root,jf 0(�)j, is small. In this case the problem of finding the root is ill-
conditioned. This is shown graphically in Figure 6.1.
�2 small
f(x)� �
ill-conditioned
x
f(x)
well-conditioned
2�1 2�2
�2
2�
�1 largef 0(�1)
�12�
FIGURE 6.1:
When the sequence
�x(0); x(1); x(2); : : :
;
converges to the root,�, then the differences��x(k) � x(k�1)
�� will decrease until��x(k) � �
�� � �.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.4 BISECTION METHOD 71
With further iterations, rounding errors will dominate and the differences will vary irregularly.
The iterations should be terminated andx(k) be accepted as the estimate for the root when the
following two conditions are satisfied simultaneously:
1.��x(k+1) � x(k)
�� � ��x(k) � x(k�1)�� and (6.3)
2.
��x(k) � x(k�1)��
1 + jx(k)j < �: (6.4)
� is some coarse tolerance to prevent the iterations from being terminated beforex(k) is close to�,
i.e. before the step sizex(k)�x(k�1) becomes “small”. The condition (6.4) is able to test the relative
offset when��x(k)�� is much larger than1 and much smaller than1. In practice, these conditions are
used in conjunction with the condition that the number of iterations not exceed some user defined
limit.
6.4 Bisection Method
Suppose thatf (x) is continuous on an interval�x(0); x(1)
�. A root is bracketedon the interval�
x(0); x(1)�
if f�x(0)�
andf�x(1)�
have opposite sign.
Let
x(2) = x(0) +x(1) � x(0)
2(6.5)
be the mid-point of the interval�x(0); x(1)
�. Three mutually exclusive possibilities exist:
� if f�x(2)�= 0 then the root has been found;
� if f�x(2)�
has the same sign asf�x(0)�
then the root is in the interval�x(2); x(1)
�;
� if f�x(2)�
has the same sign asf�x(1)�
then the root is in the interval�x(0); x(2)
�.
In the last two cases, the size of the interval bracketing the root has decreased by a factor of two.
The next iteration is performed by evaluating the function at the mid-point of the new interval.
After k iterations the size of the interval bracketing the root has decreased to:
x(1) � x(0)
2k
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
72 NONLINEAR EQUATIONS
The process is shown graphically in Figure 6.2. The bisection method is guaranteed to converge
to a root. If the initial interval brackets more than one root, then the bisection method will find one
of them.
f(x)
x(0)
x
x(5)x(4)
x(3)x(2)
x(1)
FIGURE 6.2:
Since the interval size is reduced by a factor of two at each iteration, it is simple to calculate in
advance the number of iterations,k, required to achieve a given tolerance,�0, in the solution:
k = log2
�x(1) � x(0)
�0
�:
The bisection method has a relatively slow linear convergence.
6.5 Secant Method
The bisection method uses no information about the function values,f (x), apart from whether
they are positive or negative at certain values ofx. Suppose that
��f �x(k)��� < ��f �x(k�1)��� ;Then we would expect the root to lie closer tox(k) thanx(k�1). Instead of choosing the new estimate
to lie at the midpoint of the current interval, as is the case with the bisection method, thesecant
method chooses thex-intercept of the secant line to the curve, the line through�x(k�1); f
�x(k�1)
��and
�x(k); f
�x(k)��
. This places the new estimate closer to the endpoint for whichf (x) has the
smallest absolute value. The new estimate is:
x(k+1) = x(k) � f�x(k)� �x(k) � x(k�1)
�f (x(k))� f (x(k�1))
: (6.6)
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.6 REGULA FALSI 73
Note that the secant method requires two initial function evaluations but only one new function
evaluation is made at each iteration. The secant method is shown graphically in Figure 6.3.
f(x)
x
x(1)x(0) x(2)
x(3)
FIGURE 6.3:
The secant method does not have the root bracketing property of the bisection method since
the new estimate,x(k+1), of the root need not lie within the bounds defined byx(k�1) andx(k). As
a consequence, the secant method does not always converge, but when it does so it usually does
so faster than the bisection method. It can be shown that the order of convergence of the secant
method is:
1 +p5
2� 1:618:
6.6 Regula Falsi
Regula falsi is a variant of the secant method. The difference between the secant method and
regula falsi lies in the choice of points used to form the secant. While the secant method uses
the two most recent function evaluations to form the secant line through�x(k�1); f
�x(k�1)
��and�
x(k); f�x(k)��
, regula falsi forms the secant line through the most recent points that bracket the
root. The regula falsi steps are shown graphically in Figure 6.4. In this example, the pointx(0)
remains active for many steps.
The advantage of regula falsi is that like the bisection method, it is always convergent. How-
ever, like the bisection method, it has only linear convergence. Examples where the regula falsi
method is slow to converge are not hard to find. One example is shown in Figure 6.5.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
74 NONLINEAR EQUATIONS
f(x)
xx(3) x(2)x(0) x(1)
FIGURE 6.4:
f(x)
x
x(0)
x(1)x(2)
x(3)
FIGURE 6.5:
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.7 NEWTON’ S METHOD 75
6.7 Newton’s Method
The methods discussed so far have required only function values to compute the new estimate
of the root. Newton’s method requires that both the function value and the first derivative be
evaluated. Geometrically, Newton’s method proceeds by extending the tangent line to the function
at the current point until it crosses thex-axis. The new estimate of the root is taken as the abscissa
of the zero crossing. Newton’s method is defined by the following iteration:
x(k+1) = x(k) � f�x(k)�
f 0 (x(k)): (6.7)
The update is shown graphically in Figure 6.6.
f(x)
xx(0)x(1)x(2)
FIGURE 6.6:
Newton’s method may be derived from the Taylor series expansion of the function:
f (x+�x) = f (x) + f 0(x)�x + f 00(x)(�x)2
2+ : : : (6.8)
For a smooth function and small values of�x, the function is approximated well by the first two
terms. Thusf (x+�x) = 0 implies that�x = � f(x)f 0(x)
. Far from a root the higher order terms are
significant and Newton’s method can give highly inaccurate corrections. In such cases the Newton
iterations may never converge to a root. In order to achieve convergence the starting values must be
reasonably close to a root. An example of divergence using Newton’s method is given in Figure 6.7.
Newton’s method exhibits quadratic convergence. Thus, near a root the number of significant
digits doubles with each iteration. The strong convergence makes Newton’s method attractive in
cases where the derivatives can be evaluated efficiently and the derivative is continuous and non-
zero in the neighbourhood of the root (as is the case with multiple roots).
Whether the secant method should be used in preference to Newton’s method depends upon
the relative work required to compute the first derivative of the function. If the work required to
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
76 NONLINEAR EQUATIONS
f(x)
xx(0)x(1)
FIGURE 6.7:
evaluate the first derivative is greater than0:44 times the work required to evaluate the function,
then use the secant method, otherwise use Newton’s method.
It is easy to circumvent the poor global convergence properties of Newton’s method by com-
bining it with the bisection method. This hybrid method uses a bisection step whenever the Newton
method takes the solution outside the bisection bracket. Global convergence is thus assured while
retaining quadratic convergence near the root. Line searches of the Newton step fromx(k) tox(k+1)
are another method for achieving better global convergence properties of Newton’s method.
6.8 Examples
In the following examples, the bisection, secant, regula falsi and Newton’s methods are applied to
find the root of the non-linear functionf(x) = x2 � 1 between[0; 3]. xL andxR are the left and
right bracketing values andxnew is the new value determined at each iteration.� is the distance
from the true solution,nf is the number of function evaluations required at each iteration. The
bisection method is terminated when conditions (6.3) and (6.4) are simultaneously satisfied for
� = 1� 10�4. The remaining methods determine the solution to the same level of accuracy as the
bisection method.
The examples show the linear convergence rate of the bisection and regula falsi methods, the
better than linear convergence rate of the secant method and the quadratic convergence rate of
Newton’s method. It is also seen that although Newton’s method converges in 5 iterations, 5
function and 5 derivative evaluations, to give a total of 10 evaluations, are required. This contrasts
to the secant method where for the 8 iterations, 9 function evaluations are made.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.8 EXAMPLES 77
Bisection Method - f(x) = x2-1k xL xR xnew fL(x) fR(x) f(xnew) ε nf |xk+1-xk| |xk-xk-1| |xk-xk-1|/1+|xk|
0 0.000000 3.000000 1.500000 -1.000000 8.000000 1.250000 0.500000 31 0.000000 1.500000 0.750000 -1.000000 1.250000 -0.437500 0.250000 12 0.750000 1.500000 1.125000 -0.437500 1.250000 0.265625 0.125000 1 0.375000 0.750000 0.4285713 0.750000 1.125000 0.937500 -0.437500 0.265625 -0.121094 0.062500 1 0.187500 0.375000 0.1764714 0.937500 1.125000 1.031250 -0.121094 0.265625 0.063477 0.031250 1 0.093750 0.187500 0.0967745 0.937500 1.031250 0.984375 -0.121094 0.063477 -0.031006 0.015625 1 0.046875 0.093750 0.0461546 0.984375 1.031250 1.007813 -0.031006 0.063477 0.015686 0.007813 1 0.023438 0.046875 0.0236227 0.984375 1.007813 0.996094 -0.031006 0.015686 -0.007797 0.003906 1 0.011719 0.023438 0.0116738 0.996094 1.007813 1.001953 -0.007797 0.015686 0.003910 0.001953 1 0.005859 0.011719 0.0058719 0.996094 1.001953 0.999023 -0.007797 0.003910 -0.001952 0.000977 1 0.002930 0.005859 0.00292710 0.999023 1.001953 1.000488 -0.001952 0.003910 0.000977 0.000488 1 0.001465 0.002930 0.00146611 0.999023 1.000488 0.999756 -0.001952 0.000977 -0.000488 0.000244 1 0.000732 0.001465 0.00073212 0.999756 1.000488 1.000122 -0.000488 0.000977 0.000244 0.000122 1 0.000366 0.000732 0.00036613 0.999756 1.000122 0.999939 -0.000488 0.000244 -0.000122 0.000061 1 0.000183 0.000366 0.00018314 0.999939 1.000122 1.000031 0.000031 0.000092 0.000183 0.000092
16
Secant Method - f(x) = x2-1k x1 x2 f(x1) f(x2) ε nf |xk-xk-1|/1+|xk|
0 0.000000 3.000000 -1.000000 8.000000 2.000000 21 3.000000 0.333333 8.000000 -0.888889 0.666667 1 2.0000002 0.333333 0.600000 -0.888889 -0.640000 0.400000 1 0.1666673 0.600000 1.285714 -0.640000 0.653061 0.285714 1 0.3000004 1.285714 0.939394 0.653061 -0.117539 0.060606 1 0.1785715 0.939394 0.992218 -0.117539 -0.015504 0.007782 1 0.0265156 0.992218 1.000244 -0.015504 0.000488 0.000244 1 0.0040137 1.000244 0.999999 0.000488 -0.000002 0.000001 1 0.0001238 0.999999 1.000000 0.000000 0.000000
9
Regula Falsi Method - f(x) = x2-1k x1 x2 f(x1) f(x2) xnew f(xnew) ε nf |xk-xk-1|/1+|xk|
0 0.000000 3.000000 -1.000000 8.000000 0.333333 -0.888889 0.666667 31 0.333333 3.000000 -0.888889 8.000000 0.600000 -0.640000 0.400000 1 0.1666672 0.600000 3.000000 -0.640000 8.000000 0.777778 -0.395062 0.222222 1 0.1000003 0.777778 3.000000 -0.395062 8.000000 0.882353 -0.221453 0.117647 1 0.0555564 0.882353 3.000000 -0.221453 8.000000 0.939394 -0.117539 0.060606 1 0.0294125 0.939394 3.000000 -0.117539 8.000000 0.969231 -0.060592 0.030769 1 0.0151526 0.969231 3.000000 -0.060592 8.000000 0.984496 -0.030767 0.015504 1 0.0076927 0.984496 3.000000 -0.030767 8.000000 0.992218 -0.015504 0.007782 1 0.0038768 0.992218 3.000000 -0.015504 8.000000 0.996101 -0.007782 0.003899 1 0.0019469 0.996101 3.000000 -0.007782 8.000000 0.998049 -0.003899 0.001951 1 0.00097510 0.998049 3.000000 -0.003899 8.000000 0.999024 -0.001951 0.000976 1 0.00048811 0.999024 3.000000 -0.001951 8.000000 0.999512 -0.000976 0.000488 1 0.00024412 0.999512 3.000000 -0.000976 8.000000 0.999756 -0.000488 0.000244 1 0.00012213 0.999756 3.000000 -0.000488 8.000000 0.999878 0.000122 0.000061
15
Newton’s Method - f(x) = x2-1; f’(x) = 2xk x f(x) f’(x) ε nf nf’ |xk-xk-1|/1+|xk|
0 3.000000 8.000000 6.000000 2.000000 1 11 1.666667 1.777778 3.333333 0.666667 1 1 0.5000002 1.133333 0.284444 2.266667 0.133333 1 1 0.2500003 1.007843 0.015748 2.015686 0.007843 1 1 0.0625004 1.000031 0.000061 2.000061 0.000031 1 1 0.0039065 1.000000 0.000000 0.000015
5 5
f(x) = x2-1
-10
0
10
20
-1 -0.5 3E-16 0.5 1 1.5 2 2.5 3 3.5 4
x
f(x)
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
78 NONLINEAR EQUATIONS
6.9 Laguerre’s Method
In the previous sections we have investigated methods for finding the roots of general functions.
These techniques may sometimes work for finding roots of polynomials, but often problems arise
because of multiple roots or complex roots. Better methods are available for nonlinear functions
with these types of roots. One such technique is Laguerre’s method.
Consider thenth order polynomial:
P n(x) = (x� x1)(x� x2) : : : (x� xn): (6.9)
Taking the logarithm of both sides gives:
ln jPn(x)j = ln jx� x1j+ ln jx� x2j+ : : :+ ln jx� xnj : (6.10)
Let:
G =d ln jPn(x)j
dx(6.11)
=1
x� x1+
1
x� x2+ : : :+
1
x� xn(6.12)
=P 0n(x)
Pn(x)(6.13)
and
H = �d2 ln jPn(x)j
dx2(6.14)
=1
(x� x1)2+
1
(x� x2)2+ : : :+
1
(x� xn)2(6.15)
=
�P 0n(x)
Pn(x)
�2
� P 00n (x)
Pn(x): (6.16)
If we make the assumption that the rootx1 is located a distance� from our current estimate,
x(k), and all other roots are located at a distance�, then:
x(k) � x1 = � (6.17)
x(k) � xi = � for i = 2; 3; : : : ; n: (6.18)
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.9 LAGUERRE’ S METHOD 79
Equations (6.13) and (6.16) can now be expressed as:
G =1
�+n� 1
�(6.19)
H =1
�2+n� 1
�2: (6.20)
Solving for� gives:
� =n
G�p(n� 1) (nH �G2)(6.21)
where the sign should be taken to yield the largest magnitude for the denominator. The new esti-
mate of the root is obtained from the old using the update:
x(k+1) = x(k) � �: (6.22)
In general,� will be complex.
Laguerre’s method requires thatPn
�x(k)�, P 0n
�x(k)�
andP 00n�x(k)�
be computed at each step.
The method is cubic in its convergence for real or complex simple roots. Laguerre’s method will
converge to all types of roots of polynomials, real, complex, single or multiple. It requires complex
arithmetic even when converging to a real root. However, for polynomials with all real roots, it is
guaranteed to converge to a root from any starting point.
After a root,x1, of thenth order polynomial,Pn (x), has been found, the polynomial should
be divided by the quotient,(x � x1), to yield an(n � 1) order polynomial. This process is called
deflation. It saves computation when estimating further roots and ensures that subsequent iterations
do not converge to roots already found.
6.9.1 Example
Following is a simple example that illustrates the use of Laguerre’s method. Consider the third
order polynomial:
P2(x) = x2 � 4x + 3
P 02(x) = 2x� 4
P 002(x) = 2
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
80 NONLINEAR EQUATIONS
1. Let the initial estimate of the first root be:
x(0)1 = 0:5
then
G =2x
(0)1 � 4�
x(0)1
�2� 4x
(0)1 + 3
= �2:4
H = G2 � 2�x(0)1
�2� 4x
(0)1 + 3
= 4:16
� =2
G�p(2� 1)(2H �G2)= �0:5 (choosing�4 as the largest denominator)
The first root is then:
x(1) = x(0) � � = 0:5� (�0:5) = 1:
2. The second root is found by dividingP2 by (x1 � 1). The trivial result is thatx2 = 3.
6.9.2 Horner’s method for evaluating a polynomial and its derivatives
The polynomialPn (x) = a0 + a1x + a2x2 + : : : + anx
n should not be calculated by evaluating
the powers ofx individually. It is much more efficient to calculate the polynomial using Horner’s
method by noting thatPn (x) = a0 + x (a1 + x (a2 + : : :+ xan)). Using this approach the poly-
nomial value and the first and second derivatives may be evaluated efficiently using the following
algorithm:
p an;
p0 0:0;
p00 0:0;
for i in n� 1 : : : 0 loop
p00 x � p00 + p0;
p0 x � p0 + p;
p x � p+ ai;
end loop;
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.9 LAGUERRE’ S METHOD 81
p00 2 � p00;
If the polynomial and its derivatives are evaluated using the individual powers ofx, (n2+3n)=2
operations are required for the value,(n2 + n� 2)=2 for the first derivative and(n2� n� 2)=2 for
the second derivative, i.e. a total of(3n2+3n� 4)=2 operations. Horner’s method requires6n+1
operations to evaluate the polynomial and both derivatives and thus requires less operations when
n > 3. If just the function evaluation is required, then Horner’s method will require less operations
for all n.
6.9.3 Example
Following is an example of how Horner’s method can be used to evaluate
P3(x) = 1 + 2x� 3x2 + x3
and its derivatives.
1. P = 1, P 0 = 0 andP 00 = 0.
2. P 00 = 0 + 0, P 0 = 0 + 1 andP = x� 3.
3. P 00 = 0 + 1, P 0 = x + x� 3 andP = x(x� 3) + 2.
4. P 00 = x+ 2x� 3, P 0 = x(2x� 3) + x(x� 3) + 2 andP = x(x(x � 3) + 2) + 1.
5. P 00 = 2(3x� 3)
6.9.4 Deflation
Dividing a polynomial of ordern by a factor(x � x1) may be performed using the following
algorithm:
r an;
an 0:0;
for i in n� 1 : : : 0 loop
q ai;
ai r;
r x1 � r + q;
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
82 NONLINEAR EQUATIONS
end loop;
The coefficients of the new polynomial are stored in the arrayai, and the remainder inr.
6.10 Systems of Nonlinear Equations
Consider a general system ofn nonlinear equations inn unknowns:
fi(x1; x2; : : : ; xn) = 0; i = 1; 2; : : : ; n:
Newton’s method can be generalised ton dimensions by examining Taylor’s series inn dimen-
sions:
fi�x(k+1)
�= fi
�x(k)
�+ f 0ij
�x(k)
� �x(k+1)j � x
(k)j
�+ : : :
wheref 0ij�x(k)
�is ann� n matrix called the Jacobian, having elements:
f 0ij(x) =@fi(x)
@xj: (6.23)
If these gradients are approximately calculated, the method is called a quasi-Newton method. One
way to approximately form the gradients is to calculatefi(x ��xj) andfi(x +�xj) and then
form the finite difference approximation:
f 0ij(x) �fi(x+�xj)� fi(x��xj)
2�xj(6.24)
The vector�xj is a vector of zeros, except for a small perturbation in thej th position.
Settingfi�x(k+1)
�= 0 gives Newton’s formula inn dimensions:
f 0ij�x(k)
� �x(k+1)j � x
(k)j
�= �fi
�x(k)
�(6.25)
or,
f 0ij(x(k))�
(k+1)j = �fi
�x(k)
�(6.26)
where�(k+1)j =�x(k+1)j � x
(k)j
�is the vector of updates. Equation (6.26) is a set ofn linear
equations inn unknowns,�(k+1)j . If the Jacobian is non-singular, then the system of linear equations
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 SYSTEMS OFNONLINEAR EQUATIONS 83
can be solved to provide the Newton update:
x(k+1)j = x
(k)j + �
(k+1)j : (6.27)
Each step of Newton’s method requires the solution of a set of linear equations. For smalln the
set of linear equations may be solved using LU decomposition. For largen, alternative iterative
methods may be required. As with the one-dimensional version, Newton’s method converges
quadratically if the initial estimate,x(0), is sufficiently close to a root. Newton’s method in multiple
dimensions suffers from the same global convergence problems as its one-dimensional counterpart.
6.10.1 Example 1
Consider findingx andy such that the following are true:
f1(x; y) = x2 � 2x + y2 � 2y � 2xy + 1 = 0 (6.28)
f2(x; y) = x2 + 2x+ y2 + 2y + 2xy + 1 = 0 (6.29)
This is a contrived problem, but it serves to illustrate the application of Newton’s in multiple
dimensions. Solving for the simultaneous roots of these equations is equivalent to finding the roots
of the single factored equation:
f(x; y) = (x+ y + 1)2 � (x� y � 1)2 (6.30)
Inspection shows that this equation has the root(0;�1).The first step in applying Newton’s method is to determine the form of the Jacobian and the
right hand side function vector for calculating the update vector for each iteration. These are:
2664@f1(x
(k); y(k))
@x
@f1(x(k); y(k))
@y@f2(x
(k); y(k))
@x
@f2(x(k); y(k))
@y
3775"�(k+1)x
�(k+1)y
#=
"�f1(x(k); y(k))�f2(x(k); y(k))
#(6.31)
"2(x� y � 1) 2(y � x� 1)
2(x+ y + 1) 2(x+ y + 1)
#"�(k+1)x
�(k+1)y
#=
"�x2 + 2x� y2 + 2y + 2xy � 1
�x2 � 2x� y2 � 2y � 2xy � 1
#(6.32)
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
84 NONLINEAR EQUATIONS
The algorithm for solving this problem is:
k 0;
� > �;
specify starting point:(x(k); y(k));
while: � > � andk < N
�(k+1) �f 0�1f ;(LU factorisation and solution)
x(k+1) x(k) + �(k+1);
� ��� �(k+1)
2� �(k)
2
��� ;k k + 1;
end loop;
WhereN is some specified maximum number of iterations and� is some specified convergence
tolerance.
Sequences of Newton steps converging toward the root(0;�1) for the starting points(�1;�3),(2;�3) and(2; 0) are shown in figures (6.8) and (6.9).
−1−0.5
00.5
11.5
22.5
3−3
−2.5
−2
−1.5
−1
−0.5
0
−30
−20
−10
0
10
20
y
x
f(x,
y)
FIGURE 6.8: Sequence of Newton steps plotted on the surface defined by Equation (6.30).
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 SYSTEMS OFNONLINEAR EQUATIONS 85
5 10 15 20 25 30−20
−15
−10
−5
0
5
10
Number of iterations
f(x,
y)
start at (2,−3)
start at (2,0)
start at (−1,−3)
FIGURE 6.9: Convergent behaviour of Equation (6.30) with Newton iteration.
6.10.2 Example 2
Newton’s method for finding roots in multiple dimensions becomes very useful when finding ap-
proximate solutions to non-linear partial differential equations. Consider the diffusion equation in
one-dimension, where the diffusivity,D, is an exponential function of the concentration:
@c
@t� Aec
@2c
@x2= 0 (6.33)
This is a non-linear partial differential equation.
A finite difference discretisation of this equation for theith node is:
fi =cn+1i � cni
�t� Ae(1��)c
ni+�cn+1
i
�x2�(1� �)
�cni�1 � 2cni + cni+1
�+ �
�cn+1i�1 � 2cn+1i + cn+1i+1
��(6.34)
Notice that this is a non-linear equation whose roots are the unknown concentrationscn+1i�1 , cn+1i
andcn+1i+1 . If there arem discrete nodes in our problem, then there arem equations andm unknown
concentrations, so Newton’s method for multiple dimensions can be applied.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
86 NONLINEAR EQUATIONS
A row in the Jacobian for the internal nodei will be:�0 � � � @fi
@cn+1i�1
@fi@cn+1i
@fi@cn+1i+1
� � � 0
�(6.35)
or
"0 � � � ��Ae(1��)cn
i+�c
n+1i
�x2
1�t� Ae
(1��)cni+�c
n+1i
�x2
��(1� �)
�cni�1 � 2cni + cni+1
�+
�2�cn+1i�1 � 2cn+1i + cn+1i+1
�� 2��
!
��Ae(1��)cni+�c
n+1i
�x2� � � 0
i(6.36)
The corresponding right hand side vector entry for the Newton update is�fi. Note that in contrast
to the case of the linear diffusion equation, the system of discrete finite difference equations are
now time varying and must be constructed and factorised for each time step.
If Dirichlet boundary conditions of 1 and 0 are applied at nodes 1 andn, respectively, then the
functions whose roots are to be found are:
f1 = cn+11 � 1 (6.37)
fn = cn+1n (6.38)
The corresponding rows in the Jacobian will be:
h1 0 � � � 0
i(6.39)h
0 � � � 0 1i
(6.40)
The right hand side vector entries for the Newton update are1� cn+11 and�cn+1n , respectively.
Note that the unknown dependent variables for this non-linear problem arecn+1. Let these
unknowns be referred to simply asc so that thekth Newton update becomesc(k). The algorithm
for solving this problem is similar to that of Example 1:
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 SYSTEMS OFNONLINEAR EQUATIONS 87
for t in 0 � � �T loop
k 0;
� > �;
specify starting point:c(k); (This will be the solution from the previous time step)
while: � > � andk < N
�(k+1) �f 0�1f ; (LU factorisation and solution)
c(k+1) c(k) + �(k+1);
� ��� �(k+1)
2� �(k)
2
��� ;k k + 1;
end loop;
end loop;
Results for a problem of length 10m with �x = 1 m, �t = 1 s and� = 1 are shown in
Figure 6.10. The non-linear solutions were calculated using:
D(c) = 0:368ec (6.41)
The coefficient was chosen such thatD = 1 m2 s�1 at c = 1 kg m�2. The linear solutions were
calculated usingD = 0:632 m2 s�1. This value was chosen as the integrated mean value ofD(c)
from c = 0 kg m�2 to c = 1 kg m�2. The variation of the diffusion coefficient with concentration
is shown in Figure 6.11. The non-linear diffusion coefficient is smaller than the constant diffusion
coefficient at lower concentrations. This is observed in the solutions, as the concentration profile
for the non-linear diffusion does not advance as far as for constant diffusion over the same interval
of time.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
88 NONLINEAR EQUATIONS
0 1 2 3 4 5 6 7 8 9 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x (m)
Con
cent
ratio
n
L 5s NL 5s L 10s NL 10sL 50s NL 50s
(a) Profiles
0 10 20 30 40 50 60 70 80 90 1000
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Time (s)C
once
ntra
tion
Linear Non−linear
(b) Time history atx = 5 m
FIGURE 6.10: Comparative solutions for constant diffusion and non-linear diffusion.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
Concentration
Diff
usio
n C
oeffi
cien
t
D=0.368exp(c)D=0.632
FIGURE 6.11: Non-linear and constant diffusion functions.
Solving this example problem produced the following non-linear iteration information over thecourse of the first time step:
Time: 1 second
Iteration: 1 L2Norm(F) = 0.1000E+01 Delta=0.1026E+01
Iteration: 2 L2Norm(F) = 0.5549E-01 Delta=0.9876E+00
Iteration: 3 L2Norm(F) = 0.8958E-03 Delta=0.3755E-01
Iteration: 4 L2Norm(F) = 0.2278E-06 Delta=0.5565E-03
Iteration: 5 L2Norm(F) = 0.1481E-13 Delta=0.1438E-06
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 SYSTEMS OFNONLINEAR EQUATIONS 89
Note thatF is the right hand side vector, so its magnitude (or L2 norm) may be thought of as an
indication of how good the solution at that iteration is. The solution is accurate whenF is very
small. Delta is a measure of how much the solution is changing from one iteration to the next.
The Newton iterations are clearly converging.
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002