Chapter 6

Nonlinear Equations

6.1 The Problem of Nonlinear Root-finding

In this module we consider the problem of using numerical techniques to find the roots of nonlinear
equations, f (x) = 0. Initially we examine the case where the nonlinear equations are a scalar
function of a single independent variable, x. Later, we shall consider the more difficult problem
where we have a system of n nonlinear equations in n independent variables, f (x) = . 0
Since the roots of a nonlinear equation, f (x) = 0, cannot in general be expressed in closed
form, we must use approximate methods to solve the problem. Usually, iterative techniques are
used to start from an initial approximation to a root, and produce a sequence of approximations

x(0) ; x(1) ; x(2) ; : : : ;
which converge toward a root. Convergence to a root is usually possible provided that the function
f (x) is sufficiently smooth and the initial approximation is close enough to the root.

6.2 Rate of Convergence

Let

x(0) ; x(1) ; x(2) ; : : :

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
70 N ONLINEAR E QUATIONS

be a sequence which converges to a root , and let (k) = x(k) , . If there exists a number p and
a non-zero constant c such that
 
(k+1) 
lim
k!1 j(k) jp = c; (6.1)

then p is called the order of convergence of the sequence. For p = 1; 2; 3 the order is said to be
linear, quadratic and cubic respectively.

6.3 Termination Criteria

Since numerical computations are performed using floating point arithmetic, there will always be
a finite precision to which a function can be evaluated. Let  denote the limiting accuracy of
the nonlinear function near a root. This limiting accuracy of the function imposes a limit on the
accuracy, , of the root. The accuracy to which the root may be found is given by:

 = jf 0()j : (6.2)

This is the best error bound for any root finding method. Note that  is large when the first derivative
of the function at the root, jf 0 ()j, is small. In this case the problem of finding the root is ill-
conditioned. This is shown graphically in Figure 6.1.

f (x)
f (x)  

21 22

1 2
2
2 x
f 0(1) 1 large 2 small
ill-conditioned well-conditioned

F IGURE 6.1:

When the sequence


x(0) ; x(1) ; x(2) ; : : : ;
   
converges to the root, , then the differences x(k) , x(k,1)  will decrease until x(k) ,   .
c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.4 B ISECTION M ETHOD 71

With further iterations, rounding errors will dominate and the differences will vary irregularly.
The iterations should be terminated and x(k) be accepted as the estimate for the root when the
following two conditions are satisfied simultaneously:
 (k+1)   
1. x , x(k)   x(k) , x(k,1)  and (6.3)
 
x(k) , x(k,1) 
2.
1 + jx(k) j <
: (6.4)

is some coarse tolerance to prevent the iterations from being terminated before x(k) is close to ,
i.e. before the step size x(k) , x(k,1) becomes “small”. The condition (6.4) is able to test the relative
 
offset when x(k)  is much larger than 1 and much smaller than 1. In practice, these conditions are
used in conjunction with the condition that the number of iterations not exceed some user defined
limit.

6.4 Bisection Method

,

Suppose that f (x) is continuous on an interval x(0) ; x(1) . A root is bracketed on the interval
, (0) (1)
,
,

x ; x if f x(0) and f x(1) have opposite sign.
Let
(1) (0)
x(2) = x(0) + x ,2 x (6.5)

,

be the mid-point of the interval x(0) ; x(1) . Three mutually exclusive possibilities exist:
,

 if f x(2) = 0 then the root has been found;
,
,
,

 if f x(2) has the same sign as f x(0) then the root is in the interval x(2) ; x(1) ;
,
,
,

 if f x(2) has the same sign as f x(1) then the root is in the interval x(0) ; x(2) .

In the last two cases, the size of the interval bracketing the root has decreased by a factor of two.
The next iteration is performed by evaluating the function at the mid-point of the new interval.
After k iterations the size of the interval bracketing the root has decreased to:

x(1) , x(0)
2k

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
72 N ONLINEAR E QUATIONS

The process is shown graphically in Figure 6.2. The bisection method is guaranteed to converge
to a root. If the initial interval brackets more than one root, then the bisection method will find one
of them.

f (x)
x(0) x(1)
x

x(2)
x(3) x(4)
x(5)
F IGURE 6.2:

Since the interval size is reduced by a factor of two at each iteration, it is simple to calculate in
advance the number of iterations, k , required to achieve a given tolerance, 0 , in the solution:

(1) 
(0) 
k = log2 x , x :
0
The bisection method has a relatively slow linear convergence.

6.5 Secant Method

The bisection method uses no information about the function values, f (x), apart from whether
they are positive or negative at certain values of x. Suppose that
 , (k)
  ,

f x  < f x(k,1)  ;
Then we would expect the root to lie closer to x(k) than x(k,1) . Instead of choosing the new estimate
to lie at the midpoint of the current interval, as is the case with the bisection method, the secant
, ,


method chooses the x-intercept of the secant line to the curve, the line through x(k,1) ; f x(k,1)
, ,


and x(k) ; f x(k) . This places the new estimate closer to the endpoint for which f (x) has the
smallest absolute value. The new estimate is:
, (k)
, (k) (k,1)

f x x , x
x(k+1) = x(k) , f (x(k) ) , f (x(k,1) ) : (6.6)

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.6 R EGULA FALSI 73

Note that the secant method requires two initial function evaluations but only one new function
evaluation is made at each iteration. The secant method is shown graphically in Figure 6.3.

f (x)

x(0) x(2) x(1)

x(3) x

F IGURE 6.3:

The secant method does not have the root bracketing property of the bisection method since
the new estimate, x(k+1) , of the root need not lie within the bounds defined by x(k,1) and x(k) . As
a consequence, the secant method does not always converge, but when it does so it usually does
so faster than the bisection method. It can be shown that the order of convergence of the secant
method is:
p
1 + 5  1:618:
2

6.6 Regula Falsi

Regula falsi is a variant of the secant method. The difference between the secant method and
regula falsi lies in the choice of points used to form the secant. While the secant method uses
, ,


the two most recent function evaluations to form the secant line through x(k,1) ; f x(k,1) and
, (k) , (k)


x ; f x , regula falsi forms the secant line through the most recent points that bracket the
root. The regula falsi steps are shown graphically in Figure 6.4. In this example, the point x(0)
remains active for many steps.
The advantage of regula falsi is that like the bisection method, it is always convergent. How-
ever, like the bisection method, it has only linear convergence. Examples where the regula falsi
method is slow to converge are not hard to find. One example is shown in Figure 6.5.

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
74 N ONLINEAR E QUATIONS

f (x)

x(0) x(1)
x(3) x(2) x

F IGURE 6.4:

f (x) x(3)
x(0)
x(2) x(1) x

F IGURE 6.5:

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.7 N EWTON ’ S M ETHOD 75

6.7 Newton’s Method

The methods discussed so far have required only function values to compute the new estimate
of the root. Newton’s method requires that both the function value and the first derivative be
evaluated. Geometrically, Newton’s method proceeds by extending the tangent line to the function
at the current point until it crosses the x-axis. The new estimate of the root is taken as the abscissa
of the zero crossing. Newton’s method is defined by the following iteration:
,

f x(k)
x(k+1) = x(k) , f 0 (x(k) ) : (6.7)

The update is shown graphically in Figure 6.6.

f (x)

x(2) x(1) x(0) x

F IGURE 6.6:

Newton’s method may be derived from the Taylor series expansion of the function:
2
f (x +
x) = f (x) + f 0(x)
x + f 00(x) (
2x) + : : : (6.8)

For a smooth function and small values of
x, the function is approximated well by the first two
terms. Thus f (x +
x) = 0 implies that
x = , ff0((xx)) . Far from a root the higher order terms are
significant and Newton’s method can give highly inaccurate corrections. In such cases the Newton
iterations may never converge to a root. In order to achieve convergence the starting values must be
reasonably close to a root. An example of divergence using Newton’s method is given in Figure 6.7.
Newton’s method exhibits quadratic convergence. Thus, near a root the number of significant
digits doubles with each iteration. The strong convergence makes Newton’s method attractive in
cases where the derivatives can be evaluated efficiently and the derivative is continuous and non-
zero in the neighbourhood of the root (as is the case with multiple roots).
Whether the secant method should be used in preference to Newton’s method depends upon
the relative work required to compute the first derivative of the function. If the work required to

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
76 N ONLINEAR E QUATIONS

f (x)

x(1) x(0) x

F IGURE 6.7:

evaluate the first derivative is greater than 0:44 times the work required to evaluate the function,
then use the secant method, otherwise use Newton’s method.
It is easy to circumvent the poor global convergence properties of Newton’s method by com-
bining it with the bisection method. This hybrid method uses a bisection step whenever the Newton
method takes the solution outside the bisection bracket. Global convergence is thus assured while
retaining quadratic convergence near the root. Line searches of the Newton step from x(k) to x(k+1)
are another method for achieving better global convergence properties of Newton’s method.

6.8 Examples
In the following examples, the bisection, secant, regula falsi and Newton’s methods are applied to
find the root of the non-linear function f (x) = x2 , 1 between [0; 3]. xL and xR are the left and
right bracketing values and xnew is the new value determined at each iteration.  is the distance
from the true solution, nf is the number of function evaluations required at each iteration. The
bisection method is terminated when conditions (6.3) and (6.4) are simultaneously satisfied for

= 1  10,4. The remaining methods determine the solution to the same level of accuracy as the
bisection method.
The examples show the linear convergence rate of the bisection and regula falsi methods, the
better than linear convergence rate of the secant method and the quadratic convergence rate of
Newton’s method. It is also seen that although Newton’s method converges in 5 iterations, 5
function and 5 derivative evaluations, to give a total of 10 evaluations, are required. This contrasts
to the secant method where for the 8 iterations, 9 function evaluations are made.

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.8 E XAMPLES 77

Bisection Method - f(x) = x2-1

k xL xR xnew fL(x) fR(x) f(xnew) ε nf |xk+1-xk| |xk-xk-1| |xk-xk-1|/1+|xk|
0 0.000000 3.000000 1.500000 -1.000000 8.000000 1.250000 0.500000 3
1 0.000000 1.500000 0.750000 -1.000000 1.250000 -0.437500 0.250000 1
2 0.750000 1.500000 1.125000 -0.437500 1.250000 0.265625 0.125000 1 0.375000 0.750000 0.428571
3 0.750000 1.125000 0.937500 -0.437500 0.265625 -0.121094 0.062500 1 0.187500 0.375000 0.176471
4 0.937500 1.125000 1.031250 -0.121094 0.265625 0.063477 0.031250 1 0.093750 0.187500 0.096774
5 0.937500 1.031250 0.984375 -0.121094 0.063477 -0.031006 0.015625 1 0.046875 0.093750 0.046154
6 0.984375 1.031250 1.007813 -0.031006 0.063477 0.015686 0.007813 1 0.023438 0.046875 0.023622
7 0.984375 1.007813 0.996094 -0.031006 0.015686 -0.007797 0.003906 1 0.011719 0.023438 0.011673
8 0.996094 1.007813 1.001953 -0.007797 0.015686 0.003910 0.001953 1 0.005859 0.011719 0.005871
9 0.996094 1.001953 0.999023 -0.007797 0.003910 -0.001952 0.000977 1 0.002930 0.005859 0.002927
10 0.999023 1.001953 1.000488 -0.001952 0.003910 0.000977 0.000488 1 0.001465 0.002930 0.001466
11 0.999023 1.000488 0.999756 -0.001952 0.000977 -0.000488 0.000244 1 0.000732 0.001465 0.000732
12 0.999756 1.000488 1.000122 -0.000488 0.000977 0.000244 0.000122 1 0.000366 0.000732 0.000366
13 0.999756 1.000122 0.999939 -0.000488 0.000244 -0.000122 0.000061 1 0.000183 0.000366 0.000183
14 0.999939 1.000122 1.000031 0.000031 0.000092 0.000183 0.000092
16

Secant Method - f(x) = x2-1

f(x) = x2-1
k x1 x2 f(x1) f(x2) ε nf |xk-xk-1|/1+|xk|
0 0.000000 3.000000 -1.000000 8.000000 2.000000 2 f(x) 20
1 3.000000 0.333333 8.000000 -0.888889 0.666667 1 2.000000
2 0.333333 0.600000 -0.888889 -0.640000 0.400000 1 0.166667
10
3 0.600000 1.285714 -0.640000 0.653061 0.285714 1 0.300000
4 1.285714 0.939394 0.653061 -0.117539 0.060606 1 0.178571
0
5 0.939394 0.992218 -0.117539 -0.015504 0.007782 1 0.026515
-1 -0.5 3E-16 0.5 1 1.5 2 2.5 3 3.5 4
6 0.992218 1.000244 -0.015504 0.000488 0.000244 1 0.004013
-10 x
7 1.000244 0.999999 0.000488 -0.000002 0.000001 1 0.000123
8 0.999999 1.000000 0.000000 0.000000
9

Regula Falsi Method - f(x) = x2-1

k x1 x2 f(x1) f(x2) xnew f(xnew) ε nf |xk-xk-1|/1+|xk|
0 0.000000 3.000000 -1.000000 8.000000 0.333333 -0.888889 0.666667 3
1 0.333333 3.000000 -0.888889 8.000000 0.600000 -0.640000 0.400000 1 0.166667
2 0.600000 3.000000 -0.640000 8.000000 0.777778 -0.395062 0.222222 1 0.100000
3 0.777778 3.000000 -0.395062 8.000000 0.882353 -0.221453 0.117647 1 0.055556
4 0.882353 3.000000 -0.221453 8.000000 0.939394 -0.117539 0.060606 1 0.029412
5 0.939394 3.000000 -0.117539 8.000000 0.969231 -0.060592 0.030769 1 0.015152
6 0.969231 3.000000 -0.060592 8.000000 0.984496 -0.030767 0.015504 1 0.007692
7 0.984496 3.000000 -0.030767 8.000000 0.992218 -0.015504 0.007782 1 0.003876
8 0.992218 3.000000 -0.015504 8.000000 0.996101 -0.007782 0.003899 1 0.001946
9 0.996101 3.000000 -0.007782 8.000000 0.998049 -0.003899 0.001951 1 0.000975
10 0.998049 3.000000 -0.003899 8.000000 0.999024 -0.001951 0.000976 1 0.000488
11 0.999024 3.000000 -0.001951 8.000000 0.999512 -0.000976 0.000488 1 0.000244
12 0.999512 3.000000 -0.000976 8.000000 0.999756 -0.000488 0.000244 1 0.000122
13 0.999756 3.000000 -0.000488 8.000000 0.999878 0.000122 0.000061
15

Newton’s Method - f(x) = x2-1; f’(x) = 2x

k x f(x) f’(x) ε nf nf’ |xk-xk-1|/1+|xk|
0 3.000000 8.000000 6.000000 2.000000 1 1
1 1.666667 1.777778 3.333333 0.666667 1 1 0.500000
2 1.133333 0.284444 2.266667 0.133333 1 1 0.250000
3 1.007843 0.015748 2.015686 0.007843 1 1 0.062500
4 1.000031 0.000061 2.000061 0.000031 1 1 0.003906
5 1.000000 0.000000 0.000015
5 5

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
78 N ONLINEAR E QUATIONS

6.9 Laguerre’s Method

In the previous sections we have investigated methods for finding the roots of general functions.
These techniques may sometimes work for finding roots of polynomials, but often problems arise
because of multiple roots or complex roots. Better methods are available for nonlinear functions
with these types of roots. One such technique is Laguerre’s method.
Consider the nth order polynomial:

P n(x) = (x , x1 )(x , x2 ) : : : (x , xn): (6.9)

Taking the logarithm of both sides gives:

ln jPn (x)j = ln jx , x1 j + ln jx , x2 j + : : : + ln jx , xn j : (6.10)

Let:

G = d ln jdx
Pn(x)j (6.11)

= 1 + 1 +:::+ 1
x , x1 x , x2 x , xn (6.12)
0
= PPn((xx)) (6.13)
n
and
2 jPn(x)j
H = , d lndx 2 (6.14)

= (x ,1x )2 + (x ,1x )2 + : : : + (x ,1x )2 (6.15)

1 2 n
 0 2 00
= Pn(x) , Pn (x) : (6.16)
Pn(x) Pn(x)
If we make the assumption that the root x1 is located a distance  from our current estimate,
(k )
x , and all other roots are located at a distance  , then:
x(k) , x1 =  (6.17)
x(k) , xi =  for i = 2; 3; : : : ; n: (6.18)

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.9 L AGUERRE ’ S M ETHOD 79

Equations (6.13) and (6.16) can now be expressed as:

G = 1 + n ,

1 (6.19)

H = 2 + ,2 1 :
1 n (6.20)

Solving for  gives:

= n
p (6.21)
G  (n , 1) (nH , G2)
where the sign should be taken to yield the largest magnitude for the denominator. The new esti-
mate of the root is obtained from the old using the update:

x(k+1) = x(k) , : (6.22)

In general,  will be complex.

,
,
,

Laguerre’s method requires that Pn x(k) , Pn0 x(k) and Pn00 x(k) be computed at each step.
The method is cubic in its convergence for real or complex simple roots. Laguerre’s method will
converge to all types of roots of polynomials, real, complex, single or multiple. It requires complex
arithmetic even when converging to a real root. However, for polynomials with all real roots, it is
guaranteed to converge to a root from any starting point.
After a root, x1 , of the nth order polynomial, Pn (x), has been found, the polynomial should
be divided by the quotient, (x , x1 ), to yield an (n , 1) order polynomial. This process is called
deflation. It saves computation when estimating further roots and ensures that subsequent iterations
do not converge to roots already found.

6.9.1 Example
Following is a simple example that illustrates the use of Laguerre’s method. Consider the third
order polynomial:

P2(x) = x2 , 4x + 3
P 02(x) = 2x , 4
P 00 2(x) = 2

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
80 N ONLINEAR E QUATIONS

1. Let the initial estimate of the first root be:

x(0)
1 = 0:5

then
(0)
G =  (0) 2x2 1 , 4(0) = ,2:4
x1 , 4x1 + 3
H = G2 ,  (0) 2 2 (0) = 4:16
x1 , 4x1 + 3
= p
2 = ,0:5 (choosing ,4 as the largest denominator)
G  (2 , 1)(2H , G2 )
The first root is then:

x(1) = x(0) ,  = 0:5 , (,0:5) = 1:

2. The second root is found by dividing P2 by (x1 , 1). The trivial result is that x2 = 3.

6.9.2 Horner’s method for evaluating a polynomial and its derivatives

The polynomial Pn (x) = a0 + a1 x + a2 x2 + : : : + an xn should not be calculated by evaluating
the powers of x individually. It is much more efficient to calculate the polynomial using Horner’s
method by noting that Pn (x) = a0 + x (a1 + x (a2 + : : : + xan )). Using this approach the poly-
nomial value and the first and second derivatives may be evaluated efficiently using the following
algorithm:

p an ;
p0 0:0;
p00 0:0;
for i in n , 1 : : : 0 loop

p00 x  p00 + p0 ;
p0 x  p0 + p;
p x  p + ai ;
end loop;

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.9 L AGUERRE ’ S M ETHOD 81

p00 2  p00;
If the polynomial and its derivatives are evaluated using the individual powers of x, (n2 +3n)=2
operations are required for the value, (n2 + n , 2)=2 for the first derivative and (n2 , n , 2)=2 for
the second derivative, i.e. a total of (3n2 + 3n , 4)=2 operations. Horner’s method requires 6n + 1
operations to evaluate the polynomial and both derivatives and thus requires less operations when
n > 3. If just the function evaluation is required, then Horner’s method will require less operations
for all n.

6.9.3 Example
Following is an example of how Horner’s method can be used to evaluate

P3(x) = 1 + 2x , 3x2 + x3
and its derivatives.

1. P = 1, P 0 = 0 and P 00 = 0.
2. P 00 = 0 + 0, P 0 = 0 + 1 and P = x , 3.
3. P 00 = 0 + 1, P 0 = x + x , 3 and P = x(x , 3) + 2.
4. P 00 = x + 2x , 3, P 0 = x(2x , 3) + x(x , 3) + 2 and P = x(x(x , 3) + 2) + 1.
5. P 00 = 2(3x , 3)

6.9.4 Deflation
Dividing a polynomial of order n by a factor (x , x1 ) may be performed using the following
algorithm:

r an ;
an 0:0;
for i in n , 1 : : : 0 loop

q ai ;
ai r;
r x1  r + q ;

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
82 N ONLINEAR E QUATIONS

end loop;

The coefficients of the new polynomial are stored in the array ai , and the remainder in r .

6.10 Systems of Nonlinear Equations

Consider a general system of n nonlinear equations in n unknowns:

fi (x1; x2 ; : : : ; xn ) = 0; i = 1; 2; : : : ; n:
Newton’s method can be generalised to n dimensions by examining Taylor’s series in n dimen-
sions:
,
,
,
 
fi x(k+1) = fi x(k) + f 0ij x(k) x(jk+1) , x(jk) + : : :
,

where f 0 ij x(k) is an n  n matrix called the Jacobian, having elements:

f 0ij (x) = @f@xi (x) : (6.23)

j
If these gradients are approximately calculated, the method is called a quasi-Newton method. One
way to approximately form the gradients is to calculate fi (x ,
x ) and fi(x +
x ) and then
j j

form the finite difference approximation:

f 0ij (x)  fi(x +
xj2

) , fi (x ,
xj )
x (6.24)
j

,


The vector xj is a vector of zeros, except for a small perturbation in the j th position.


Setting fi x(k+1) = 0 gives Newton’s formula in n dimensions:

, (k)
 (k+1) ( )
 ,

0
f ij x xj , xj = ,fi x(k)
k (6.25)

or,

,

f 0ij (x(k) )j(k+1) = ,fi x(k) (6.26)

where j
(k+1) (k+1) (k) 
= xj , xj is the vector of updates. Equation (6.26) is a set of n linear
(k+1) . If the Jacobian is non-singular, then the system of linear equations
equations in n unknowns, j

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 S YSTEMS OF N ONLINEAR E QUATIONS 83

can be solved to provide the Newton update:

x(jk+1) = x(jk) + j(k+1) : (6.27)

Each step of Newton’s method requires the solution of a set of linear equations. For small n the
set of linear equations may be solved using LU decomposition. For large n, alternative iterative
methods may be required. As with the one-dimensional version, Newton’s method converges
quadratically if the initial estimate, x(0) , is sufficiently close to a root. Newton’s method in multiple
dimensions suffers from the same global convergence problems as its one-dimensional counterpart.

6.10.1 Example 1
Consider finding x and y such that the following are true:

f1(x; y) = x2 , 2x + y2 , 2y , 2xy + 1 = 0 (6.28)

f2(x; y) = x2 + 2x + y2 + 2y + 2xy + 1 = 0 (6.29)

This is a contrived problem, but it serves to illustrate the application of Newton’s in multiple
dimensions. Solving for the simultaneous roots of these equations is equivalent to finding the roots
of the single factored equation:

f (x; y) = (x + y + 1)2 , (x , y , 1)2 (6.30)

Inspection shows that this equation has the root (0; ,1).
The first step in applying Newton’s method is to determine the form of the Jacobian and the
right hand side function vector for calculating the update vector for each iteration. These are:
2 3
@f1 (x(k) ; y(k)) @f1 (x(k) ; y(k)) " (k+1) # " #
 , f (x (k) ; y (k))
6 @x @y 7 x 1
4 @f2 (x(k) ; y (k) ) @f2 (x(k) ; y (k) ) 5  (k+1) = ,f (x(k) ; y (k) )
6 7 (6.31)
y 2
"
@x @y #" # " #
2(x , y , 1) 2(y , x , 1) x(k+1) , x 2 + 2x , y 2 + 2y + 2xy , 1
=
2(x + y + 1) 2(x + y + 1) y(k+1) ,x2 , 2x , y2 , 2y , 2xy , 1
(6.32)

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
84 N ONLINEAR E QUATIONS

The algorithm for solving this problem is:

k 0;

> ;
specify starting point: (x(k) ; y (k) );

while:
>  and k < N

 (k+1) ,f ,1f ;(LU factorisation and solution)
0

x(k+1) x(k) + (k+1) ;

 
 (k+1) (k) 

  ,   ;
2 2
k k + 1;
end loop;

Where N is some specified maximum number of iterations and  is some specified convergence
tolerance.
Sequences of Newton steps converging toward the root (0; ,1) for the starting points (,1; ,3),
(2; ,3) and (2; 0) are shown in figures (6.8) and (6.9).

20

10

0 0
f(x,y)

−0.5
−10
−1
−20
−1.5
−30 y
−1 −2
−0.5
0
0.5 −2.5
1
x 1.5
2
2.5 −3
3

F IGURE 6.8: Sequence of Newton steps plotted on the surface defined by Equation (6.30).

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 S YSTEMS OF N ONLINEAR E QUATIONS 85

10

5
start at (−1,−3)

0
f(x,y)

−5

−10
start at (2,0)

−15
start at (2,−3)

−20
5 10 15 20 25 30

Number of iterations

F IGURE 6.9: Convergent behaviour of Equation (6.30) with Newton iteration.

6.10.2 Example 2
Newton’s method for finding roots in multiple dimensions becomes very useful when finding ap-
proximate solutions to non-linear partial differential equations. Consider the diffusion equation in
one-dimension, where the diffusivity, D , is an exponential function of the concentration:

@c , Aec @ 2 c = 0
@t @x2 (6.33)

This is a non-linear partial differential equation.

A finite difference discretisation of this equation for the ith node is:

cn+1 , cn Ae(1,)cni +cni +1

 ,
,

fi = i
t i ,
x2 (1 , ) cni,1 , 2cni + cni+1 +  cni,+11 , 2cni +1 + cni+1
+1
(6.34)

Notice that this is a non-linear equation whose roots are the unknown concentrations cni,+1 n+1
1 , ci
+1 . If there are m discrete nodes in our problem, then there are m equations and m unknown
and cni+1
concentrations, so Newton’s method for multiple dimensions can be applied.

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
86 N ONLINEAR E QUATIONS

A row in the Jacobian for the internal node i will be:

 
@f
0


n+1 i @f i @f i
+1


0
@ci,1 @cni +1 @cni+1 (6.35)
" !
,)cn +cn+1 1 , Ae(1,)cni2+cni +1 (1 , ) ,cn , 2cn + cn
+
0


,  Ae i i
t
x, i,1
i i+1
(1
or
x2 +1 , 2
2 cni,+11 , 2cni +1 + cni+1
i
,)cni +cni +1
, Ae (1

x2


0 (6.36)

The corresponding right hand side vector entry for the Newton update is ,fi . Note that in contrast
to the case of the linear diffusion equation, the system of discrete finite difference equations are
now time varying and must be constructed and factorised for each time step.
If Dirichlet boundary conditions of 1 and 0 are applied at nodes 1 and n, respectively, then the
functions whose roots are to be found are:

f1 = cn1 +1 , 1 (6.37)
fn = cnn+1 (6.38)

The corresponding rows in the Jacobian will be:

h i
1 0


0 (6.39)
h i
0


0 1 (6.40)

The right hand side vector entries for the Newton update are 1 , cn1 +1 and ,cnn+1 , respectively.
Note that the unknown dependent variables for this non-linear problem are cn+1 . Let these
unknowns be referred to simply as c so that the k th Newton update becomes c(k) . The algorithm
for solving this problem is similar to that of Example 1:

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 S YSTEMS OF N ONLINEAR E QUATIONS 87

for t in 0


T loop

k 0;

> ;
specify starting point: c(k) ; (This will be the solution from the previous time step)
while:
>  and k < N
 (k+1) ,f ,1f ; (LU factorisation and solution)
0

c(k+1) c(k) +  (k+1) ;

 

 (k+1) , (k)  ;
2 2
k k + 1;
end loop;

end loop;

Results for a problem of length 10 m with
x = 1 m,
t = 1 s and  = 1 are shown in

Figure 6.10. The non-linear solutions were calculated using:

D(c) = 0:368ec (6.41)

The coefficient was chosen such that D = 1 m2 s,1 at c = 1 kg m,2. The linear solutions were
calculated using D = 0:632 m2 s,1 . This value was chosen as the integrated mean value of D (c)
from c = 0 kg m,2 to c = 1 kg m,2 . The variation of the diffusion coefficient with concentration
is shown in Figure 6.11. The non-linear diffusion coefficient is smaller than the constant diffusion
coefficient at lower concentrations. This is observed in the solutions, as the concentration profile
for the non-linear diffusion does not advance as far as for constant diffusion over the same interval
of time.

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
88 N ONLINEAR E QUATIONS

1
L 5s 0.5
NL 5s
0.9 L 10s
NL 10s 0.45
L 50s
0.8 NL 50s
0.4

0.7
0.35

0.6
Concentration

0.3

Concentration
0.5
0.25

0.4
0.2

0.3 0.15

0.2 0.1

0.1 0.05
Linear
Non−linear
0 0
0 1 2 3 4 5 6 7 8 9 10 0 10 20 30 40 50 60 70 80 90 100
x (m) Time (s)

(a) Profiles (b) Time history at x = 5 m

F IGURE 6.10: Comparative solutions for constant diffusion and non-linear diffusion.

1.1
D=0.368exp(c)
D=0.632
1

0.9
Diffusion Coefficient

0.8

0.7

0.6

0.5

0.4

0.3
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Concentration

F IGURE 6.11: Non-linear and constant diffusion functions.

Solving this example problem produced the following non-linear iteration information over the
course of the first time step:

Time: 1 second
Iteration: 1 L2Norm(F) = 0.1000E+01 Delta=0.1026E+01
Iteration: 2 L2Norm(F) = 0.5549E-01 Delta=0.9876E+00
Iteration: 3 L2Norm(F) = 0.8958E-03 Delta=0.3755E-01
Iteration: 4 L2Norm(F) = 0.2278E-06 Delta=0.5565E-03
Iteration: 5 L2Norm(F) = 0.1481E-13 Delta=0.1438E-06

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002
6.10 S YSTEMS OF N ONLINEAR E QUATIONS 89

Note that F is the right hand side vector, so its magnitude (or L2 norm) may be thought of as an
indication of how good the solution at that iteration is. The solution is accurate when F is very
small. Delta is a measure of how much the solution is changing from one iteration to the next.
The Newton iterations are clearly converging.

c Department of Engineering Science, University of Auckland, New Zealand. All rights reserved. July 11, 2002