Conditioning[edit]
Ill-conditioned problem: Take the function f(x) = 1/(x − 1). Note that f(1.1) = 10 and f(1.001) = 1000: a
change in x of less than 0.1 turns into a change in f(x) of nearly 1000. Evaluating f(x) near x = 1 is an
ill-conditioned problem.
Well-conditioned problem: By contrast, evaluating the same function f(x) = 1/(x − 1) near x = 10 is a
well-conditioned problem. For instance, f(10) = 1/9 ≈ 0.111 and f(11) = 0.1: a modest change
in x leads to a modest change in f(x).
Discretization[edit]
Furthermore, continuous problems must sometimes be replaced by a discrete problem whose
solution is known to approximate that of the continuous problem; this process is called
'discretization'. For example, the solution of a differential equation is a function. This function must
be represented by a finite amount of data, for instance by its value at a finite number of points at its
domain, even though this domain is a continuum.
Generation and propagation of errors[edit]
Further information: Error propagation
The study of errors forms an important part of numerical analysis. There are several ways in which
error can be introduced in the solution of the problem.
Round-off[edit]
Round-off errors arise because it is impossible to represent all real numbers exactly on a machine
with finite memory (which is what all practical digital computers are).
Truncation and discretization error[edit]
Truncation errors are committed when an iterative method is terminated or a mathematical
procedure is approximated and the approximate solution differs from the exact solution. Similarly,
discretization induces a discretization error because the solution of the discrete problem does not
coincide with the solution of the continuous problem. In the example above to compute the solution
of , after ten iterations, the calculated root is roughly 1.99. Therefore, the truncation error is
roughly 0.01.
Once an error is generated, it propagates through the calculation. For example, the operation + on a
computer is inexact. A calculation of the type is even more inexact.
A truncation error is created when a mathematical procedure is approximated. To integrate a
function exactly, an infinite sum of regions must be found, but numerically only a finite sum of
regions can be found, and hence the approximation of the exact solution. Similarly, to differentiate a
function, the differential element approaches zero, but numerically only a nonzero value of the
differential element can be chosen.
Numerical stability and well-posed problems[edit]
An algorithm is called numerically stable if an error, whatever its cause, does not grow to be much
larger during the calculation.[13] This happens if the problem is well-conditioned, meaning that the
solution changes by only a small amount if the problem data are changed by a small amount. [13] To
the contrary, if a problem is 'ill-conditioned', then any small error in the data will grow to be a large
error.[13] Both the original problem and the algorithm used to solve that problem can be well-
conditioned or ill-conditioned, and any combination is possible. So an algorithm that solves a well-
conditioned problem may be either numerically stable or numerically unstable. An art of numerical
analysis is to find a stable algorithm for solving a well-posed mathematical problem.