Numerical Integration
Numerical integration is used for definite integrals. Most
techniques apply to low dimensional integration - we look at these
today. Integration in many-dimensional space typically relies on
Monte Carlo techniques (next semester).
We want to find a numerical approximation to
f(xi+1)
b
a f (x) dx f(xi)
h
a xi xi+1 b
Newton-Cotes methods - use equidistant points:
b a
x i = a + ih i = 0,...,n h=
n
Winter Semester 2006/7 Computational Physics I Lecture 10 1
Lagrange Polynomials
For n+1 points (xi,yi), with
i = 0,1,, n xi x j i
there is a unique interpolating polynomial of degree n with
p(xi ) = yi i = 0,1,, n
Can construct this polynomial using the Lagrange polynomials,
defined as:
(x x0 )(x xi 1 )(x xi +1 )(x xn )
Li (x) =
(xi x0 )(xi xi 1 )(xi xi +1 )(xi xn )
Degree n (denominator is constant), and
Li (xk ) = i, k
Winter Semester 2006/7 Computational Physics I Lecture 10 2
Lagrange Polynomials
The Lagrange Polynomials can be used to form the interpolating
polynomial:
n n n
x xk
p(x) = yi Li (x) = yi
i=0 i=0 k = 0, k i xi x k
In our application, y i = f (x i )
Integration of the polynomial yields:
n n x xk n n s k
b
a p(x) dx = f (x i ) ab dx = f (x i ) 0
n
hds
i= 0 k = 0,k i x i x k i= 0 k = 0,k i i k
where x = a + sh
n s k
Note that
n
0 hds = i h
k = 0,k i i k
So i is just a number depending on i,n
Winter Semester 2006/7 Computational Physics I Lecture 10 3
Newton-Cotes Formulas
n
So,
b
a p(x) dx = h f (x i ) i
i= 0
Look at some specific examples:
x x1 x x0
n =1 p(x) = f (x 0 ) + f (x1 )
x 0 x1 x1 x 0
1 s k s 1 12
1
0 = 0
1
ds = 0
1
ds = + 1 =
k = 0,k i i k 0 1 2 2
1 s k s 0 12
1
1 = 0
1
ds = 0
1
ds = =
k = 0,k i i k 1 0 2 2
n 1 1 h
b
a p(x) dx = h f (x i ) i = hf (x 0 ) + hf (x1 ) = ( f (x 0 ) + f (x1 ))
i= 0 2 2 2
Trapezoidal rule
Winter Semester 2006/7 Computational Physics I Lecture 10 4
Trapezoidal Rule
Graphically: f(x1)
A=f(x0)h+1/2[f(x1)-f(x0)]h
f(x0)
h
x0 x1
Of course, we can apply this to many subdivisions (composite
trapezoidal rule) :
m
b
a f (x) dx = zz f (z) dz where z0 = a, zm +1 = b
i
i +1
i= 0
f (x 0 ) f (x1 ) f (x1 ) f (x 2 ) f (b h) f (b)
b
a f (x) dx h + + + + + +
2 2 2 2 2 2
f (x 0 ) f (b)
a f (x) dx h + f (x1 ) + + f (b h) +
b
2 2
Winter Semester 2006/7 Computational Physics I Lecture 10 5
Error Order
To find the error order of a particular approximation, we consider a
series approximation. For the numerical simulation of derivatives,
we compared to a Taylor series. Here we compare to the Euler-
Maclaurin summation formula:
[ ]
n 1 1 B
F(i) =
n
0 F(s) ds [ F(0) + F(n)] + 2k F (2k 1) (n) F (2k 1) (0)
i=1 2 k =1 (2k)!
where
F (k ) is the k - th derivative of F and the B2k are the Bernoulli numbers
n Bn
0 1
1 /2 = .5 Bn=0 for n3, odd
2 1/6 0.1667
4 /30 .0333
6 1/42 0.02381
8 /30 .0333
10 5/66 0.07576
12 91/2730 .2531
14 7/6 1.1667
Winter Semester 2006/7 Computational Physics I Lecture 10 6
Error Order
Make the transformation: x = sh + a
[ ]
n 1 1 b 1 B
f (x i ) = a f (x) dx [ f (a) + f (b)] + 2k f
(2k 1)
(b) f ( 2k 1) (a) h 2k
i=1 h 2 k =1 (2k)!
or
n 1 2
h h
a f (x) dx = h f (x i ) + [ f (a) + f (b)] [ f (b) f (a)] + O(h )
b 4
i=1 2 12
Trapezoidal Rule
We see that the error from the trapezoidal rule is of order h2.
The precision can be improved by setting h smaller and smaller
but we will eventually hit rounding errors and computing time
limitations. Look for methods with higher order of the error.
Winter Semester 2006/7 Computational Physics I Lecture 10 7
Simpson’s Rule
n f (x 0 ) + 4 f (x1 ) + f (x 2 )
n=2 a p(x) dx = h f (x i ) i = 2h
b
i= 0 6
f(x2)
f(x1) Error order h4
(interpolating
f(x0) polynomial of
2h
2nd order)
x0 x1 x2
Composite Simpson’s rule:
f (a) + 4 f (a + h) + f (a + 2h) + Note: range divided
b 2h f (a + 2h) + 4 f (a + 3h) + f (a + 4h) + into an even
f (x) dx
a 6 + multiple of h
f (b 2h) + 4 f (b h) + f (b)
h
= [ f (a) + 4 f (a + h) + 2 f (a + 2h) + 4 f (a + 3h) + + 4 f (b h) + f (b)]
3
Winter Semester 2006/7 Computational Physics I Lecture 10 8
Midpoint Rule
It is sometimes preferable to sample the function at the midpoint
of the interval rather than at the ends (e.g., if the function has a
singularity at the endpoint). Simplest example:
f((a+b)/2)
a + b
b
a f (x) dx hf h = b a
2
h
The composite midpoint rule gives: a b
b
a f (x) dx 2h ( f (a + h) + f (a + 3h) + + f (b h))
The error order is h2
Winter Semester 2006/7 Computational Physics I Lecture 10 9
Extrapolation Method
Let Rij be our estimate of the integral when 2i-1 intervals are used
and j is the order of the interpolating polynomial (previously n).
E.g., j=1 is Trapezoidal Rule, j=2 is Simpson’s Rule.
Look at the error formula:
b
a f (x) dx = TS (h) + 2 h 2 + 4 h 4 +
h
where TS (h) is e.g. the Trapezoidal rule. Now take h
2
a f (x) dx = TS (h /2) + 2 h /4 + 4 h /16 +
b 2 4
so
4TS (h /2) TS (h)
b
a f (x) dx = 4 h 4 /4 +
3
i.e., can get a higher order error by combining results from
different step sizes (recall extrapolation method for derivatives).
Winter Semester 2006/7 Computational Physics I Lecture 10 10
Extrapolation Method
There is a general formula for calculating the higher order terms -
Richardson Formula:
Ri+1, j Ri, j
Ri+1, j +1 = Ri+1, j + See Scherer lecture notes
4 1
j
for derivation
So we can gain higher order of error at the same time as making
the step size smaller (technique called Romberg integration).
R1,1 So, e.g., can get up h6
precision by extrapolating
R2,1 R2,2
the trapezoidal rule
R3,1 R3,2 R3,3 calculations with spacing
h,h/2,h/4
Winter Semester 2006/7 Computational Physics I Lecture 10 11
Example
Let’s investigate a concrete example:
K = 0 / 2 1 k 2 sin 2 d
We know the exact value for some k, e.g., k=1 K=1.0
Romberg Integration (3 step)
i
1 0.785398126
2 0.94805944 1.00227988
3 0.987115741 1.00013447 0.999991417
j 1 2 3
Trapezoidal rule used 3 times
Winter Semester 2006/7 Computational Physics I Lecture 10 12
Example
Next example:
K = 02 4 x 2 dx =
Convergence much
slower because of infinite
slope at x=2
d 4 x 2 1 2x x
= = x
2
dx 2 4 x 2
4 x 2
Winter Semester 2006/7 Computational Physics I Lecture 10 13
Magnetic Field
x
Current I flowing in wire.
Biot-Savart Law:
μ 0 I dz r r
dB =
4 r 3
dz r = rdz sin yˆ z
-L dz +L
L μ I r sin L μ I x
B = Bˆy, B = 0
dz =
0
dz
L 4 L 4 (x 2 + z 2 ) 2
3 3
r
μ0I
Consider L, x small: Ampere’s Law gives B = where r is
2r
the distance from the wire. Try our numerical calculations.
Winter Semester 2006/7 Computational Physics I Lecture 10 14
Magnetic Field
Note that Ampere’s law applied to an infinite length wire. To
reproduce the result numerically, need to have L>>r.
x 1 x 1
r = x = 1, 3 2 = 3 2 , 3 2 = 3 2
x + z min 1 + L x + z max 1 + 0
2 2 2
x
3 2
x + z 2 min 1
= 3 2
x 1 + L2
3 2
x + z 2 max
23 13
for this ratio , L > 1
For L = 1000, = 10 9
Winter Semester 2006/7 Computational Physics I Lecture 10 15
Magnetic Field
Step too big
Take x=1, L=1000
Rounding errors
Winter Semester 2006/7 Computational Physics I Lecture 10 16
Magnetic Field
Try double precision: L=10000
Winter Semester 2006/7 Computational Physics I Lecture 10 17
Gaussian Quadrature
We now move away from the requirement of equidistant points
for the integration. General technique called Gaussian
quadrature which yields polynomial accuracy of order (2n-1)
using nth order polynomials.
Consider the scalar product of two functions in the interval [-1,1]
fg = 1
1
f (x)g(x) dx
Now consider an orthogonal set of polynomials on this interval,
such that
Pi P j = ij
E.g., Legendre polynomials (1,x,x2-1/3,…)
Winter Semester 2006/7 Computational Physics I Lecture 10 18
Gaussian Quadrature
Assume we have a polynomial p(x) of order 2n-1. We can
interpolate this polynomial at n points xi using our usual Lagrange
method of order n-1: n
p˜ (x) = L j (x) p(x j )
j =1
We can rewrite p(x) as
p(x) = p˜ (x) + (x x1 )(x x 2 )(x x n )q(x)
where q(x) is a polynomial of degree n -1
If we pick the x i as the roots of an n th order polynomial
from an orthogonal set (e.g., Legendre Polynomials), then
Pn (x) = (x x1 )(x x 2 )(x x n )
n 1
1
1 Pn (x)q(x) =0 since q(x) = a j P j (x)
j=0
Winter Semester 2006/7 Computational Physics I Lecture 10 19
Gaussian Quadrature
n n
So,
1
1 p(x) dx = 1
1
p˜ (x) dx = 1
1
L j (x) p(x j ) dx = w j p(x j )
j =1 j =1
where the wj are determined by the choice of xj which in turn
come from the choice of orthogonal set of polynomials.
w j = 1
1
L j dx
Note what we have achieved: 2n-1 accuracy using a polynomial of
degree n by picking sampling points in special way.
Procedure (Legendre polynomials):
1. Choose which order you want to use (n)
2. Find the n roots of Pn (look them up in a table)
3. Find the corresponding Lagrange Polynomials
4. Calculate the weight factors
6. Evaluate the integral
Winter Semester 2006/7 Computational Physics I Lecture 10 20
Gaussian Quadrature
E.g., use Legendre polynomials, n=2
1 1
P2 (x) = x with roots x1 =
2
3 x2 = 13
3
The Lagrange polynomials are
(x x0 )(x xi 1 )(x xi +1 )(x xn )
Li (x) =
(xi x0 )(xi xi 1 )(xi xi +1 )(xi xn )
x+ 1 x+ 1
(x x 2 ) 3 3
L1 = = = w1 = 1
1
L1 dx = 1
(x1 x 2 ) (
1 1
3 3
2 1
)3
x 13
L2 = w 2 = 1
1
L2 dx = 1
2 1 3
Note that these can be used for any integrand you want to
evaluate
Winter Semester 2006/7 Computational Physics I Lecture 10 21
Gaussian Quadrature
So, this gives the 2-point rule: 1
1 ( ) ( 1 3)
f (x) dx f 1 3 + f
For different integration limits, make a change of variables:
a+b b a
x= + u
2 2
a + b b a b a
a f (x) dx = 1 f +
b 1
u du
2 2 2
for our example
b a a + b b a 1 a +b b a 1
b
a f (x) dx f + f +
2 2 2 3 2 2 3
/2 /2 /2 /2 1 /2 /2 1
e.g., K = 1 sin 2 d cos + cos + 3
0
2 2 2 3 2 2
= [0.9454092 + 0.3258856] = 0.9984726
4
Winter Semester 2006/7 Computational Physics I Lecture 10 22
Gaussian Quadrature
Of course, can apply composite Gaussian Quadrature, or use a
higher order Legendre polynomial.
n xi wi
1 0 2
2 ±1/3 1
0 8/9
3
±3/5 5/9
±0.339981 0.652145
4
±0.861136 0.347855
0 0.568889
5 ±0.538569 0.478629
±0.906180 0.236927
Winter Semester 2006/7 Computational Physics I Lecture 10 23
Gaussian Quadrature
One can also choose different sets of orthogonal polynomials by
breaking up the integral as follows ab w(x) f (x) dx
Interval w(x) Orthogonal Polynomials
[-1,1] 1 Legendre
(-1,1) (1-x)(1-x) ,>-1 Jacobi
(-1,1) 1/(1-x2) Chebyshev (first kind)
[-1,1] (1-x2) Chebyshev (second kind)
[0,) e-x Laguerre
a w(x)PN (x)PM (x) dx = 0 MN
b
Orthogonality:
The weights are given by w i = a w(x)Li (x) dx
b
Winter Semester 2006/7 Computational Physics I Lecture 10 24
Field due to Current Loop
Example: the magnetic field due to a current loop:
z
. x,y,z
L
r
r )
Magnetic field can be calculated analytically for points along the
axis, but not for other points. Magnetic field will have components
in all directions. Need to calculate them one at a time.
Winter Semester 2006/7 Computational Physics I Lecture 10 25
Field due to a Current Loop
Calculation of the magnetic field due to a current loop at a point P
off the axis of the loop. Following integrals appear:
x 5 Note singularity
I= 1
1 3
dx take a =
(a x) 2 1 x 2 4 at endpoints !
1
Note that 2 already appears in integral, so we use this as
1 x
the weight function and use the roots of the Chebyshev
polynomials of the first kind. For n=4, these are
x1 = cos 8 , x 2 = cos 3 8 , x 3 = cos 3 8 , x 4 = cos 8
giving
w1 = w 2 = w 3 = w 4 = 4
so that I w1 f (x1 ) + w 2 f (x 2 ) + w 3 f (x 3 ) + w 4 f (x 4 ) = 5.02
x
where f (x) = 3 Correct answer 5.33
Winter Semester 2006/7 (a x) 2 Computational Physics I Lecture 10 26
Exercizes
1. Compare the Trapezoidal rule, Simpson’s rule, the Romberg
extrapolation method, and Gaussian quadrature with
Legendre Polynomials for the following integral (try different
values of m) for different step sizes:
d
8 0 m
cos cos m
Winter Semester 2006/7 Computational Physics I Lecture 10 27
