Advance Numerical Analysis

Course

Baghdad University,
Faculty of Sciences,
Department of Computer Sciences
Second Class Syllabus Academic
Second Semester
Dr. Ali H. Kashmar
Year 2018/2019
1
 Syllabus for Advance Numerical Analysis
Course (Second Semester), 2018-2019
• Second class; Computer Science Dept.; Collage of
Sciences; University of Baghdad
• Weekly hours: 4/hours:10:30 am -12:30 pm
• 2/h Theoretical:(A)(Room No.3 on Saturday)
(B)(Room No.7 on Tuesday)
• 2/h practical/MatLab No.1(Numerical Analysis
Lab):(A,G1,G2 and B, G1)8:30 am-1:30 pm.
(Sunday/(B, G2)Tuesday: 10:30-12:30) pm
• Dr. Ali H. Kashmar
2
• Room No. SC 212
 Chapter 1
Polynomial Interpolation and Extrapolation:
1. Polynomial Interpolation (Proper Method).
2. Lagrange Interpolation Polynomial .
3. Divided Differences Algorithms.
• Finite Divided Difference (F.D.D).
• Forward Finite Difference (F. F. D.)
• Backward Finite Difference (B. F. D.)
• Central Difference (C. D.)
4. Least Square Method.(L.S.M)

3
 Chapter 2
Integration Methods
1. Triangular Method.
2. Trapezoidal Method.
3. Simpson Method
4. Lagrange Method

4
 Chapter 3
Numerical Solutions of Ordinary Differential
Equations (O.D.E)

1. Initial Value Problems (I.V.P.).

2. Boundary Value Problems (B.V.P.).
3. Euler Method.
4. Modified Euler Method.
5. Taylor’s Method.

5
 References
• Curtis F. Gerald and Patrick O. Wheatly,2016,
“Applied Numerical Analysis”, 4th Edition,
• William S. Dorn and Daniel D.Mc Craben, 2014,
“Numerical Methods with Matlab”, John Wiley
and Sons.
• Curtio F. Gerald, 2012, “Applied methods for
Mathematics”, Science and Engineering, Prentice
Hall of India Private limited New Delhi.
• Kendall Atkenson, 2010, “Elementary
Numerical Analysis”, 6John Wiley and Sons.
 Lecture 1
Introduction to Polynomial
Interpolation and Extrapolation

 What are Interpolation and Extrapolation Methods?

 Why do we need them?

7
 Polynomial Interpolation and
Extrapolation Methods
Interpolation and Extrapolation Methods:
Algorithms that are used to obtain numerical solutions of a
mathematical problems.
Why do we need them?
1. No analytical solutions exists,
2. An analytical solutions are difficult to obtain
or not practical.

8
 What do we need?
Basic Needs in the Interpolation and Extrapolation Methods:
– Practical:
Can be computed in a reasonable amount of time.
– Accurate:
• Good approximate to the true value,
• Information about the approximation error (Bounds,
error order,… ).

9
Given data points
Obtain a function, P(x)
P(x) goes through the data points
Use P(x)
To estimate values at intermediate
points
 Outlines of the Course
• Solution of nonlinear • Least Squares curve fitting
Equations • Solution of ordinary
• Solution of large systems of differential equations
linear Equations • Solution of Partial
• Interpolation/Exterpolation differential equations
• Numerical Differentiation
• Numerical Integration

11
Interpolation produces a function
that matches the given data exactly.
The function then can be utilized to
approximate the data values at
intermediate points.
Interpolation may also be used
to produce a smooth graph of a
function for which values are
known only at discrete points,
either from measurements or
calculations.
Given data points
Obtain a function, P(x)
P(x) goes through the data points
Use P(x)
To estimate values at intermediate
points
Given data points:
At x0 = 2, y0 = 3 and at x1 = 5, y1 = 8
Find the following:
X=4
1) At x = 4, y = ? (inter) Y=?

X=6
2) At x=6, y=? (extra) Y=?
y
P(x)

2 4 5
x
Interpolation as well as extrapolation are the
methods that can be used to calculate the
value of a function at specific points which is
(the function) value unknown.
The main idea is to use the information from
the known points to construct a mathematical
model (equation) and use it to calculate at
unknown points.

The form of extracted function will be a

polynomial f(x).
The general form of a single variable polynomial can
be given by:
f x   a 0  a1 x  a2 x  a3 x  ...  an1 x
2 3 n 1

If there exist (n) points (given pints), then we can

write the following:
pn ( x0 )  f x0   a 0  a1 x0  a2 x0  a3 x0  ...  an1 x0
n 1
 y0
2 3

pn ( x1 )  f x1   a 0  a1 x1  a2 x1  a x  ...  a x
n 1
 y1
2 3
3 1 n 1 1
pn ( x2 )  f x2   a 0  a1 x2  a2 x2  a x  ...  a x
n 1
 y2
2 3
3 2 n 1 2

pn ( xm )  f xm   a 0  a1xm  a2 xm  a3 xm  ...  an1xm  ym

2 3 n 1
construct a mathematical model as a matrix system:

1 x0 x0
2
x0   a 0   y0 
n 1

 2 n 1     
1 x1 x1 x1   a1   y1 

. . . .  .   . 
 2 n 1     
1 xm xm xm  an 1   ym 

xA  b
We can solve this matrix system by……..
(1): Using a proper method to solve such system:
ex(1): Solve the following system of dataset
x 0 -1 2 3
y 2 11 24 41

Sol: N=4, the order of P(x) = n-1, d=4-1=3

P3 x   a0  a1 x  a2 x  a3 x
2 3

1 0 0 0  a 0   2 
1  1 1  1  a  11 Solve the system by
     
1 using GEM, a0 = 2,
1 2 4 8   a2  24 a1 = -7/3, a2 = 33/6
      and a 3 = -7/6
1 3 9 27   a3   41
P3 x   2  (7 / 3) x  (33 / 6) x  (7 / 6) x
2 3
 x 0 1 2
y 1 2 5

Sol: N=3, the order of P(x) = n-1, d= 3-1=2

P2 x   a0  a1 x  a2 x 2

1 0 0 a0  1
1 1 1  a   2
  1   
1 2 4 a2  5
Solve the system by using GEM
a0 = 1, , a1 = 0 and a2 = 1
ex(3): solve the following system of dataset
x 0 1 2 3
y 1 2 5 10

Sol: N=4, the order of P(x) = n-1, d=4-1=3

P3 x   a0  a1 x  a2 x  a3 x2 3

1 0 0 0  a 0   1  Solve the system by

1     
1 1 1   a1   2  using GEM we have
  a0 = 1, a1 = 0, a2 = 1
1 2 4 8   a2   5  and a3 = 0
    
1 3 9 27  a3  10

P3 x   1  (0) x  (1) x  (0) x  1  x

2 3 2
ex(4): Find f(0) and f(6) from the following system
of dataset: x -1 2 3
y 3 1 2

Sol: N=3, the order of P(x) = n-1, d=3-1=2

P2 x   a0  a1 x  a2 x 2

1  1 1 a0  3
1 2 4  a   2 Solve the system by using
      a = -13/12, and a = 5/12
1 GEM, we have: a0 = 3/2,
1 3 9 a2  1 1 2

P2 x   3 / 2  (13 / 12) x  (5 / 12) x 2

P2 0  3 / 2  (13 / 12)(0)  (5 / 12)(0)  3 / 2

2

P2 6  3 / 2  (13 / 12)(6)  (5 / 12)(6)  102

 Lecture 2
Lagrange Interpolation Polynomial

 What is Lagrange Interpolation Methods?

 Why do we need it?

24
 n
Pn x    f (xi ) * Li ( x)
i 0

( x  x0 )( x  x1 ).....( x  xi 1 )( xi 1)....( x  xn )

Li x  
( xi  x0 )( xi  x1 )...( xi  xi 1 )( xi  xi 1 )...( xi  xn )
n (x  x )
Li x   
j

i  j ( xi  x j )
i 0 25
 2 n (x  x j )
P2 x    f (xi ) * Li ( x) Li x   
i 0 i j ( xi  x j )
i 0
(x  x j ) x  x1 x  x2
i  0, L0  x     .
( x0  x j ) x0  x1 x0  x2
x 1 x  3 1
 .  ( x  1)( x  3)
0 1 0  3 3
(x  x j ) x  x0 x  x2
i  1, L1  x     .
( x1  x j ) x1  x0 x1  x2
x 0 x 3 1
 .  x ( x  3)
1 0 1 3 2
( x  xi ) x  x0 x  x1
i  2, L2  x     .
( x2  x j ) x2  x0 x2  x1
x  0 x 1 1
 .  ( x 2  x)
3  0 3 1 6

26
 x 0 1 3
y -1 2 7

n (x  x j )
Li x   
2
P2 x    f (xi ) * Li ( x) ( xi  x j )
i j
i 0 i 0
(x  x j ) x  x1 x  x2
i  0, L0  x     .
( x0  x j ) x0  x1 x0  x2
( x  1)( x  3) ( 2  1)(2  3) 1
  
(0  1)(0  3) 3 3
(x  x j ) x  x0 x  x2
i  1, L1  x     .
( x1  x j ) x1  x0 x1  x2
( x  0)( x  3) ( 2  0)(2  3) 2
   1
(1  0)(1  3) 2 2
( x  xi ) x  x0 x  x1
i  2, L2  x     .
( x2  x j ) x2  x0 x2  x1
( x  0)( x  1) ( 2  0)(2  1) 2 1
   
(3  0)(3  1) 6 6 3
27
 2
P2 x    f (xi ) * Li ( x)
1 1
 ( )( 1)  (1)(2)  ( )(7)
3 3
i 0 1 7 14
 2 
3 3 3

To find the equation, sub. X instead of 2

( x  0)( x  3)  x( x  3)
L0 x  
( x  1)( x  3) L1 x   
3 2 2
2
Pn x    f (xi ) * Li ( x)
( x  0)( x  1) x( x  1)
L2  x   
6 6
i 0
i j
( x  1)( x  3)  x( x  3) x( x  1)
P2 x   (1)  (2)  (7)
3 2 6
 ( x 2  4 x  3) 7 2 7
  ( x  3x)  x  x
2

3 6 6
 x 2 19
  x 1
6 6 28
 2
P2 x    f (xi ) * Li ( x)
i 0

1 1 1
 ( 1)( x  1)( x  3)  ( 2)( ) x ( x  3)  (7)( )( x 2  x )
3 2 6
1 2 19
 x  x 1
6 6

n (x  x j )
Li  x   
i j ( xi  x j )
i 0
2
Pn x    f (xi ) * Li ( x)
i 0
i j

29
 n (x  x j )
3
Li x   
P3 x    f (xi ) * Li ( x) i j ( xi  x j )
i 0 i 0

(x  x j ) x  x1 x  x2 x  x3
i  0, L0  x     . .
( x0  x j ) x0  x1 x0  x2 x0  x3
(x  x j ) x  x0 x  x2 x  x3
i  1, L1  x     . .
( x1  x j ) x1  x0 x1  x2 x1  x3
( x  xi ) x  x0 x  x1 x  x3
i  2, L2  x     . .
( x2  x j ) x2  x0 x2  x1 x2  x3
( x  xi ) x  x0 x  x1 x  x2
i  3, L3  x     . .
( x3  x j ) x3  x0 x3  x1 x3  x2

30
Ex (4): find f(4) from the flowing dataset: At x = x0
= 2, L0(x) = 1 and L1(x) = 0 and
at x = x1= 5, L0(x) = 0 and L1(x) = 1
The conditions can be satisfied if L0(x) and L1(x)
are defined in the following way.
x5 x2
L0 x   and L1 x  
25 52

x  x1 x  x0
L0 x   and L1 x  
x0  x1 x1  x0
Px   3L0 x   8L1 x 

Px   L0 x y0  L1 x y1

Lagrange Interpolating Polynomial

Px   L0 x  f x0   L1 x  f x1 

Px   L0 x  f x0   L1 x  f x1 

 x5  x2
P x    3   8
 25  52

5x  1
P x  
3
5 4 1
P4   6.333
3
The Lagrange interpolating polynomial
passing through three given points; (x0, y0),
(x1, y1) and (x2, y2) is:

Px   L0 x y0  L1 x y1  L2 x y2

Px   L0 x  f x0   L1 x  f x1 

 L2 x  f x2 
L0 x  
 x  x1 x  x2 
x0  x1 x0  x2 
At x0, L0(x) becomes 1. At all
other given data points L0(x) is 0.
L1 x  
 x  x0 x  x2 
x1  x0 x1  x2 
At x1, L1(x) becomes 1. At all
other given data points L1(x) is 0.
L2 x  
 x  x0 x  x1 
x2  x0 x2  x1 
At x2, L2(x) becomes 1. At all
other given data points L2(x) is 0.
General form of the Lagrange Interpolating
Polynomial

Px   L0 x  y0  L1 x  y1  L2 x  y2
 ...........  Ln x  yn

Px   L0 x  f x0   L1 x  f x1 

 L2 x  f x2   ...........  Ln x  f xn 
 ( x  x0 )( x  x1 )...( x  xk 1 )( x  xk 1 )...( x  xn )
Lk ( x ) 
( xk  x0 )( xk  x1 )...( xk  xk 1 )( xk  xk 1 )...( xk  xn )

( x  xi )
n
Lk ( x )  
i 0 ( x k  xi )
i k
Numerator of Lk  x 

 x  x0  x  x1  x  x2  
  x  xk 1  x  xk 1  
  x  xn 1  x  xn 
Denominator of Lk  x 

 xk  x0  xk  x1  xk  x2  
  xk  xk 1  xk  xk 1  
  xk  xn 1  xk  xn 
Ex(5): Find the Lagrange Interpolating
Polynomial using the three given points.

x0 , y0   2, 0.5

x1 , y1   2.5, 0.4
x2 , y2   4, 0.25
L0 x  
 x  x1 x  x2 
x0  x1 x0  x2 

L0 x  
 x  2.5x  4
2  2.52  4
 x  6.5 x  10
2
L1 x  
 x  x0 x  x2 
x1  x0 x1  x2 
L1 x  
 x  2 x  4
2.5  22.5  4
 x  6x  8
2

0.75
L2 x  
 x  x0 x  x1 
x2  x0 x2  x1 

L2 x  
 x  2x  2.5
4  24  2.5
x  4.5 x  5
2

3
Px   L0 x  f x0   L1 x  f x1 
 L2 x  f x2 
  
Px   x  6.5 x  10 0.5
2

  x  6x  8 
2
  0.4 
 0.75 
 x  4.5 x  5 
2
  0.25
 3 
Px   0.05 x  0.425 x  1.15
2

The three given points were taken from

the function

f x  
1
x
f 3   0.333
1
3
An approximation can be obtained
from the Lagrange Interpolating
Polynomial as:

P3  0.053  0.4253  1.15

2

 0.325
 x 1 2 3
y 0 3 8

x 2 10 15 8
f(x) 1 3 4 6

x 0.2 0.3 0.5

y 0.4 -0.1 -0.45
 Lecture 3
Divided differences Interpolation
Polynomial

 What is Divided differences Interpolation Methods?

 Why do we need it?
Newton Divided differences?

51
Divided differences and the coefficients
The divided difference of a function, f
with respect to xi is denoted as f  xi 

It is called as zeroth divided difference and is

simply the value of the function, f
at xi

f xi   f xi 
The divided difference of a function, f
with respect to xi and x i 1
called as the first divided difference, is denoted

f  xi , xi 1 

f  xi 1   f  xi 
f  xi , xi 1  
xi 1  xi
The divided difference of a function, f
x
with respect to xi , i 1 and i 2x
called as the second divided difference, is
denoted as

f  xi , xi 1 , xi 2 

f  xi 1 , xi 2   f  xi , xi 1 
f  xi , xi 1 , xi 2  
xi 2  xi
The third divided difference with respect to
xi , xi 1 , xi 2 and xi 3

f  xi , xi 1 , xi 2 , xi 3 
f  xi 1 , xi 2 , xi 3   f  xi , xi 1 , xi 2 

xi 3  xi
x f x  First Second Third
divided differences divided differences divided differences
x0 f x0 

f x1   f x0 
f x0 , x1  
x1  x0

x1 f x1  f x 0 , x1 , x 2  
f x1 , x 2   f x 0 , x1 
x 2  x0

f x2   f x1  f x1 , x 2 , x3   f x 0 , x1 , x 2 

f x1 , x2   f x 0 , x1 , x 2 , x3  
x2  x1 x3  x 0

x2 f x2  f x1 , x 2 , x3  
f x 2 , x 3   f x1 , x 2 
x3  x1

f x3   f x2  f x 2 , x3 , x 4   f x1 , x 2 , x 3 

f x2 , x3   f x1 , x 2 , x 3 , x 4  
x3  x 2 x 4  x1

x3 f x3  f x 2 , x 3 , x 4  
f x 3 , x 4   f x 2 , x3 
x4  x2

f x4   f x3  f x 3 , x 4 , x 5   f x 2 , x 3 , x 4 
f x3 , x4   f x 2 , x3 , x 4 , x5  
x 4  x3 x5  x 2

x4 f x4  f x3 , x 4 , x5  
f x 4 , x 5   f x 3 , x 4 
x5  x3

f x5   f x4 
f x4 , x5  
x5  x 4

x5 f x5 
0 -2 -3

1 1 0

2 3 22 0

3 4 57

4 6 205
i

0 0 -3

1 1 -2

2 2 1

3 3 6

4 5 22
Newton’s Interpolating Polynomials
Drive Newton’s divided difference formula
The divided difference of a function, f
with respect to xi and i 1 x
f  xi , xi 1 
f  xi 1   f  xi 
f  xi , xi 1  
xi 1  xi
f  xi 1 , xi 2   f  xi , xi 1 
f  xi , xi 1 , xi 2  
xi 2  xi
ex (2): Use Newton Divided Difference formula to find f(2)from the following
data:{(0,-10),(1,20),(4,14),(6,30)}? Sol: Make a Table of the dataset: f(2)=xm
i

0 0 -10

1 1 20

2 4 14

3 6 30
Newton’s Interpolating Polynomials

Newton’s equation of a function that

passes through two points

 x0 , y 0  and x1 , y1  is

Px   a0  a1 x  x0 
Px   a0  a1 x  x0 
Set x  x
0
P  x0   y0  a0
Set x  x1
P  x1   y1  a0  a1  x1  x0 
 y1  y 0
a1 
x1  x0
Newton’s equation of a function that passes
through three points

x0 , y0  x1 , y1  and  x2 , y2 

is
P  x   a0  a1  x  x0 
 a2  x  x0  x  x1 
To find a 2 , set x  x2
P  x2   a0  a1  x2  x0 
 a2  x2  x0  x2  x1 
 y2  y1 y1  y0

x2  x1 x1  x0
a2 
x2  x0
Newton’s equation of a function that passes
through four points can be written by adding a
fourth term .

P  x   a0  a1  x  x0 
 a2  x  x0  x  x1 
 a3  x  x0  x  x1  x  x2 
P  x   a0  a1  x  x0 
 a2  x  x0  x  x1 
 a3  x  x0  x  x1  x  x2 

The fourth term will vanish at all three

previous points and, therefore, leaving all
three previous coefficients intact.
The coefficients of Newton’s interpolating
polynomial are:

a0  f x0 
a1  f x0 , x1 
a2  f x0 , x1 , x2 

a3  f x0 , x1 , x2 , x3 

a4  f x0 , x1 , x2 , x3 , x4  and so on.

Example
Find Newton’s interpolating polynomial to
approximate a function whose 5 data points are
given below.

x f  x
2.0 0.85467
2.3 0.75682
2.6 0.43126
2.9 0.22364
3.2 0.08567
 f xi 1 , xi  f xi 2 , xi 1 , xi  f xi 3 , , xi  f xi 4 , , xi 
i xi f xi 
0 2.0 0.85467
-0.32617
1 2.3 0.75682 -1.26505
-1.08520 2.13363
2 2.6 0.43126 0.65522 -2.02642
-0.69207 -0.29808
3 2.9 0.22364 0.38695
-0.45990
4 3.2 0.08567
The 5 coefficients of the Newton’s interpolating
polynomial are:

a0  f  x0   0.85467
a1  f  x0 , x1   0.32617

a2  f  x0 , x1 , x2   1.26505

a3  f  x0 , x1 , x2 , x3   2.13363
a4  f  x0 , x1 , x2 , x3 , x4   2.02642
P  x   a0  a1  x  x0 
 a2  x  x0  x  x1 
 a3  x  x0  x  x1  x  x2 
 a4  x  x0  x  x1  x  x2  x  x3 
P  x   0.85467  0.32617  x  2.0 
-1.26505  x  2.0  x  2.3
 2.13363  x  2.0  x  2.3 x  2.6 
2.02642  x  2.0  x  2.3 x  2.6  x  2.9 

P(x) can now be used to estimate the value of the

function f(x) say at x = 2.8.
P  2.8   0.85467  0.32617  2.8  2.0 
-1.26505  2.8  2.0  2.8  2.3
 2.13363  2.8  2.0  2.8  2.3 2.8  2.6 
2.02642  2.8  2.0  2.8  2.3 2.8  2.6  2.8  2.9 

f  2.8  P  2.8  0.275

 Lecture 4
Finite Differences Interpolation
Polynomial

 What is Finite Differences Interpolation Methods?

 Why do we need it?
Forward Finite Difference
Newton Finite Forward Differences

77
Forward Finite Difference
i
0 2 5
5
1 3 10 2
7 0
2 4 17 2
9 0
3 5 26 2
11
4 6 37
Drive Formula of Newton Forward Difference
i
0 4 1
2
1 6 3 3
5 4
2 8 8 7
12
3 10 20
i
0 2 5
5
1 3 10 2
7 0
2 4 17 2
9 0
3 5 26 2
11
4 6 37
i
0 4 1
2
1 6 3 3
5 4
2 8 8 7
12
3 10 20
Theorem
Example(4):Make Table for Newton Forward Finite
Difference for the following dataset:
{(0,-3),(2,53),(4,381),(6,1221), (8,2813)}

i
0 0 -3
56
1 2 53 272
328 240
2 4 381 512 0
840 240
3 6 1221 752
1592
4 8 2813
 Homework

x 10 20 30 40 50 60 70 80
f(x) 0.984 0.939 0.866 0.766 0.642 0.500 0.342 0.173
 Lecture 5
Finite Differences Interpolation
Polynomial

 What is a Backward Finite Differences Methods?

 Why do we need it?
Newton Finite Backward Differences
Central Differences

90
Example(1)Make a Table Backward Finite Difference for the
following dataset: {(2,5),(3,10),(4,17),(5,26),(6,37)}

i
0 2 5
5
1 3 10 2
7 0
2 4 17 2
9 0
3 5 26 2
11
4 6 37
Drive Formula of Newton Backward Difference
i
0 4 1
2
1 6 3 3
5 4
2 8 8 7
12
3 10 20
Central Differences Formula
 x 4 6 8 10
f(x) 1 3 8 20

i
0 4 1
2
1 6 3 3
5 4
2 8 8 7
12
3 10 20
 Lecture 6
Least Square Methods
 What is a Least Square Methods?
 Why do we need it?
Shapes of Least Square Methods
1. Linear Form
2. Exponential Form
3. Geometrically Form
Examples

101
Least Square Methods
Shapes of Least Square Methods
 x 1 3 4 6 8 9 11 14
y 1 2 4 4 5 7 8 9

x y Xy
1 1 1 1
3 2 9 6
4 4 16 16
6 4 36 24
8 5 64 40
9 7 81 63
11 8 121 88
14 9 196 126
 x y x.y
1 -1 1 -1
0 0 1 0
1 1 2 2
9 3 3 9
 x y x.y
1 1 3 3
4 2 5 10
9 3 7 21
16 4 9 36
 Lecture 7
Least Square Methods
2. Exponential Form
3. Geometrically Form
Examples

110
2. Exponential Form
 x 0 1 2 4
f(x) 1 2 5 17

w.x x y Ln(y)=w
(0)(0)=0 0 0 1 Ln(1)=0
(0.693)(1)=0.693 1 1 2 Ln(2)=0.693
(2)(1.609) =3.218 4 2 5 Ln(5)=1.609
(4)(2.833) =11.332 16 4 17 Ln(17)=2.833
3. Geometrically Form
 x 0 1 2 4
f(x) 1 2 5 17

w.t x y Ln(y)=w Ln(x)=t

(0)(1)=0 0 1 Ln(1)=0 Ln(0)=1 1

(0.693)(0)=0 1 2 Ln(2)=0.693 Ln(1)=0 0

(1.609)(0.963) 2 5 Ln(5)=1.609 Ln(2)=0.963

=1.549
(2.833)(1.386) 4 17 Ln(17)=2.83 Ln(4)=1.386
=3.926 3