Lecture 1

Introduction to Numerical Methods

T. Gambill

Department of Computer Science

University of Illinois at Urbana-Champaign

June 16, 2014

T. Gambill (UIUC) CS 357 June 16, 2014 1 / 53

Course Info
http://courses.engr.illinois.edu/cs357/su2014/

Book: Numerical Methods Design, Analysis, and

Computer Implementation of Algorithms, by Anne
Greenbaum & Timothy P. Chartier

Contact Information: TA:

Sweta Yamini Seethamaraju
Office: 2209 SC email: ta1cs357 AT cs DOT illinois
Office Hour: 1:00-2:00 PM Monday or DOT edu
by appointment Office Hour: tbd
email: gambill AT illinois DOT edu Office Hour Location: 209 Siebel
Center
T. Gambill (UIUC) CS 357 June 16, 2014 2 / 53
Assessment

http://courses.engr.illinois.edu/cs357/su2013/
Compass quizzes 150 points maximum
MPs 150 points maximum
Grades:
Midterm Exam 150 points
Final Exam 250 points
Homework:
no dropped scores
May discuss MPs with TAs or other students, but do not copy!
I copied partial solutions is still copying
I see departmental policy re: cheating (
https://agora.cs.illinois.edu/display/undergradProg/Honor+Code)

T. Gambill (UIUC) CS 357 June 16, 2014 3 / 53

Finally. . .

http://courses.engr.illinois.edu/cs357/su2013/

Schedule and Notes:

Final exam is on Friday August 8th time and location: TBA

Questions?

T. Gambill (UIUC) CS 357 June 16, 2014 4 / 53

Topics
Section 1: What is Numerical Analysis / Numerical Methods?
Section 2: Numbers
Section 3: IEEE754
Section 4: Errors-Data, Roundoff,Truncation,machine epsilon(m ),unit
roundoff(µ)
Section 5: Operations on Numbers
Section 6: Functions

T. Gambill (UIUC) CS 357 June 16, 2014 5 / 53

Section 1: What is Numerical Analysis / Numerical
Methods?

Definition
Numerical Analysis - The study of algorithms (methods) for problems involving
quantities that take on continuous (as opposed to discrete) values.

T. Gambill (UIUC) CS 357 June 16, 2014 6 / 53

Numerical Calculation vs. Symbolic Calculation

Numerical Calculation: involve numbers directly

I manipulate numbers to produce a numerical result
Symbolic Calculation: symbols represent numbers
I manipulate symbols according to mathematical rules to produce a symbolic
result

Example (numerical) Example (symbolic)

2
(17.36) − 1 x2 − 1
= 16.36 =x−1
17.36 + 1 x+1

T. Gambill (UIUC) CS 357 June 16, 2014 7 / 53

Analytic Solution vs. Numerical Solution

Analytic Solution (a.k.a. symbolic): The exact numerical or symbolic

representation of the solution
I may use special characters such as π, e, or tan (83)
Numerical Solution: The computational representation of the solution
I entirely numerical

Example (analytic)
Example (numerical)
1
4 0.25
1 0.33333 . . . (?)
3 3.14159 . . . (?)
π
0.88472 . . . (?)
tan (83)

T. Gambill (UIUC) CS 357 June 16, 2014 8 / 53

Numerical Computation and Approximation

Numerical Approximation is needed to carry out the steps in the

numerical calculation. The overall process is a numerical computation.

Example (symbolic computation, numerical solution)

1 1 1 1
+ + −1= = 0.083333333 . . .
2 3 4 12

Example (numerical computation, numerical approximation)

0.500 + 0.333 + 0.250 − 1.000 = 0.083

T. Gambill (UIUC) CS 357 June 16, 2014 9 / 53

Method vs. Algorithm vs. Implementation

Method: a general (mathematical) framework describing the solution

process
Algorithm: a detailed description of executing the method
Implementation: a particular instantiation of the algorithm

Is it a “good” method?
Is it a robust (stable) algorithm?
Is it a fast implementation?

T. Gambill (UIUC) CS 357 June 16, 2014 10 / 53

The Big Theme

Accuracy Cost

T. Gambill (UIUC) CS 357 June 16, 2014 11 / 53

Numerical Analysis

Definition (Trefethen)
Study of algorithms for the problems of continuous mathematics

We’ve been doing this since Calculus (and before!)

Riemann sum for calculating a definite integral
Newton’s Method
Taylor’s Series expansion + truncation

T. Gambill (UIUC) CS 357 June 16, 2014 12 / 53

History: Numerical Algorithms

date to 1650 BCE: The Rhind Papyrus of ancient Egypt contains 85

problems; many use numerical algorithms (T. Chartier, Davidson)

Approximates π with
(8/9)2 ∗ 4 ≈ 3.1605

T. Gambill (UIUC) CS 357 June 16, 2014 13 / 53

History: Archimedes

287-212BC developed the ”Method of

Exhaustion”
Method for determining π
I find the length of the permieter of a
polygon inscribed inside a circle of
radius 1/2
I find the permiter of a polygon
circumscribed outside a circle of
radius 1/2
I the value of π is between these two
lengths

T. Gambill (UIUC) CS 357 June 16, 2014 14 / 53

History: Method of Exhaustion

A circle is not a polygon

A circle is a polygon with an infinite number of sides
Cn = circumference of an n-sided polygon inscribed in a circle of radius
1/2
limn→∞ = π
Archimedes deterimined
223 22
<π <
71 7
3.1408 <π < 3.1429

two places of accuracy....

http://www.pbs.org/wgbh/nova/archimedes/pi.html

T. Gambill (UIUC) CS 357 June 16, 2014 15 / 53

History: Method of Machin

Around 1700, John Machin discovered the trig identity

1 1
π = 16 arctan ( ) − 4 arctan ( )
5 239
Led to calculation of the first 100 digits of π
Uses the Taylor series of arctan in the algorithm

x3 x5 x7
arctan (x) = x − + − ...
3 5 7
Used until 1973 to find the first Million digits

T. Gambill (UIUC) CS 357 June 16, 2014 16 / 53

Big Questions

How algorithms work and how they fail

Why algorithms work and why they fail

Connects mathematics and computer science

Need mathematical theory, computer programming, and scientific inquiry

T. Gambill (UIUC) CS 357 June 16, 2014 17 / 53

Numerical Analysis

A Numerical Analyst needs

computational knowledge (e.g. programming skills)
understanding of the application (physical intuition for validation)
mathematical ability to construct and meaningful algorithm

T. Gambill (UIUC) CS 357 June 16, 2014 18 / 53

Numerical Analysis

Numerical focus:
Approximation An approximate solution is sought. How close is this to the
desired solution?
Efficiency How fast and cheap (memory) can we compute a solution?
Stability Is the solution sensitive to small variations in the problem setup?
Error What is the role of finite precision of our computers?

T. Gambill (UIUC) CS 357 June 16, 2014 19 / 53

Numerical Analysis

Why?
Numerical methods improve scientific simulation
Some disasters attributable to bad numerical computing (Douglas Arnold)

I The Patriot Missile failure, in Dharan, Saudi Arabia, on February 25, 1991
which resulted in 28 deaths, is ultimately attributable to poor handling of
rounding errors.
I The explosion of the Ariane 5 rocket just after lift-off on its maiden voyage off
French Guiana, on June 4, 1996, was ultimately the consequence of a
simple overflow.
I The sinking of the Sleipner A offshore platform in Gandsfjorden near
Stavanger, Norway, on August 23, 1991, resulted in a loss of nearly one
billion dollars. It was found to be the result of inaccurate finite element
analysis.

T. Gambill (UIUC) CS 357 June 16, 2014 20 / 53

Section 2: Numbers

Sets of Numbers
Natural Numbers = N = {1, 2, 3, ...}
Integers = Z = {0, ±1, ±2, ±3, ...}
Rationals = Q = {a/b | a ∈ Z, b ∈ Z, b , 0}
Reals = R = {±dn dn−1 ...d2 d1 .d−1 d−2 ... | n ∈ N and dj ∈ {0, 1, 2..., 9}, j =
n, n − 1, n − 2, ..., 1, −1, −2, ...}
n-tuples of Reals = Rn = {(r1 , r2 , ..., rn ) | ri ∈ R and n ∈ N}
Complex Numbers = C = {(a, b) = a + bi | a ∈ R, b ∈ R, i2 = −1}
Extended Reals = R = R ∪ ±∞.
Interval Numbers = IR = {[a, b] | a 6 b, a ∈ R , b ∈ R}

T. Gambill (UIUC) CS 357 June 16, 2014 21 / 53

Section 3: Floating Point Numbers, Precision

Computer Representation of Real Numbers

Normalized Floating Point Numbers
F(β, t, L, U) =
{±βe (d0 .d1 d2 ...dt−1 ) | 0 6 di 6 β − 1, i = 0, ..., t − 1, L 6 e 6 U}
β is called the ”base”
t is called the ”precision”
L,U represent the range of the exponent
d0 .d1 d2 ...dt−1 is called the mantissa or significand
if d0 = 0 then the number is zero (Normalization)
e is called the exponent

T. Gambill (UIUC) CS 357 June 16, 2014 22 / 53

Computer Representation of Real Numbers
(Toy)Normalized Binary Floating Point Numbers
F(10, 2, −1, 1)
= {±10e (d0 .d1 ) | 0 6 di 6 9, i = 0, 1, − 1 6 e 6 1}
= {±10e (d0 + d1 /10) | 0 6 di 6 9, i = 0, 1, − 1 6 e 6 1, d0 =
0 implies number is zero}

Problem: Write out all numbers on the number line

Hint:

0.d0 d1 e = −1
d0 .d1 e=0
d0 d1 .0 e=1

T. Gambill (UIUC) CS 357 June 16, 2014 23 / 53

Python double precision real numbers

Computer Representation of Real Numbers

For Python floating point numbers:
β = 2, t = 53, L = −1022, U = 1023 so an individual floating point number has
the bit form,

s1 e1 e2 ...e11 d1 d2 ...d52
as seen from the picture in memory

T. Gambill (UIUC) CS 357 June 16, 2014 24 / 53

Python representation of real numbers

Computer Representation of Real Numbers

To convert the bit string s1 e1 e2 ...e11 d1 d2 ...d52 to a decimal (base ten) real
number,
1 Convert e1 e2 ...e11 to a decimal number E.
2 Multiply 2E−1023 with 1.d1 d2 ...d52 (just shift the decimal point E − 1023 bits)
and convert this to a decimal real number R.
3 Compute (−1)s1 ∗ R
The above sequence can be compactly expressed in the formula,

number = (−1)s1 ∗ (1.d1 d2 ...d52 ) ∗ 2E−1023 where 0 < E < 2047

If we write 1.d1 d2 ...d52 as 1.f then the above expression can be written in the
compact form,

number = (−1)s1 ∗ 1.f ∗ 2E−1023 where 0 < E < 2047

T. Gambill (UIUC) CS 357 June 16, 2014 25 / 53

What happened to zero?

”Special” numbers
Special numbers use values of E = 0 or f = 0.
For example, zero is represented by E = 0, f = 0
(−1)s 2−1022 (0.0)

s 0...0 0...0
bit : 63 bits : 62 ← 52 bits : 51 ← 0
subnormal(denormalized) numbers by E = 0, f , 0
(−1)s 2−1022 (0.f )

s 0...0 f
bit : 63 bits : 62 ← 52 bits : 51 ← 0
Infinity is represented by E = 2047, f = 0
(−1)s 21024 (1.0)
NaN is represented by E = 2047, f , 0
(−1)s 21024 (1.f )

T. Gambill (UIUC) CS 357 June 16, 2014 26 / 53

Test your understanding

Which values in Q are exactly representable with a finite binary

decimal expansion?
Hint: Use the Fundamental Theorem of Arithmetic: Any integer greater
than 1 can be written as a unique product (up to ordering of the factors)
of prime factors.
Is 0.1 exactly representable in Python?
How do we display the value Python assigns to 0.1?

Are floating point values equally spaced?

For a toy Normalized Binary Floating point system F(2, 3, −1, 1) =
{(−1)s 2e (1.d1 d2 ) | s = 0 or s = 1 , di = 0 or 1, L 6 e 6 U}
are the values equally spaced? (We are ignoring ”special” numbers.)

T. Gambill (UIUC) CS 357 June 16, 2014 27 / 53

Test your understanding

What are the largest and smallest(positive both subnormal and

not subnormal) IEEE numbers?
Check your answer by typing:
import numpy as np
np.finfo(np.float).tiny
np.finfo(np.float).max
at the Python prompt.

T. Gambill (UIUC) CS 357 June 16, 2014 28 / 53

IEEE representable numbers

Overflow/Underflow
computations too close to zero may result in underflow
computations too large may result in overflow
overflow error is considered more severe
underflow can just fall back to 0
T. Gambill (UIUC) CS 357 June 16, 2014 29 / 53
 Test your understanding

Example
Effects of spacing of floating point values
1 first = []
2 a=1.0
3 while a > 0.0:
4 a = a/2.0
5 first. append (a)
6 second = []
7 a=1.0
8 while a+1.0 > 1.0:
9 a = a/2.0
10 second . append (a)
11 print(’len( first) =’, len(first))
12 print(’first [ -1] = ’, first [ -1])
13 print(’len( second ) = ’,len( second ))
14 print(’second [-1] = ’, second [ -1])

T. Gambill (UIUC) CS 357 June 16, 2014 30 / 53

Fraction Algorithm
An algorithm to compute the binary representation of a fraction x:

x = 0.b1 b2 b3 b4 . . .
= b1 · 2−1 + . . .

Multiply x by 2. The integer part of 2x is b1

2x = b1 · 20 + b2 · 2−1 + b3 · 2−2 + . . .

Example
Example:Compute the binary representation of 0.625

2 · 0.625 = 1.25 ⇒ b1 = 1
2 · 0.25 = 0.5 ⇒ b2 = 0
2 · 0.5 = 1.0 ⇒ b3 = 1

So (0.625)10 = (0.101)2

T. Gambill (UIUC) CS 357 June 16, 2014 31 / 53

Section 4: Errors

Data Error
Data Error can occur when making a measurement of a physical quantity. But
Data Error can also occur by means of a data entry error.

Numerical Error: Roundoff

Roundoff occurs when digits in a decimal point (0.3333...) are lost (0.3333)
due to a limit on the memory available for storing one numerical value.

Numerical Error: Truncation

Truncation error occurs when approximations such as truncating an infinite
series, replacing a derivative by a finite difference quotient, replacing an
arbitrary function by a polynomial, or terminating an iterative sequence before
it converges.

T. Gambill (UIUC) CS 357 June 16, 2014 32 / 53

Floating Point Errors

Not all reals can be exactly represented as a machine floating point

number. Then what?
Roundoff error
IEEE options:
I Round to next nearest FP (preferred), Round to 0, Round up, and Round
down
I We will use the notation fl(x) to denote the floating point value representing
a real number x.
Let x+ and x− be the two floating point machine numbers closest to x
round to nearest: round(x) = x− or x+ , whichever is closest
round toward 0: round(x) = x− or x+ , whichever is between 0 and x
round toward −∞ (down): round(x) = x− (used in representing interval
numbers)
round toward +∞ (up): round(x) = x+ (used in representing interval
numbers)

T. Gambill (UIUC) CS 357 June 16, 2014 33 / 53

 m

Machine Epsilon
The machine epsilon m is the smallest positive machine number such that

1 + m , 1

The double precision machine epsilon is 2−52 .

1 >> eps
2 ans = 2.2204e -16

T. Gambill (UIUC) CS 357 June 16, 2014 34 / 53

Measuring Error - Accuracy
Here we define the absolute error:

Absolute Error(xa ) = xa − xt
= approximate value − true value

This doesn’t tell the whole story. For example, if the values are large, like
billions, then an Error of 100 is small. If the values are smaller, say
around 10, then an Error of 100 is large. We need the relative error:
xa − xt
Relative Error(xa ) =
xt
approximate value − true value
=
true value
when true value , 0

After some simple algebra on the previous equation above we can write,

approximate value = true value ∗ (1 + Relative Error)

T. Gambill (UIUC) CS 357 June 16, 2014 35 / 53

Floating Point Errors

How big is roundoff error?

Suppose x is a real number and fl(x) = x− where x− is not a subnormal
floating point number or zero. We further assume that rounding is performed
by ”rounding to nearest”.

x = (1.d1 d2 d3 . . . d52 . . . )2 × 2e
x− = (1.d1 d2 . . . d52 )2 × 2e
x+ = (1.d1 d2 . . . d52 )2 + 2−52 × 2e

|x+ − x− |
|x− − x| 6 = 2e−53
2
 x− − x  2e−53
 
−53
 x  6 2e = 2 = m /2
 

Unit Roundoff Error

unit roundoff error = µ = m /2

T. Gambill (UIUC) CS 357 June 16, 2014 36 / 53

Floating Point Errors - Accuracy

From the previous slides

fl(x) = x ∗ (1 + Relative Error)
 
 fl(x) − x 
 6µ
 x 

We have established that for x ∈ R and fl(x) not subnormal,

fl(x) = x ∗ (1 + δ(x)) where |δ(x)| 6 µ

T. Gambill (UIUC) CS 357 June 16, 2014 37 / 53

Relationship between error and Significant Digits
Example
Given x = 123.01234 and y = 123.0123 is an approximation of x then

|y − x| = 0.00004 = 0.4 ∗ 10−4

|y − x|
6 3.26 ∗ 10−7
|x|

From the above example we would like to make the following assertions:
If absolute error is less than 0.5 ∗ 10−t then there are t equal digits to the
right of the decimal point between y and x, when both numbers are in the
non-scientific notation form,

y = d1 d2 ...dn .dn+1 dn+2 ...dn+t ...

If the relative error is less than 5.0 ∗ 10−t then there are t equal digits
digits total between y and x when both numbers are in the scientific
notation form, .e.g. no leading zeros)
x = 0.d1 d2 ...dt ... ∗ 10e
T. Gambill (UIUC) CS 357 June 16, 2014 38 / 53
Relationship between error and Significant Digits
However, the following example shows that the above assertions are not
strictly correct.

Example
Given x = 0.00351 and an approximation y = .00346 then

|y − x| = 0.00005 = 0.5 ∗ 10−4

|y − x|
6 1.43 ∗ 10−2
|x|

Now x and y agree in three digits to the right of the decimal but the first
assertion says it should be four. However if we rounded (to nearest) the fifth
decimal digits of x and y then the assertion would be true.
If we re-write x = 0.351 ∗ 10−2 and y = 0.346 ∗ 10−2 these numbers agree only
with one digit but the assertion says it should be two. Again, however, if we
round (to nearest) the third decimal digits of x and y then the assertion would
be true. How can we overcome this discrepancy in our assertions?

T. Gambill (UIUC) CS 357 June 16, 2014 39 / 53

Significant Digits - Accuracy

We overcome the previous problem be re-defining ”equal digits” as follows:

Significant Decimal Digits

If y = d1 d2 ...dn .dn+1 dn+2 ...dn+t ... is an approximation of x and we have
|y − x| 6 0.5 ∗ 10−t then the t digits starting in the position > 10−t of y are
called ”significant decimal digits”.

Significant Digits
|y−x|
If y is an approximation of x = 0.d1 d2 ...dt ... ∗ 10e and we have |x| 6 5.0 ∗ 10−t
then the t digits starting in the position > 10−t of y are called ”significant
digits”.

T. Gambill (UIUC) CS 357 June 16, 2014 40 / 53

Our Job
Given a specific problem. A numerical analyst will do the following:
Determine the condition of the problem.
I A problem is ill-conditioned if small changes to input values create large
errors in the solution, assuming that our implementation is mathematically
perfect, i.e. does not introduce round-off or truncation errors.
I A problem is well-conditioned if it is not ill-conditioned.
Choose a method and specific algorithm that is stable.
I An algorithm is stable if it achieves the level of accuracy defined by the

condition of the problem.

Implement the algorithm so that the calculation is not susceptible to large
roundoff error and the approximation has a tolerable truncation error.

How?
Compute the condition ”number” for the problem.
incorporate roundoff-truncation knowledge into
I the method

I the algorithm

I the software design

awareness → correct interpretation of results

T. Gambill (UIUC) CS 357 June 16, 2014 41 / 53
Section 5: Operations on Numbers

For C - Complex Numbers

(a + bi) + (c + di) = (a + c) + (b + d)i.
(a + bi) − (c + di) = (a − c) + (b − d)i.
(a + bi) ∗ (c + di) = (ac − bd) + (bc + ad)i.
a+bi c−di ac+bd bc−ad
(a + bi)/(c + di) = c+di c−di = c2 +d2 + c2 +d2 i where c2 + d2 , 0.

T. Gambill (UIUC) CS 357 June 16, 2014 42 / 53

C - Complex Numbers

An alternative view
 
a b
a + bi ↔ .
−b a

Example
(2 + 3i) ∗ (−1 − 4i) = 10 − 11i
since      
2 3 −1 −4 10 −11
∗ =
−3 2 4 −1 11 10

T. Gambill (UIUC) CS 357 June 16, 2014 43 / 53

Operators
For IR - Interval Numbers
Where ◦ denotes the operators +, −, ∗, /.
[a, b] ◦ [c, d] = {x ◦ y | x ∈ [a, b], y ∈ [c, d] }.

Example
[−1, 1] + [−0.1, 0.1] = [−1.1, 1.1]

Example
[2, 3] − [2, 3] = [−1, 1] thus x − x , 0 except for intervals of zero width, e.g.
[3, 3].

Example
[−1, 2] ∗ ([−2, 3] + [3, 4]) = [−1, 2] ∗ [1, 7] = [−7, 14] but
[−1, 2] ∗ [−2, 3] + [−1, 2] ∗ [3, 4] = [−4, 6] + [−4, 8] = [−8, 14] so the distributive
law doesn’t hold.

T. Gambill (UIUC) CS 357 June 16, 2014 44 / 53

Operators

For Floating Point Numbers

Denote the double precision floating point numbers F(2, 53, −1022, 1023) as
DPFP. We will assume that the following holds where ◦ denotes the operators
+, −, ∗, /.
fl(x ◦ y) = (x ◦ y)(1 + δ) where x, y ∈ DPFP and |δ| 6 µ.

Example
x ◦ y need not be in DPFP even if both x, y ∈ DPFP. Consider x = 1, y = 3 and
x/y

Example
Note that fl(fl(x + y) + z) = fl(x + fl(y + z)) fails for some choice of
x, y, z ∈ DPFP. For example, when x = 1.111...100 ∗ 20 1.—fifty bits of 1’s
followed by two bits of 0’s and y = z = 1.0...0 ∗ 2− 53.

T. Gambill (UIUC) CS 357 June 16, 2014 45 / 53

Fields

The Rationals, Reals and Complex Numbers form a field based on the
operations + , ∗.

Properties of a Field F
closure: If a ∈ F and b ∈ F then a + b ∈ F and a ∗ b ∈ F.
associativity: a + (b + c) = (a + b) + c and a ∗ (b ∗ c) = (a ∗ b) ∗ c
for all a ∈ F, b ∈ F, c ∈ F.
commutativity: a + b = b + a and a ∗ b = b ∗ a for all a ∈ F, b ∈ F.
additive and multiplicative identity: a + 0 = a and a ∗ 1 = a for all a ∈ F.
additive and multiplicative inverses: a + (−a) = 0 and b ∗ (1/b) = 1
for all a ∈ F and b ∈ F, b , 0.
distributivity: a ∗ (b + c) = a ∗ b + a ∗ c for all a ∈ F, b ∈ F, c ∈ F.

T. Gambill (UIUC) CS 357 June 16, 2014 46 / 53

Completeness of the Real Numbers

Upper Bound
Given any non-empty set S ⊂ R then we say that S has an upper bound if
there exists a real number r ∈ R with the property that for any s ∈ S then s 6 r.

Least Upper Bound or Supremum

Given any non-empty set S ⊂ R that has an upper bound then we say that S
has a least upper bound if there exists an upper bound r0 ∈ R for S with the
property that for any upper bound r for S then r0 6 r.

The Real Numbers satisfy the Dedekind Completeness Property.

Dedekind Completeness
For any non-empty set S ⊂ R that has an upper bound, then S has a least
upper bound.

T. Gambill (UIUC) CS 357 June 16, 2014 47 / 53

Section 6: Functions

Definition
A function is an ordered triple of sets f = (Domain, CoDomain, Graph)
where Graph = {(x, y) | x ∈ Domain, y ∈ CoDomain} and
Graph has the property that if (x, y) ∈ Graph and (x, z) ∈ Graph then y = z.

The set {y | (x, y) ∈ Graph} is called the Range of the function.

If Domain = A and CoDomain = B we say that f maps A into B and write,

f :A→B

If (x, y) ∈ Graph we can write y = f (x).

Example
The triple [−1, 1], [−1, 1], {(x, y) | x2 + y2 = 1, x, y ∈ [−1, 1]} is NOT


a function,
however [−1, 1], [0, 1], {(x, y) | x2 + y2 = 1, x ∈ [−1, 1], y ∈ [0, 1]} is a function.

T. Gambill (UIUC) CS 357 June 16, 2014 48 / 53

Classification of Functions

Definition
Given a function,

f = (Domain, CoDomain, Graph)

f is called onto or surjective if CoDomain = Range.

f is called 1-1 or injective if (x1 , y), (x2 , y) ∈ Graph implies x1 = x2 .
f is called bijective if it is both surjective and injective.

Example
The function R, R, {(x, y) | y = x3 , x, y ∈ R} is a bijection.

T. Gambill (UIUC) CS 357 June 16, 2014 49 / 53

Inverse of a Function
Definition
Given a bijective function,

f = (Domain, CoDomain, Graph)

then we can define an inverse function f −1 as follows:

f −1 = (CoDomain, Domain, GraphInv) where

GraphInv = {(y, x) | x ∈ Domain, y ∈ CoDomain}

Example
The function below is a bijection.

[0, +∞), [0, +∞), {(x, y) | y = x2 , x, y ∈ [0, +∞)}

To determine it’s inverse swap x with y in the equation y = x2 to obtain, x = y2 .

√
[0, +∞), [0, +∞), {(x, y) | x = y2 (y = + x), x ∈ [0, +∞)}

T. Gambill (UIUC) CS 357 June 16, 2014 50 / 53

Inverse of a Function

Example
The function below is a bijection.
    
1 0
R , R , (x, y) | y = M ∗ x, x, y ∈ R , M =
2 2 2
2 1

To determine it’s inverse swap x with y in the equation y = M ∗ x to obtain,

x = M ∗ y (or M−1 x = y).
    
1 0
R , R , (x, y) | y = M ∗ x, x ∈ R , M =
2 2 −1 2 −1
−2 1

T. Gambill (UIUC) CS 357 June 16, 2014 51 / 53

Norm Function

Example - Measuring the ”size” of Numbers

The function below is called an `2 ”` two-norm”.
 
 X 
 v
u n
Rn , R, (x, y) | y = |x|2 = t
u
x2i 
 
i=1

The `2 function has the following properties:

|u|2 > 0 and |u|2 = 0 only if u = (0, ..., 0) ∈ Rn
|r ∗ u|2 = |r| ∗ |u|2 for all r ∈ R and u ∈ Rn (|r| is just the absolute value of r)
|u + v|2 6 |u|2 + |v|2 for all u, v ∈ Rn (triangle inequality)

T. Gambill (UIUC) CS 357 June 16, 2014 52 / 53

Outlook

Next time:
Taylor Series
Order of Convergence
Condition Number
Stability

T. Gambill (UIUC) CS 357 June 16, 2014 53 / 53