Analysis of
Algorithm
LECTURE 4
INSTRUCTOR: DR . HUSNAIN ASHFAQ
Lecture
Contents
Algorithms Efficiency
Measurement of Efficiency
Time & Space Complexity
Time / Space Tradeoff
Tips for Execution Friendly Development
Execution Time Function
Execution Time Comparison for Different Time
Functions
Efficiency
“Algorithm efficiency is the quality of an algorithm that
describes the computational resources
required to run it”
Computational resources
include:
350
Execution Time
300
Space occupancy
250
Memory Requirements
200
150
100
50
1 2 3 4 5 6 7
n T(n)
Algorithm
Efficiency
Less the resources
350
utilized by
algorithm, an
more efficient it is
300
Time & space often go
250
in opposite
200
directions
150
Priority of time and/or space
100
can dictate the choice of
50
algorithm
0
1 2 3 4 5 6 7
In today’s world time is mostly
n 2 4 7 11 16 29 60
T(n) 1 10 21 48 82 178 290
given higher priority than
n T(n)
space
Measuring Algorithm
Efficiency
Two different ways:
1. Measure & Compare Execution Time
2. Measure & Compare Running time (Asymptotic
Analysis)
Note: Running time mostly depends upon input size (n) and is
Measuring Algorithm Efficiency… Measuring
Execution Time
Dependent upon hardware configuration
Dependent upon IO operations
Dependent upon memory in system
Dependent upon other applications installed on system
May differ for same machine on different times
Does not really help to predict affect on execution time when
input size is significantly increased / decreased
May differ greatly for parallel infrastructure
Involve function call overhead
Measuring Algorithm Efficiency… Measuring
Execution Time
Multiple threads trying to access a common resource can increase
execution time significantly
If the program is running on server with multiple disks, some
particular raid configuration might work best for it
Choice of language can increase / decrease the execution time
e.g. C language is faster than Java
Time & Space
Complexity
Time Complexity
Time required to execute an algorithm
Space Complexity
Total memory taken by an algorithm during its
execution
Time & Space Tradeoffs
Time & Space Tradeoffs …
Example
Application data may be stored in arrays, link lists, trees, graphs
etc
For banking / financial transactions time
may
For be compromised
audio/video stream (a bit) but
based problems
accuracy is a mustmay
requirements
preferring solutions withbe
accuracy compromised
low execution time
We want google to respond to our queries promptly while few
irrelevant links may be ignored
Types of
Measurement
Worst Case (BIG O)
Average Case (Theta notation )
Best case (omega notation)
Best, Average & Worst
Cases
For same algorithm, not all inputs take same time to execute
There are input values for which execution time is least (best
cases).
Examples:
incase of sorting data is already sorted
You are looking for a key (linear search) and first element is your
required key
You are looking for a key (binary search) and middle value is your
required key
Best, Average & Worst
Cases
There are input values for which execution time is maximum
(worst cases).
Examples:
We want to sort data in ascending order while it is already in
descending order
You are looking for a key (linear search) and it is not present in
array
You are looking for a key (binary search) and it is not present in
array
Best, Average & Worst
Cases
Average case is present when no prediction is possible about
data
A data value can exist anywhere in the available list of data
It is more difficult to be measured in comparison to best case
or worst case
Example:
Taking n random values and trying to sort them
Asymptotic
Analysis
Independent of hardware, platform & software
Expresses complexity of an algorithm in terms of a known
function related to input size
Analysis describes the growth of running time with reference to
input size
e.g. when n grows then T(n) will grow on order or n log n
Notation for this expression is: T(n) = O(f(n))
Asymptotic Analysis … Constant
Growth O(c)
No growth at all
The runtime does not grow at all as a function of n (constant)
Basically, it is any operation that does not depend on the value
of n to do its job
Has the slowest growth pattern (none!)
Examples:
1. Accessing an element of array
2. Accessing maximum value from a
MAX HEAP
3. Accessing header node from a link
list
4. Accessing root node from a tree
5. Hashing
Asymptotic Analysis … Logarithmic Growth
O(log n)
Logarithmic Growth
The runtime growth is proportional to the base 2
logarithm (log) of n
Examples:
1. Binary Search
2. Max/Min value from a complete
binary tree
Asymptotic Analysis … Linear
Growth O(n)
Linear Growth
Runtime grows proportional to the
value of n
Examples:
1. Linear Search
2. Max/Min value from an
array
3. Sum of value from an
array
4. Link list traversal
Asymptotic Analysis … O(n
log n)
(n log n) Growth
Any sorting algorithm that uses comparisons between
elements is O(n log n), based on divide an conquer
approach
Examples:
1. Merge
Sort
2. Quick
Sort
Asymptotic Analysis …
O(n2)
(n2) Growth
Running Time grows
rapidly
Slow sorting
algorithms Examples:
1. Bubble Sort
2. Insertion Sort
3. Selection Sort
4. Quick Sort (Worst
Case)
Asymptotic Analysis … Polynomial
Growth O(nk)
(nk) Growth
Running Time grows
rapidly
Suitable for small n
Examples:
1. Matrix multiplication
2. Maximum matching for
bipartite graph
3. Multiplying n-digit numbers by
simple algorithm
Asymptotic Analysis … Polynomial
Growth O(2k)
(2k) Growth
Running Time grows extremely
rapidly
Suitable only for very small n
Examples:
1. Exact solution for travelling
salesman
problem
2. Brute force search problems
Asymptotic Analysis … Graph for n
(1-15)
log (n) vs n
16
14
12
10
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
log (n) n
Asymptotic Analysis … Graph for n
(1-15)
n vs n log (n)
70
60
50
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
15
n
n log (n)
Asymptotic Analysis … Graph for n
(1-15)
n log (n) vs n^2 vs n^3
4000
3500
3000
2500
2000
1500
1000
500
0
1 2 3 4 5 6 8 9 10 11 12 13 14 15
7
n^2 n^3
n log (n)
Asymptotic Analysis … Graph for n
(1-15)
n^3 vs 2^n
35000
30000
25000
20000
15000
10000
5000
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
n^3 2^n
Asymptotic Analysis … Graph for n
(1-100)
log n – n – n log n – n2 – n3-2n
35000
30000
25000
20000
15000
10000
5000
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
n log (n) n n log (n) n^2 n^3 2^n
Execution Time Functions (Running
Time)
Functions
Constant
log n
n
n log n
n2
n3
2n
nn
Running time functions are listed above in
ascending order
Execution Time
Functions ….
Execution Example Problems
Time
Function
n2 Bubble Sort, Insertion Sort, Selection Sort
n log(n) Merge Sort, Quick Sort, Heap Sort, Huffman
Encoding
n+k Bucket Sort
nk Radix Sort
n3 Matrix Multiplication
log n Binary Search (sorted data)
n Linear Search
How to Calculate time Complexity
(in Big O notation)
Examples
Int X= 15- (35/2) (linear statement)
Independent of n, the time to execute this statement is constant. If you vary value of input size (n) this will not effect this single
statement
When we have constant time the its complexity is big O of 1 or order of 1: O(1)
X= 15- (35/2); O(1)
Cout<< x; O(1)
y= 5*9; O(1)
Cout<< y; O(1)
Total = O(1) + O(1) +O(1) + O(1) = 4O(1)
Implement rule 2: ignore constant then we left with O(1), means to implement all these statement the total time will be O(1).
Order of 1 is also term as Constant
For (i=0; i<n; i++){
Cout<< I; O(1)
}
We need to find how many times the cout statement will be executed
Lets assume N = 5
Total time= n*O(1) = n*K or k*n (here k is constant)
Implement rule 2: ignore constant then we left with n is our complexity will be O(n),
Cout << “Enter the value of n”; O(1)
Cin>> n; O(1)
For (i=0; i<n; i++){
Cout<< I; O(1) n*O(1) or K*n
}
Total = O(1) + O(1) + kO(n) = 2O(1) + kO(n)
Implement rules. What will be the answer
Nested for loop
Nested for loop
Y = 4*5;
for (i=1; i≤ n; i++){
cout<< I;}
for (x=1; x≤ n; x++){
for (y=1; y≤ n; y++)
cout << x*y;}
If Else Statements
If (x>10){
Cout<< “large”;}
Else { Cout<<“small”
}
For conditional statements the complexity is
calculated by the branch which higher
complexity. Here both branches are O(1)
Practice Problems
Practice Problems
Practice Problems
Suppose N= 5
Practice Problems
