Algorithm Analysis

Introduction
● Algorithm analysis is done for estimating the resource requirements of
algorithms
● Resources include time, space (memory), communication bandwidth,
computer hardware etc.
● Among all, computational time is of prime focus
● In short, algorithm analysis is done mainly to estimate the running time of an
algorithm
● Execution time of an algorithm is also called its time complexity
● Machine-independent notion:
○ If we run the same algorithm/program on computers of different hardware specifications, the execution times
will be different. The computer with faster CPU and more RAM can finish the job faster. Algorithm analysis tries
to describe running time of algorithms independent of the hardware differences/dependence.
● So, the execution time of algorithms is expressed as a function of input size.
○ That is, how the execution time of an algorithm varies with linear increase in input size
● The best notion for input size depends on the problem being handled
○ For sorting problems, input size is the number of individual items to be sorted
○ For integer multiplication, input size means the number of bits needed to store the integers. The more the
number of bits, the bigger the integers are and the more the multiplication time is.
for j=1 to 5

print “hello”

1. How many times hello will be printed? Ans: 5 times.

2. How many times j will be assigned values? Ans: 5 or 6 (Depends on
programming languages. In Python, say, j is assigned 5 different values,
whereas in C j is assigned 6 different values)
for j=2 to 5

print “hello”

1. How many times hello will be printed? Ans: 4 times

2. How many times j will be assigned values? Ans: 4 or 5
for j=1 to 5

for k=1 to 5

print “hello”

1. How many times hello will be printed? Ans: 5 * 5 = 25 times

2. How many times k will be assigned values? 25 or 30 times
for j=1 to 5

for k=1 to 5 and k is odd

print “hello”

1. How many times hello will be printed? 15 times

2. How many times k will be assigned values? 25 or 30 times
Analysis of Algorithms: Example 1
Best case: the array is already sorted
Thus t j =1 for j = 2, 3….. n, and the best-case running time is

● We can express this running time as an + b, where a and b are constants that depend on
the statement costs ci ; it is thus a linear function of n.
● We say that the best case running time is of order of n
Worst-case: Array is sorted in reverse order
● We can express this worst-case running time as an2 + bn + c where a, b and
c are constants that depend on the statement costs ci ; it is thus a quadratic
function of n.
● We say that the worst case running time is of order of n2
Analysis of Algorithms: Example 2
● The following user defined function finds the number of even numbers in an
array of integers.
 Cost Times of execution

C1 1

C2 1

C3 n +1

C4 n

C5 t (between 0 & n)

C6 1

Line 1 is considered as an executable statement as it involves declaration of function variables

(a[ ] & n) & copying parameters from invocation statement to the function variables
● Execution time T(n) = C1 + C2 + C3*(n+1) + n*C4 + t*C5 + C6
● Worst case: All elements in the array are even; t = n
○ T(n) = C1+C2+C3+C6+n*(C3+C4+C5) = a + b*n which is a linear function.
○ So, the worst case running time is of order of n
● Worst case: All elements in the array are odd; t = 0
○ T(n) = C1+C2+C3+C6+n*(C3+C4) = a + b*n which is again a linear function.
○ So, the best case running time is of order of n too
Logarithmic Running Time E.g. (Binary Search)
● Binary search: Searching for an element
on a sorted array
● Algorithm
1. Let x be the element to be
searched for.
2. Compare x with the middle
element.
3. If x matches with the middle
element, we return the mid index.
4. Else If x is greater than the mid
element, then x can only lie in the
right half subarray after the mid
element. So we recur for the right
half.
5. Else (x is smaller) recur for the left
half.
● Execution time
○ Best case: The element is found at the first lookup. Constant time. Happens rarely.
○ Worst case: The element is found on the last comparison/ element not found. In both situations the number of
comparisons will be maximum
■ First comparison: In array of size n
■ Second comparison: In array of size n/2
■ Third comparison: In array of size n/4
■ ..
■ ..
■ Second last comparison: In array of size 2
■ Last comparison: In array of size 1

How many comparisons in total? Ans: Equal to the number of terms in the sequence 1, 2, 4, ….,
n/4, n/2, n
● Number of elements in the sequence 1, 2 is 2 = log2(2)+1
● Similarly,

Sequence Number of terms

1, 2, 4 3 = log2(4)+1

1, 2, 4, 8 4 = log2(8)+1

1, 2, 4, 8, 16 5 = log2(16)+1

1, 2, 4, 8, 16, ….., n/4, n/2, n log2(n)+1

● So, the total number of comparisons required in the worst case of binary search is
log2(n)+1
● Considering the additional processing steps (i.e other than comparisons), the running time is
log2(n)+1 + c1 + c2 + …. = log2(n)+k. So, the running time is of order of log2(n).
Mathematical Notations
● The mathematical notations O, Ω and θ, are used to compare the values of functions.
● These functions are used to express the execution time of algorithms too.
● The notations are also called asymptotic notations
○ Asymptotics is the study of functions when function variable(here n) reaches very high values.
● Big Theta (θ): Asymptotic tight bound

● Big-O (O): Asymptotic upper bound

● Big-Omega (Ω): Asymptotic lower bound

● As said earlier, the asymptotic functions are used to express the time complexity
of algorithms
● Let T(N) be the execution time of an algorithm with input size N.
General Rules
● Rule 1—FOR loops
○ The running time of a for loop is at most the running time of the statements inside the for loop (including tests)
times the number of iterations. In the following example, if the print statement has a running time c, the total
running time for the loop is n*c. So we can say that the running time of the fragment is O(n).

for( i = 0; i < n; ++i )

print “Hello”
● Rule 2—Nested loops
○ The total running time of a statement inside a group of nested loops is the running time of the statement
multiplied by the product of the sizes of all the loops
○ As an example, the following program fragment is O(n2)

for( i = 0; i < n; ++i )

for( j = 0; j < n; ++j )
++k;
○ The time complexity of the following program fragment is O(n*m)

for( i = 0; i < n; ++i )

for( j = 0; j < m; ++j )
++k;
● Rule 3—Consecutive statements

Add the execution times of different fragments (or individual statements)

for the total execution time. T(n) = O(n+n2) = O(n2)
for( i = 0; i < n; ++i )
a[ i ]=0

for( i = 0; i < n; ++i )

for( j = 0; j < n; ++j )
++k;
● Rule 4—If/Else
For the fragment

if( condition )
S1
else
S2
the running time of an if/else statement is never more than the running time of the
test(i.e evaluating the condition) plus the larger of the running times of S1 and S2.
Discussion on Big-O and Big-Omega
Consider the following algorithm.
a=100
b=a*2+1
elems=list()
n<- read the number of elements
for i=1 to n
Read an element
Add the element to the list
 Task Cost

a=100 1

b=a*2+1 4

elems=list() 1

n<- read the number of elements 2

for i=1 to n n + 1 (or n if programming language is Python)

Read an element n*1

Add the element to the list n*1

The costs assigned to the individual steps are random values which reflect the relative time complexities of those
steps with each other. You may assume them as c1, c1, c3.. too. Here, actual integer values are chosen for the
purpose of illustration.
Anyway, the execution time of first 4 steps do not depend on input size, and that of the last 3 depend on the input
size.
Total time of execution T(n) = 1 + 4 + 1 + 2 + (n+1) + (n * 1) + (n * 1) = 3n + 9
Loop execution time = 3n + 1
Preamble execution time = 8
T(n) = 1 + 4 + 1 + 2 + (n+1) + (n * 1) + (n * 1) = 3n + 9

n T(n) Loop time Preamble time

1 12 4 8

2 15 7 8

3 18 10 8

4 21 13 8

5 24 16 8

6 27 19 8

7 30 22 8

8 33 25 8

100 309 301 8

● The execution time is dominated by 3n
○ When n reaches very high values, the execution time becomes approximately equal to 3n
○ E.g when n=100, 3n = 300, 3n+9 = 309; 300 and 309 are comparable (close)
● So, we can say that the execution time is 3n
https://www.desmos.com/calculator
3n+1, 3n+9 and 3n are plotted

How the execution time changes

is decided by 3n and not the
constants.

In algorithm analysis, how the

execution time changes with input
size is the important aspect, and
not the exact execution time

That’s why we ignore constants

when writing the execution times
of algorithms.

But, it is not wrong to say that the

execution time of the algorithm is
3n+9.
The value of 3n never exceeds 3n+9.

So 3n is a lower bound for 3n+9.

That is, 3n+9 = Ω(3n)

In other words, the execution time of

the algorithm, T(n) = Ω(3n)
Consider two functions n and 3n.

n and 3n are plotted.

The value of n never exceeds the value of

3n. So, n is a lower bound of 3n.

We can write it as: 3n = Ω(n). If 3n is the

execution time of an algorithm, we can say,
the execution time, T(n)=Ω(n).

(Like n there can be many lower bounds to

3n, say 2n,n/2 etc)

Also, 3n can be said as an upper bound of

n, as 3n is greater than n always.

We can write this as n=O(3n). If n is the

execution time of an algorithm, we can say,
the execution time T(n)=O(3n)

(Like 3n there can be many upper bounds

to n, say 2n, 4n etc.
Assume the execution time of another algorithm is T(n) = n2 + n

n n2 T(n) = n2 + n

1 1 2

2 4 6

3 9 12

4 16 20

5 25 30

6 36 42

7 49 56

8 64 72
Here the functions n2 (red) and
n2+n (blue) have been plotted.

Both the functions vary the same

way.

So, the execution time of the

algorithm can be stated as Ω(n2)
even though the actual execution
time is n2+n.

Mathematically speaking, the

execution time of the algorithm
can be said as O(n2) too, as 2n2,
3n2 etc (i.e some constant
multiple of n2) is greater than
n2+n. (Refer the definition of
big-O notation. Look at the next
slide for illustration.)
Functions n2+n (red) and 2n2 (blue) are plotted.

As 2n2 exceeds n2+n, we can say n2+n is O(n2).

As n2+n = O(n2) = Ω(n2), we can also say that n2+n=θ(n2)

In the following algorithm, if ti represents the number of times line-5 is executed per
iteration (i.e for a single value of i),

a) what are the minimum and maximum possible values of ti?

b) what are the minimum and maximum possible values of Σi=1n ti?
a) 3n2+2n+9
b) 2n2+5
c) n2+10
d) All of the above
for i=1 to n
for j=1 to n
if (k is odd and k>j)
print “Hi”