Algorithm Analysis and Design

Module I

Dr. Afzal A. L.
College of Engineering Muttathara
 Algorithm
• Definition:
An algorithm can be defined as a step by step procedure to solve a problem or a
task.
It is a sequence of unambiguous instructions used for solving a problem, which
can be implemented (as a program) on a computer by using any one of the programming
language.
• Every algorithm must satisfy the following specifications / properties
(Characteristics )
• Input - every algorithm must take zero or more number of input values from external.
• Output - every algorithm must produce an output as result.
• Definiteness - every statement/instruction in an algorithm must be clear and unambiguous (only one
interpretation).
• Finiteness - for all different cases, the algorithm must produce result within a finite number of steps.
• Effectiveness - every instruction must be basic enough to be carried out and it also must be feasible.
• Algorithms that are definite and effective are also called computational procedure
1/28/2025 Afzal A.L. CE Muttathara 2
 Example
Algorithm Largest (Num1, Num2)
Begin:
if (Num1 > Num2 ) then
print Largest is Num1
else
print Largest is Num2
End If
End

1/28/2025 Afzal A.L. CE Muttathara 3

 Study of algorithm
• There are four distinct areas of study
• How to devise algorithms ?
• various designing techniques are available
• How to validate algorithms ?
• To show that the algorithm compute correct output for all possible inputs – Program validation
• Program proving or sometimes as program verification.
• How to analyse algorithm ?
• Analysis of algorithms or performance analysis refers to the task of determining how much computing time
and storage an algorithm requires
• An important result of this study is that it allows you to make quantitative judgments about the value of one
algorithm over another.
• How to test a program ?
• Debugging is the process of executing programs on sample data sets to determine whether faulty results
occur and, if so, to correct them.
• Profiling or performance measurement is the process of executing a correct program on Datasets and
measuring the time and space it takes to compute the result.
1/28/2025 Afzal A.L. CE Muttathara 4
 Analysis of Algorithm
• Algorithm analysis is an important part of computational complexity theory,
which provides theoretical estimation of the required resources for an algorithm
to solve a specific computational problem.

• Analysis of algorithms is usually the determination of the amount of time and

space resources required to execute it.

• The relative performance of the algorithms might depend on

• characteristics of the hardware
• the details of the dataset
• the size of the problem
1/28/2025 Afzal A.L. CE Muttathara 5
 Algorithm Analysis (Cont..)
• Performance analysis of an algorithm helps to select the best algorithm among
the multiple algorithms designed to solve a problem.
• It predict the resources which are required for an algorithm to perform its task.
• Two main evaluating criteria are space and time required by that particular
algorithm.
• Performance of an algorithm is usually expressed as Complexity of Algorithm,
measured in terms of its input size.
• The complexity of an algorithm is the function f(n) which gives the running time
and/or storage space requirement of the algorithm in terms of the input data size
'n’.
• Space required to complete the task of an algorithm is termed as Space
Complexity.
• The time required to complete the task of an algorithm is termed as Time
complexity of that algorithm.
1/28/2025 Afzal A.L. CE Muttathara 6
 Space needed by a program
• The space needed by a program has the following components:
• Instruction space: Instruction space is the space needed to store the compiled version of
the program instructions. It depends on factors such as:
• The compiler used to complete the program into machine code.
• The compiler options in effect at the time of compilation
• The target computer.
• Data space: Data space is the space needed to store all constant and variable values. It has
two components:
• Space needed by constants and simple variables in program.
• Space needed by dynamically allocated objects such as arrays and class instances.
• Environment stack space: The environment stack is used to save information needed to
resume execution of partially completed functions.

1/28/2025 Afzal A.L. CE Muttathara 7

 Space Complexity of an Algorithm
• The space complexity of an algorithm is the amount of memory it needs to
run to completion.
• The space needed by an algorithm is computed as sum of two components.
• A fixed part that is independent of the characteristics such as number and
size of the inputs and outputs. It includes :
• instruction space(i.e., space for the code),
• Space for simple variables and fixed size component variables
• Space for constants, and so on.
• A variable part that consists of
• the space needed by component variables whose size is dependent on the particular
problem instance being solved
• the space needed by referenced variables
• the recursion stack space
1/28/2025 Afzal A.L. CE Muttathara 8
 Space Complexity (Cont..)
• The space complexity of an algorithm is the amount of memory it needs to
run to completion.
• A fixed part
• A variable part
• The space requirement S(P) of any algorithm P may be therefore written as
S(P)= c + Sp , here c is a constant and Sp is the instance characteristics.
• Hence analyzing the space complexity of an algorithm mainly concentrate
only on instance characteristics (Sp).

1/28/2025 Afzal A.L. CE Muttathara 9

 Space Complexity – Exercise
• Space Complexity of Algorithms with Examples | FACE Prep

Find the space complexity of above algorithm.

Let a, b, c are integers and the size is 2 bytes.
S(abc) = C + Sabc
= 3* 2 + 0 = 3 = O(1)
• Since no variable part- instance characteristics are used , space complexity
for the above-given program is O(1), or constant
1/28/2025 Afzal A.L. CE Muttathara 10
 Space Complexity – Exercise
Find the space complexity of this algorithm.

The problem instances for this Algorithm are characterized by n, the number of elements to be summed.
The space needed by the integers n and i are constant , C1
The space needed by floating point numbers S and a is again constant (C2)
Since a must be large enough to hold the n floating point numbers to be summed, C2 * n
So, Ssum(a,n) = C + Sp = 2C1 + C2 +C2*n = O(n)
.
• complexity for the above-given program is O(n)

1/28/2025 Afzal A.L. CE Muttathara 11

 Time Complexity
• Every algorithm requires some amount of computer time to execute its
instruction to perform the task.
• The time complexity of an algorithm is the total amount of time required by
an algorithm to complete its execution.
• Generally, the running time of an algorithm depends upon the following...
• Whether it is running on Single processor machine or Multi processor
machine.
• Whether it is a 32 bit machine or 64 bit machine.
• Read and Write speed of the machine.
• The amount of time required by an algorithm to
perform Arithmetic operations, logical operations, return value
and assignment operations etc.,
• Input data

1/28/2025 Afzal A.L. CE Muttathara 12

• All these factors except input data depends on the system
configuration in which we are executing the program.
• Calculating Time Complexity of an algorithm based on the system
configuration is a very difficult task because the configuration
changes from one system to another .
• Actual execution time of an algorithm can be computed only by
implementing and executing the program in a computer.
• It may varies from system to systems.
• It does not seems to be an effective method for analayzing algorithms.
• Hence, requires a unified system model to analyze the algorithm.
• Generally , RAM, Random Access Model is used as an Unified
System Model

1/28/2025 Afzal A.L. CE Muttathara 13

 Random Access Machine model
Algorithms can be measured in a machine-independent way using the Random Access
Machine (RAM) model.
This model of computation is an abstraction that allows us to compare algorithms on the basis
of performance.
The assumptions made in the RAM model to accomplish this are:
This model assumes a single processor.
Instructions are executed one after the other, with no concurrent operations.
Each simple operation takes 1 time step.
Loops and subroutines are not simple operations.
Each memory access takes one time step, and there is no shortage of memory.
For any given problem the running time of an algorithm is assumed to be the number of time
steps.
The space used by an algorithm is assumed to be the number of RAM memory cells.
1/28/2025 Afzal A.L. CE Muttathara 14
 Best , Average and Worst case complexities
• The complexity of an algorithm can be measured in 3 cases:

• Best case complexity : - It gives the minimum possible complexity f(n).

• Example : in the case of linear search, if the item to be searched is the first element in the list itself then it

can be searched with the minimum number of iteration. It will be the best case complexity of linear search,

Ω(1).

• Worst case complexity : -The maximum number of steps taken on any instance of size n. It gives maximum

possible complexity f(n) of an algorithm.

• In the case of linear search, if the element to be searched is at the last position then maximum number of

iteration has to be performed. O(n)

1/28/2025 Afzal A.L. CE Muttathara 15

• Average case complexity :- It gives the average complexity f(n). It is the average of complexities of

all the possible cases.

• In linear search

• if the element at index 1 the number of comparison will be 1

• if the element at index 2 the number of comparison will be 2

• …

• if the element at index n the number of comparison will be n

𝑛(𝑛+1)/2 𝑛+1
Average comparison is (1+2+3+…n )/n ➔ =
𝑛 2

• Ø(n/2)

1/28/2025 Afzal A.L. CE Muttathara 16

Asymptotic Notations
 Asymptotic analysis of an algorithm
• The most common method of analyzing an algorithm is to express
the asymptotic growth rate of algorithms with respect to its input
data size.
• Asymptotic behavior describes a function (f (n)) with a defined limit (Another
function – g(n) ). The function may approach this limit, getting closer and
closer as the size of function’s input changes, but will never reach it.
• Asymptotic analysis of an algorithm, refers to defining the
mathematical bound of its run-time performance.
• The notations used to represent the asymptotic growth rate of
algorithms are called asymptotic notations.

8/24/2020 Afzal A.L. CE Muttathara 18

 Asymptotic Notations
• Asymptotic notations are used to analyze an algorithm's running
time by identifying its behavior as the input size for the algorithm
increases.
• It gives the rate at which the complexity of an algorithm increases
as the input data size increase.
• Using asymptotic analysis, we can very well conclude the best case,
average case and worst case scenario of an algorithm Behavior of
this function is usually expressed in terms of one or more standard
functions.
• Expressing the complexity function with reference to other known
functions is called asymptote.
• Commonly used Asymptotic notations are : Big-oh (O) Notation, Big-
omega(Ω) Notation and Big-theta(Ø)Notation.

8/24/2020 Afzal A.L. CE Muttathara 19

 Asymptotic Notations
• Commonly used Asymptotic notations are : Big-oh (O) Notation, Big-
omega(Ω) Notation and Big-theta(Ø) Notation.
• Let f(n) be the actual time complexity of an algorithm
• (Eg:- f(n) = 3n2 + 2n + 4)
• Let g(n) be the asymptote, by which the growth rate of f(n) is to be
expressed.
• Big-oh (O) : The upper limit (worst case ) growth rate
• f(n ) = O(g(n))
• Big-theta(Ø) : Growth in between two boundaries (average case)
• f(n ) = Ø (g(n))
• Big- omega(Ω) : Asymptotic lower bound (Best case )
• f(n) = Ω(g(n))
8/24/2020 Afzal A.L. CE Muttathara 20
 Big-oh (O) Notation
• The Big-oh notation determines the upper limit of f(n).
• Let g(n) be the said upper limit of the function f(n)
• Formal definition:
• f(n) = O(g(n)) if and only if, for any two positive constants c and n0, the
inequality f(n) <= c * g(n) holds for any input size n > n0.
• Example:
• f(n) = 2n + 10
• let g(n) = n
• We can write f(n) = O(n) only when the 2n + 10 <= c * n hold.
• 2n+ 10 – c*n =0
• (2-c)* n = -10
• let n =1 , -c = -10-2 =-12 => c = 12
• This inequality hold for all the value n0 >1 and c >= 12 hence 2n + 10
an be represented as O(n)
8/24/2020 Afzal A.L. CE Muttathara 21
 Big-oh (O) Notation
• The Big-oh notation can be represented graphically as :

8/24/2020 Afzal A.L. CE Muttathara 22

 Big - Omega Notation (Ω)
• Big - Omega notation is used to define the lower bound of an algorithm in
terms of Time Complexity.
• Big-Omega notation always indicates the minimum time required by an
algorithm for all input values.
• That means Big-Omega notation describes the best case of an algorithm
time complexity.
• Formal Definition
• f(n) = Ω(g(n)) if and only if, for any two positive constants c and n0, the inequality
f(n) >= c * g(n) holds for any input size n > n0.
• Explanation :
• Consider function f(n) as time complexity of an algorithm and g(n) is the most
significant term. Then we can represent f(n) as Ω(g(n)) if and only if f(n) >= C *g(n)
holds for all he input data size n >= n0 where C >=1 and n0 >= 1 are any two
constants

8/24/2020 Afzal A.L. CE Muttathara 23

 Big - Omege Notation (Ω)
The Big-Omega notation can be represented graphically as
Example
Let the complexity f(n) = 3n + 2 and g(n) = n
If we want to represent f(n) as Ω(g(n)) then it must
satisfy f(n) >= C g(n) for C>0 and n0>= 1

f(n) >= C g(n)

⇒3n + 2 >= C n

Above condition is always TRUE for all values of C=

1 and n >= 1.

By using Big - Omega notation we can represent the time

complexity as:
3n + 2 = Ω(n)

8/24/2020 Afzal A.L. CE Muttathara 24

 Big - Theta Notation (Θ)

• Big - Theta notation is used to define the average bound of an

algorithm in terms of Time Complexity.
• Big - Theta notation always indicates the average time required
by an algorithm for all input values.
• Big - Theta notation describes the average case of an algorithm
time complexity.

8/24/2020 Afzal A.L. CE Muttathara 25

 Big - Theta Notation (Θ)

• Definition :
 f(n) = Θ(g(n)) if and only if, for any three positive constants c1,c2
and n0, the inequality c1*g(n) <= f(n) <= c2 * g(n) holds for any
input size n > n0.
 Explanation : -
 Consider function f(n) as time complexity of an algorithm and
g(n) is the most significant term. Then we can represent f(n) as
Θ(g(n)) if and only if C1 g(n) <= f(n) <= C2 g(n) hold for all input
data size n >= n0. where C1, C2 and n0 are positive constants.

8/24/2020 Afzal A.L. CE Muttathara 26

 Big - Theta Notation (Θ)
The Big-Theta notation can be represented graphically as

Example
Let the complexity f(n) = 3n + 2 and g(n) = n
f(n) = 3n + 2
g(n) = n
If we want to represent f(n) as Θ(g(n)) then it must
satisfy C1 g(n) <= f(n) <= C2 g(n) for all values of C1 >
0, C2 > 0 and n0>= 1
C1 g(n) <= f(n) <= C2 g(n)
⇒C1 n <= 3n + 2 <= C2 n
Above condition is always TRUE for all values of C1 =
1, C2 = 4 and n >= 2.
By using Big - Theta notation we can represent the
time complexity as follows...
3n + 2 = Θ(n)
8/24/2020 Afzal A.L. CE Muttathara 27
Properties of Asymptotic notations
Following are the properties of Asymptotic notations
• General Properties
• Reflexive Properties
• Transitive Properties
• Symmetric Properties
• Transpose Symmetric Properties
Ref :1. https://youtu.be/NI4OKSvGAgM
2. https://dotnettutorials.net/lesson/properties-of-asymptotic-
notations/

1/28/2025 Afzal A.L. CE Muttathara 28

• General Properties:
• If f(n) is O(g(n)) then a*f(n) is also O(g(n)) ; where a is a constant.
• Example:
f(n) = 2n²+5 is O(n²)
then 7*f(n) = 7(2n²+5)
= 14n²+35 is also O(n²)
• Similarly, this property satisfies both Θ and Ω notation. We can say
• If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)); where a is a constant.
• If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)); where a is a constant.
• Reflexive Properties:
• If f(n) is given then T(n) is O(f(n)).
• Example: f(n) = n² ; O(n²) i.e O(f(n))
• Similarly, this property satisfies both Θ and Ω notation. We can say
• Transitive Properties :
• If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)) .
• Example: if f(n) = n , g(n) = n² and h(n)=n³
n is O(n²) and n² is O(n³) then n is O(n³)
• Similarly this property satisfies for both Θ and Ω notation
1/28/2025 Afzal A.L. CE Muttathara 29
• Symmetric Properties :
• If f(n) is Θ(g(n)) then g(n) is Θ(f(n)) .
• Example: f(n) = n² and g(n) = n² then f(n) = Θ(n²) and g(n) = Θ(n²)
• This property only satisfies for Θ notation.
• Transpose Symmetric Properties :
• If f(n) is O(g(n)) then g(n) is Ω (f(n)).
• Example: f(n) = n , g(n) = n² then n is O(n²) and n² is Ω (n)
• Ie. The upper bound for n is O(n²) and lower bound for n² is Ω (n)
• This property only satisfies for O and Ω notations.
• Some More Properties :
• If f(n) = O(g(n)) and f(n) = Ω(g(n)) then f(n) = Θ(g(n))
• If f(n) = O(g(n)) and d(n)=O(e(n))
then f(n) + d(n) = O( max( g(n), e(n) ))

1/28/2025 Afzal A.L. CE Muttathara 30

8/24/2020 Afzal A.L. CE Muttathara 31
Problem :-
• Derive Big-O notation for 3n+2
• Big-O notation provides an upper bound on the growth rate of a function as n becomes large.
• Specifically, if f(n) is O(g(n)), then there exist constants C and n0​ such that for all n≥n0 ​, the
inequality: f(n) ≤ C⋅g(n) holds true.
• f(n)=3n+2
• As n grows larger, the term 3n becomes the dominant term, because 2 is constant and grows much
slower than n. The constant factor 3 does not significantly affect the asymptotic growth rate.
Hence, we can choose g(n) as n.
• Let g(n) = n
• 3n+2 <= C n
• 3n+2 – C n =0 , let n = 1, then 3 + 2 – C = 0 ➔ C = 5

• Hence 3n+2 can be represented as O(n) for all c>=5 and n0 >=1

1/28/2025 Afzal A.L. CE Muttathara 32

Problem-
Let the complexity f(n) = 3n + 2 . Can you represent f(n) = O(n2) ?

• 3n+2 <= c* n2
• 3n+2 –c* n2 = 0 O(n2) is an upper bound on any function that
• let n =1 grows slower than or at the same rate as n2.
• 3 +2 –c = 0
• -c = -5 2 O(n2) is certainly a valid upper bound for
• c=5 3n+2, but it is not the tightest or most precise
f(n) is O(n2) for any C ≥ 5 and n0 ≥ 1 bound.

The tighter bound would-be O(n).

1/28/2025 Afzal A.L. CE Muttathara 33

 Derive Big O notation for f(n) = n2+2n+5.
f(n) = n2+2n+5. let g(n) = n2
f(n) <= c*g(n)
n2+2n+5 <= c* n2
→ n2+2n+5 - c* n2 =0

(1-c) n2+2n+5 =0
let n =1
(1-c)+2+5 =0
-c +8 =0 => c = 8.
n2+2n+5 = O(n2 ) , for all the values of c >=8 and n0 >=1

8/24/2020 Afzal A.L. CE Muttathara 34

N2+ N = O (N3) Justify your answer..
N2+ N = O (N3) iff
N2+ N <= c*N3
N2+ N - c*N3 = 0
let N =1
1+1-c =0
c =2
Hence we can say that N2+ N = O (N3) for all c >=2 and N0 >= 1
N2+ N = O (N3) is true by Equation. However, as per the rule of
Big-Oh notation, the smallest funtion that satisfying the
required condition should be chosen. Hence N2+ N = O (N2)
8/24/2020 Afzal A.L. CE Muttathara 35
 N2+ N = O (N3) Justify your answer..
N2+ N = O (N3) iff
N2+ N <= c*N3
N2+ N - c*N3 = 0
let N =1
1+1-c =0
c =2
Hence we can say that N2+ N = O (N3) for all c >=2 and
N0 >= 1

8/24/2020 Afzal A.L. CE Muttathara 36

1. Is 2n+1 = O(2n) 2. Is 22n = O(2n)?
Answer • 22n <= C* 2n
f(n) = 2n+1 g(n) = 2n • (2n)2<= C*2n
f(n) <= c*g(n) ? • 22n grows exponentially faster than 2n
because it is the square of 2n
2n+1 <= C* 2n • 2n grows without bound, so there is no
2n+1 / 2n <= C constant C that can satisfy this inequality
for all n. Therefore, 22n grows faster than
2 <= C 2n and cannot be upper-bounded by a
constant multiple of 2n .
2n+1 <= C* 2n this condition hold • C >= 22n / 2n A ratio which is not bound
by a constant.
for all C >=2 and n0 >=1
Hence • Hence 22n = O(2n) is not true
2n+1 = O(2n) is true

1/28/2025 Afzal A.L. CE Muttathara 37

Problem : Is 22n = O(2n)?
• 22n <= C* 2n
• (2n)2<= C*2n
• 22n grows exponentially faster than 2n because it is the square of 2n
• 2n grows without bound, so there is no constant C that can satisfy this
inequality for all n. Therefore, 22n grows faster than 2n and cannot be
upper-bounded by a constant multiple of 2n .
• C >= 22n / 2n A ratio which is not bound by a constant.
• Hence 22n = O(2n) is not true

1/28/2025 Afzal A.L. CE Muttathara 38

 Computing complexity of algorithm
• O(1): Time complexity of a function (or set of statements) is
considered as O(1) if it doesn’t contain loop, recursion and call to any
other non-constant time function.

• O(1) :- A loop or recursion that runs a constant number of. For

example the following loop is O(1), for the constant c =10 or 100..
• for (int i = 1; i <= c; i++) {
// some expressions
}

1/28/2025 Afzal A.L. CE Muttathara 39

O(n): Time Complexity of a loop is considered as O(n) if the loop
variables is incremented / decremented by a constant amount.
For example following functions have O(n) time complexity

1/28/2025 Afzal A.L. CE Muttathara 40

• O(nc): Time complexity of nested loops is equal to the number of times
the innermost statement is executed. For example the following sample
loops have O(n2) time complexity

1/28/2025 Afzal A.L. CE Muttathara 41

• O(Logn) Time Complexity of a loop is considered as O(Log n) if the loop
variables is divided / multiplied by a constant amount

1/28/2025 Afzal A.L. CE Muttathara 42

• O(LogLogn) Time Complexity of a loop is considered as O(LogLogn) if
the loop variables is reduced / increased exponentially by a constant
amount.

1/28/2025 Afzal A.L. CE Muttathara 43

• Time complexities of consecutive loops When there are consecutive
loops, we calculate time complexity as sum of time complexities of
individual loops.

1/28/2025 Afzal A.L. CE Muttathara 44

• Compute the complexity of following function
void fun (m )
{
for ( int j=1; j<=m; j++) ---- m+1 or O(m)
// Statement with O(1 ) complexity----- m or m* O(1)
}
-----------------
m+1+m = 2m+1
Here the varying part is m only .
Hence the complexity of this function is O(m) O(m)

1/28/2025 Afzal A.L. CE Muttathara 45

• Analyze the complexity of the following function
void function(int n)
{ int count = 0;
for (int i=n/2; i<=n; i++) ----------- (n/2)+1 → O(n )
for (int j=1; j<=n; j = 2 * j) ----------- (n/2)logn→O(n*log n)

for (int k=1; k<=n; k = k * 2)...... (n/2)lognlogn+1→O(n*logn*logn)

count++; ---------------- (n/2)lognlogn →O(n*logn*logn)
}
• The complexity of this function is O(n*logn*logn)

1/28/2025 Afzal A.L. CE Muttathara 46

Exercise
Compute Time complexity of following
Functions
1.
function(int n) Frequency
Count
{
if (n ==1 ) return 1 =O(1) n * number of times j
loop executed .
for(i= 1;i<=n;i++) { n+1 = O(n) j loop executed only
for (j=1;j<=n;j++) { n = O(n) once because of break
statement
print (“*”); n= O(n)
break; n = O(n)
}
}
} Complexity is O(n)
1/28/2025 Afzal A.L. CE Muttathara 47
Exercise
Compute Time complexity of following
Functions
1.
function(int n) Frequency
Count
{
if (n ==1 ) return 1 =O(1)
for(i= 1;i<=n;i++) { N+1 = O(n)
for (j=1;j<=n;j++) { n*(n+1) = O(n^2)
print (“*”); n*n = O(n^2)

}
}
} Complexity is O(n^2)
1/28/2025 Afzal A.L. CE Muttathara 48