Course title : CSE2001

Course title : Data Structures and Algorithms

Module :1
Topic :1

Introduction to Algorithm
Analysis

1
Module No. 1 - Introduction to Algorithm Analysis
Basic Algorithmic Analysis: Asymptotic notations: Big O, little o, omega, and
theta notations, Running time calculations, Identifying differences among
best, average, and worst case behaviors; Complexity analysis- Time and
space

2
Algorithm

• Algorithm:
• Step by Step procedure for solving a computational problem.
• An algorithm is a finite set of instructions that, if followed,
accomplishes a particular task
• Infinitely many correct algorithms for the same problem.
• A good algorithm should be optimized in terms of time and space.

3
Algorithm
Characteristics or Properties of an algorithm:
• Input: Zero or more quantities are externally supplied.
• Output: At least one output is produced.
• Definiteness: Each instruction is clear and unambiguous.
Ex: sqrt(-1) , 15/0, add 6 or 7 to x are not permitted.
• Finiteness: The algorithm must terminate after a finite number of steps.
• Some programs run continuously, Operating System that continues in a "wait" loop
until more jobs are entered, data base servers, web servers keep on running, they
are providing service, not algorithms.
• Effectiveness: Every instruction must be very basic so that it can be carried out,
in principle, by a person using only pencil and paper in a finite amount of time.
• Algorithms that are definite and effective are also called computational procedures.

4
Algorithm
Algorithm Program
During Software development i.e., Implementation time
phases of development of the project
life cycle or manufacturing process
through engineering
procedures, first design it, make sure it is
correct, then only start development.
Can’t construct or develop by trail basis.
Should have problem or domain knowledge. Programmer
Ex: Accounts – Accountant, Hospital -
Doctors
Any Language can be used to write Programming Languages like C, C+
algorithm, by using mathematical +, Java, Python, etc.
notations as long as it is
understandable.
Hardware, Software, or Operating Hardware, Software, or
Systems Operating
independent. Systems dependent. 5
Algorithm
Algorithm Specification: Algorithm can be described in three ways.
1. Natural language like English:
2. Graphic representation called flowchart:
o This method will work well only when the algorithm is small & simple.
3. Pseudo code:
o This method describes algorithms as program, which resembles
language like C
and Pascal.

6
Analysis of Algorithms
• Determining how much time how much space an algorithm will take to
and problem solve a
• Good Algorithms
• Run in less time
• Consume less memory
• Time Complexity: The time complexity of an algorithm is the amount of
computer
time it needs to run to completion.
• Space Complexity: The space complexity of an algorithm is the amount of
memory it needs to run to completion.
• Factors
• Hardware
• Operating
System
• Compiler
• Size of input 7
•
Analysis of Algorithms - Time Complexity:
• Time Complexity of an algorithm (basically when converted to program) is
the amount of computer time it needs to run to completion.
• The time taken by a program is the sum of the compile time and the
run/execution time.
• The compile time is independent of the instance (problem specific)
characteristics.
• Following factors affect the time complexity:
• Characteristics of compiler used to compile the program.
• Computer Machine on which the program is executed and physically
clocked.
• Multiuser execution system.
• Number of program steps.
• The time complexity consists of two components fixed (factor 1 only)
and variable /
instance (factor 2,3 & 4), so for any algorithm 'A' it is provided as:
8
Time(A) = Fixed Time(A) + Instance Time(A)
Analysis of Algorithms - Time Complexity:
Few objective measures to compare algorithms:
• Execution times:
• Not a good measure as execution times is specific to a particular
computer.
• A number of statements executed:
• Not a good measure, since the number of statements varies with the
programming language as well as the style of the individual
programmer.
• Ideal solution:
• Express the running time of a given algorithm as a function of the input
size n (i.e.,
f(n)) and compare these different functions corresponding to running
times.
• This kind of comparison is independent of machine time, programming
style, etc.
• Time complexity of an algorithm quantifies the amount of time taken by
9
an algorithm
Analysis of Algorithms - Time Complexity:
• The number of steps in any program statement is assigned depends on
the kind of statement like
• Comments count as zero steps,
• An assignment statement which does not involve any calls to other
algorithm is counted as one step,
• For iterative statements consider the steps count only for the control
part of the statement etc.
• Time complexity is not going to examine the total execution time of an
algorithm. Rather, it is going to give information about the variation
(increase or decrease) in execution time when the number of operations
(increase or decrease) in an algorithm.
• Time complexity measures the time taken to execute each statement
in an
algorithm.
• If a statement is set to execute repeatedly then the number of times
that statement gets executed is equal to N multiplied by the time10
required to run that function each time.
Analysis of Algorithms - Time Complexity:
Common Problems Associated Primitive Operations Input Size
Find x in an array A Comparison of x with other Array size
(Search Problem) elements of A
Multiply 2 matrices A and B Multiplication and Addition Dimension of the matrix
(Arithmetic on Matrices)
Sorting (Arrangement of Comparison Array size
elements in some order)
Graph Traversal Number of times an edge is The number of vertices and
traversed edges
Any Recursive Procedure Number of recursive calls + time Array size (for array related
spent on each recursive call problems).
Finding Factorial Multiplication The input number.
Finding LCM(a,b) Basic arithmetic (division, The number of bits needed
subtraction) to represent a and b.
The number of primitive operations increases with the
problem size. Therefore, we express the step count (time 11
Analysis of Algorithms
Space Complexity:
 Space Complexity of an algorithm is the amount of memory it needs
to run to completion i.e. from start of execution to its termination.
 Space need by any algorithm is the sum of following components:
o Fixed Component: This is independent of the characteristics of the
inputs and outputs. This part includes: Instruction Space (i.e., space
for code), Space of simple variables, fixed size component
variables, and space for constants variables.
o Variable Component: This consist of the space needed by component
variables whose size is dependent on the particular problems
instances (Inputs/Outputs) being solved, the space needed by
referenced variables and the recursion stack space is one of the
most prominent components, also this included the data
structure components like Linked list, heap, trees, graphs etc.
oTherefore, the total space requirement of any algorithm 'A' can be
provided as S(A) = c + SA (Instance Characteristics i.e., Variable 12
Analysis of Algorithms
oEvery Simple statement (direct statements) of an algorithm takes 1
unit of time.
Algorithm Time analysis Space Analysis
Algorithm Swap(a,b) //Swapping of two Three variables used
numbers
{
temp = a; 1 a -- 1
a = b; 1 b -- 1
b = a; 1 temp -- 1
}
Total Time 3
Total Space 3 words
Time function f(n) 3 - Constant
Space function S(n) 3 - Constant
Time function f(n) O(1)
Space function S(n) O(1)

13
Analysis of Algorithms
Frequency Count Method: (Step Count)
• Used to find the time complexity of an algorithm.
• Step Count: The number of times each statement in the algorithm is
executed.
• The time taken by an algorithm known by assigning one unit of time
for each statement and if any statement is repeating for some
number of times, the frequency of the execution of that
statement will calculate and find the time taken by statement.

14
Analysis of Algorithms
Algorithm Space Analysis Time Analysis
Steps per execution Frequency Total Steps
(s/e)
Algorithm Sum(A,n) Variables
//Finding Sum of array elements are A – n 0 - 0
{ word 0 - 0
s = 0; n–1 1 1 1
for (i=0; i<n;i++) { s-1 1 n+ n+
s = s + A[i]; i–1 1 1n 1n
} S(n) = n + 0 - -
return s; 3 1 1 1
} 0 F(n) = 2n + 3
Example: Degree: 2n + 3
A = {3,6,7,9,2} O(n)
n = 5; Degree:
O(n)
Understanding Loop Statement time complexity
i=0  1 time, i++  n times
i < n  n+1 times, how?
Since i = 0, n=5, i<n will be evaluated for 6
times- 1
0<5 1<5 - 2 2<5 – 3 3<5 – 4 4<5 – 5<5 – 6
5
So n+1 times condition is checked.
Time complexity of loop is: 1 + n + 1 + n  2n + 2
1
but we can consider only n+1 [Highest term] as we are doing briefly. 5
Analysis of Algorithms
Algorithm Space Analysis Time Analysis
Steps per Frequency Total Steps
execution
(s/e)
Algorithm AddM(A,B,n) Variables are
//Finding Sum of array elements A – n2 0 - 0
{ B - n2 0 - 0
for (i=0; i<n;i++) { C – n2 1 n+1 n+1
for (j=0; j<n;j+ n–1 1 n(n+ n(n+
+) { C[i,j] = i-1 1 1) 1)
A[i,j] + B[i,j] j–1 0 n*n n*n
} S(n) = 3n2 + 3 0 - -
} - -
Degree: O(n2) = 2n2 + 2n
Matrices are n * n +1
Loop i  time taken is n+1 times Degree: O(n2)
Inside loop i, j loop and statements in j loop executed for Highest
Degree
n times
Inside loop i, j is a loop,
j loop itself will take n+1 times for comparisons
So, its value is n*(n+1)
1
Inside j loop, statement will be executed for 6
Analysis of Algorithms
Algorithm Space Time Analysis
Analysis
Algorithm MulM(A,B,n) Variables are Time Loop i  time taken
//Finding Mul of A – n2 Take is n+1 times
array B - n2 n Inside loop i, j loop
elements C – n2 and Statements in j
{ n–1 loop executed for n
for (i=0; i<n;i++) { i-1 n+ times, similarly k
for (j=0; j<n;j++) j–1 1 loop and
{ k–1 n statements in k loop
C[i,j] = 0; n
for (k=0; k<n;k+ S(n) = 3n2 + 4 n
+) { Degree: O(n2) n
C[i,j] = C[i,j] +
A[i,k]*B[k,j]
}
} 17
Analysis of Algorithms
Algorithm Space Time Analysis
Analysis
Algorithm MulM(A,B,n) Variables are Time Loop i  time taken is n+1
//Finding Mul of array elements times
{ A – n2 Take
Inside loop i, j loop and
B - n2 n statements in j loop
C – n2 executed for n times,
for (i=0; i<n;i++) { n–1 similarly k loop and
for (j=0; j<n;j++) { i-1 n+1 statements in k loop.
C[i,j] = 0; j–1 n*
for (k=0; k<n;k++) { Inside loop i, j is a loop,
C[i,j] = C[i,j] + k–1 (n+1) j loop itself will take n+1
A[i,k]*B[k,j] n * times for comparisons
} S(n) = 3n2 + 4 n n So, its value is n*(n+1)
}
Degree: O(n2) * n
n * Statements in j loop
executed for n times,
n similarly k loop and
statements in k loop will
be executed for n 18times.
Analysis of Algorithms
Algorithm Space Time Analysis
Analysis
Algorithm MulM(A,B,n) Variables are Time Taken
//Finding Mul of array A – n2
elements B - n2
{ C – n2 n+1
for (i=0; i<n;i++) { n–1 n*
for (j=0; i-1 (n+1) n Inside loop j, k is a
j<n;j++) j–1 *n loop, k loop itself will
take n+1 times for
{ C[i,j] = 0; k–1 n*n* comparisons So, its
for (k=0; (n+1) value is n*(n+1)
k<n;k++) { S(n) = 3n2 + n*n*n
C[i,j] = C[i,j] + 4 = n+1+n2+n+n2 Statements in k loop
A[i,k]*B[k,j] Degree: O(n2) +n3 + n2 + n3 executed for n times
} = 2n3 + 3n2 +
} 2n + 1
F(n) = O(n3) 19
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
1 for (i=0; n+ 2n+1 O(n)
i<n;i++) 1n
{ stmt;
}
2 for (i=n; i>0;i--) { n+1 2n+1 O(n)
stmt; n
}
3 for (i=1; i<n; (n+1)/2 n+1/2 O(n)
i=i+2) { stmt; n/2
}

4 for (i=1; (n+1)/20 (n/20) O(n)

i<n;i=i+20) n/20
{ stmt;
}
5 for (i=0; i<n;i++) { n+1 n2 O(n2)
for (j=0; j<n;j++) { n*
stmt; (n+1) n
} *n
}
20
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
6 for (i=0; i<n;i++) { n+1 n(n+1)/2 O(n2)
for (j=0; j<i;j++) { n*
stmt; n
}
} 0,1,2,3;;;n

7 k = 0; Not repeating for n O(sqrt(n))

for (i=1; times.
k<=n;i++) Assume, k > n, stop it.
k = p (p+1)/2
{ k = k + i; p (p+1)/2 > n, will stop
} p2 > n
p > sqrt(n)
Not repeating for n
times. Not i <= n,
Here k <=n

21
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
8 for (i=1; i<n; i=i*2) { Assume, i >= n, stop it. O(log2n)
stmt; Since i = 2k
} 2k >= n
Not repeating for n times. Let us take
Not i++ 2k = n
Here i = i * 2 k=
log2n

Stmt
executed for
log n times
i=1
*2*2*2*…..
=n
2k = n
k = log2n

9 for (i=n; i>=1; i=i/2) { Assume, i <1, stop it. O(log2n)

stmt; i.e., n/2k < 1
} Let equate to n/2k = 1
n = 2k
Not repeating for n times. k = log2n
Not i++ 2
2
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
10 for (i=0; i*i<n; i++) { Assume, i *i >= n stop it. O(sqrt(n))
stmt; i.e., i2 = n
} So, i = sqrt(n)
Not repeating for n times.
Here i*i < n

11 for (i=0; i<n;i++) { n+1 F(2n) O(n)

stmt; n
}
for (j=0; j<n;j++) { // j < m n+1 // m+1
stmt; F(n+m) O(n+m)
n //m
}

Two independent loops

12 k=0; logn O(log logn)

for (i=1; i<n;i = i*2) {
k++;

for (j=0; j<k;j=j*2) { logp  log log n

stmt;
}} p = logn

2
3
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
13 for (i=0; i<n;i++) { (n+1) O(nlogn)
for (j=0; j<n;j=j*2) { n * logn
stmt; n * logn
} 2nlogn
} +n+1

14 sum = 0; 1 O(n3)
for (i=0; i<n;i++) { 1 + n+1 + 1
for (j=0; j<n*n;j++) { n * (1 + n*n+1 + 1)
sum = sum + 1; n * (n*n)
} Highest Order will
} be n3

15 sum = 0; 1 O(n5)
for (i=0; i<n;i++) { 1 + n+1 + 1
for (j=0; j<i*i;j++) { n * (1 + n*n+1 + 1)
for (k=0; k<j;k++) { n *( n*n ( 1+ n*n+1 + 1)
sum = sum + Highest Order will be n5
1; j can be as large as i2, which
} could be as large as n2
} k can be as large as j, which
} is n2
The running time is thus
proportional to n · n2· n2
2
4
Analysis of Algorithms
S.No Code Time Analysis F(n) Order
16 sum = 0; The if statement is executed O(n4)
for (i=0; i<n;i++) { at most n3 times,
for (j=0; j<i*i;j++) { but it is true only O(n2) times
if(j%i == 0) (because it is true exactly i
for (k=0; k<j;k++) { times for each i).
sum = sum + 1; The inner most loop is
} executed O(n2) times.
} Total time complexity
} = O(n4)

17 for (i=0; i<n;i++) O(n)

for (i=0; i<n;i=i+2) O(n)
for (i=0; i<n;i=i+20) O(n)
for (i=n; i>1;i--) O(n)
for (i=1; i<n;i=i*2) O(log2n)
for (i=1; i<n;i=i*3) O(log3n)
for (i=n; i>1;i=i/2) O(log2n)

25
Analysis of Algorithms
Types of Time functions:
S.No Time Meaning F(n) Examples
Function
1 O(1) Constant 2, 100, 5000  Find if a number is even or odd.
 Check if an item on an array is null.
 Print the first element from a list.
 Find a value on a map
2 O(logbn) Logarithmic, b is constant  Algorithms that divide problems in half every time.
 Binary search
3 O(sqrt(n)) Square root
4 O(n) Linear 2n+3,  Get the max/min value in an array.
500n+70  Find a given element in a collection.
0,  Print all the values in a list.
(n/500) +
6
5 O(nlogbn) Linearithmic, b is constant  Efficient sorting algorithms like merge sort, quicksort, and
others.
6 O(n2) Quadratic  Check if a collection has duplicated values.
 Sorting items in a collection using bubble sort,
insertion sort, or selection sort.
 Find all possible ordered pairs in an array.
7 O(nk) Polynomial, where k is O(n3)  Find the solutions for a multi-variable equation
some  Matrix Multiplication
constant > 1
8 O(2n), Exponential, where c is  Power Set: finding all the subsets on a set.
O(3n), some  Fibonacci, Travelling salesman problem using dynamic
26
O(cn) constant programming.
Analysis of Algorithms

Order the following functions by growth rate: N, √ N, N1.5, N2, N log N, N log log N, N log2N, N log(N2), 2/N, 2N , 2N/2 , 37,
N2log N, N3. Indicate which functions grow at the same rate and show why this is the case.

• Solution:

• 2/N < 37 < √ N < N < N log log N < N log N ≤ N log(N2) < N log2N < N1.5 < N2 < N2log N < N3 < 2N/2 < 2N

• N log N ≤ N log(N2)  are equal.

27
Asymptotic Notations
 Method of describing limiting Behavior.
 Example:
• 2n2+ 3n+ 1  The order of growth depends on n2 for large n. Other
terms are
relatively insignificant as n increases.
• The function asymptotically equivalent to n2 as n tends to Ꝏ
• Written as f(n) ~ n2 and read as f(n) asymptotic to n2
 Topic comes from Mathematics.
 Performing step count calculation for large algorithms is a time-consuming
task.
 A natural way is to upper bound the time complexity instead of finding
the exact step count.
 Order of Growth or Rate of Growth of an algorithm gives a
simple characterization of the algorithm’s efficiency by identifying
relatively significant term in the step count.
 Notations are used to represent the simple form of the functions or
showing
term’. the class of a function. 2
8
Asymptotic Notations
 Asymptotic notation describes the algorithm efficiency and performance in
a meaningful way.
• It describes the behavior of time or space complexity for large
instance characteristics.
• Representing time or space complexity functions in a communicable
form.
• Asymptotic analysis is input bound i.e., if there's no input to the
algorithm, it is concluded to work in a constant time. Other than
the "input" all other factors are considered constant.

29
Asymptotic Notations

Symbol How to Read Meaning

O Big-oh Upper bound
Ω Big-omega Lower bound
Θ Theta Average bound or tight bound
o Small - oh
ω Small-omega Lower bound

Θ is more useful and there is no Small Theta

30
Asymptotic Notations
Asymptotic upper bounds:
 Big-Oh: The function f(n) = O(g(n)) if and only if there exist positive
constants c, nf(n) ≤ c*g(n), ∀n ≥ n0 [we may read it as f(n) is O(g(n)) or f(n)
∈
such that 0 0

≤ O(g(n)]
f(n) = 2n + 3
2n+3 ≤
Most ------
important is on RHS side only one term, even though LHS
has more than one term.
Single term has one coefficient.
2n+3 ≤ 10*n
n = 1 2*1+3 ≤10  5 ≤ 10
n=2 2*2+3 ≤ 10*2  7 ≤
n=3 20
f(n) ≤ c g(n)  g(n)
2*3+3 = n therefore
≤ 10*3 9≤ f(n) = O(n)
30 2n+3 ≤ 7n, 2n+3 ≤ 100*n,
we can write
But to how to write?
In simple write it as
2n + 3 ≤ 2n  2n + 3 2n +3n  2n + 3 ≤
5n n ≥
+3 ≤ 1 , 31
Asymptotic Notations
Can we write
2n + 3 ≤ 2n2 +3 
2n + 3 ≤ 2n2 +3n2  2n + 3 ≤ 5n2 n ≥ 1
Here g(n) = n2  therefore f(n) = O(n2), true
Here f(n) means class of Time function. If f(n) = n then

f(n) = O(n2), O(n3), O(2n)  true

f(n) = O(logn)  false, part of lower bound

Do we need to write all?

i.e., f(n) = O(n2), O(n3), O(2n)
 Try to write the closest function. Writing higher order is not wrong but not useful, hence we will write the closest one.
32
Asymptotic Notations

f(n) ≥ c*g(n), ∀n ≥ n0 [we may read it as f(n) is Ω (g(n)) or f(n) ∈ Ω (g(n)]

Omega: The function f(n) = Ω (g(n)) if and only if there exist positive constants c, n0 such that

f(n) = 2n + 3
2n+3 ≥ ------
Most important is on RHS side only one term, even though LHS
has more than one term.
Single term has one coefficient.
2n+3 ≥ 1*n
for n = 1 2*1 + 3 ≥ 1  5 ≥ 1
for n = 2 2*2 + 3 ≥ 2  7 ≥ 2
f(n) ≥c g(n)  g(n) = n  therefore f(n) = Ω (n)
Here f(n) means class of Time function.
Can we write If f(n) = n then
f(n) = Ω(n2), Ω(n3), Ω(2n)  false
2n + 3 ≥ logn , f(n) = Ω (logn)  True f(n) = Ω(1), Ω(logn), Ω(sqrt(n)), Ω(n)  true, part of lower bound
Do we need to write all?
i.e., f(n) = Ω(1), Ω(logn), Ω(sqrt(n)), Ω(n)
2n + 3 ≥ n2  f(n) = Ω (n2)  false 3
 Try to write the closest function. Nearest one is useful. 3
Asymptotic Notations

that c1*g(n) ≤ f(n) ≤ c2*g(n), ∀n ≥ [we may read it as f(n) is Θ(g(n)) or f(n) ∈
Theta: The function f(n) = Θ(g(n)) if and only if there exist positive constants c1, c2, and n0 such

n0 Θ(g(n)]

f(n) = 2n + 3
----- ≤ 2n+3 ≤ ------
Lower bound and upper bound to be found.
1*n ≤ 2n+3 ≤ 5*n

c1 = 1, c2 = 5 g(n) = n

So, f(n) = Θ(n)

f(n) = Θ (n2), Θ (n3),
Θ (2n)  false
• Price
f(n) = Θ (1),ofΘa mobile phone: 30000 is maximum, 2000 is minimum
• Answer
(logn)  false is correct, but that will not be a suitable phone for me.
• In the same way when we have any function, n2 , Θ(n2) is perfect, but O(n2) may be
Example:
be nearest,
nearest, Ω(nbut not give exact
2) may 3
4
Asymptotic Notations
 Big-Oh: The function f(n) = O(g(n)) if and

n0 such that 0 ≤ f(n) ≤ c*g(n), ∀n ≥ n0 [we

only if there exist positive constants c,

may read it as f(n) is O(g(n)) or f(n) ∈

O(g(n)]

• Omega: The function f(n) = Ω (g(n)) if and

f(n) ≥ c*g(n), ∀n ≥ n0 [we

only if there exist positive constants c,
n0 such that
may read it as f(n) is Ω (g(n)) or f(n) ∈
Ω (g(n)]

• Theta: The function f(n) = Θ(g(n)) if and

only if there exist positive
constants c1,√ Nc<2,N < and
1< log N <
n such that
N log log N0 < N log N < N1.5 < N2 < N2log N < N3 < 2N/2 < 2N < 3N < CN 3
5
Asymptotic Notations
Given f(n), Write O, Ω,
Θ.
f(n) = 2n2 + 3n f(n) = n!
+ 4 f(n) = n2 n! = n * ( n-1) * (n-2) * -----
logn + n f(n) = *1 1*2*3*…..*n ≤ n * n *
2n2 + 3n n! + ≤4 2n2 + 3n2 + n* ….. * n
f(n)
2n
4n2 + = 3n
2n2+ + 43n≤ for all n ≥ 1, g(n) = 1*1*1*…..*1 ≤
+
9n 4
hence
2 f(n) O(n2) n2, 1*2*3*…..*n
= 2n2 + ≥1 *  f(n) Ω (n2) 1*1*1*…..*1 ≤
1
3n* + n24≤ 2n2 +n23n ≤9n= 2  f(n) Θ(n2) 1*2*3*…..*n ≤ n
+4 = * n*
f(n) = n2 logn n*
n2 n
+ logn + n ≤ 10 n2 O(n2 logn) ….. * n
* n2 logn≥ nf(n)
logn
1 2 logn
=+n Ω(n2 logn) 1 ≤ n! ≤ nn
 f(n)
1 * n2 =
logn≤ n2 logn + n ≤ 10 f(n) =
We can’t ) -- the
O(nnfix Upper
particularplace
nf(n)
2 logn Θ(n2 logn)
bound f(n) = Ω(1) –
average
for
= Lower bound
bound. 3
6
Asymptotic Notations
Give f(n) = 5n4 + 3n3 + 2n2 + 4n + 1. Find Upper bound
Ans
:
5
n
4

f(n) O(n4)
+= = 5n2 + 3nlogn + 2n + 5. Find Upper bound
Give f(n)
Ans:
3 2 + 3nlogn + 2n + 5 ≤ (5 + 3 + 2 + 5) n2
5n
n ≤ 15n2
3 ≤ cn2 when n ≥ n0 =1 O(n2)
f(n) =
+
Give f(n) = 2n+2. Find Upper bound
Ans: 2 2n+2 = 2n * 22 =
n c =
4*24,n n =1 f(n) O(2n)
0 3
7
Asymptotic Notations
Give f(n) = 3log n+2. Find Upper bound
Ans:
3log n+2 ≤ 5log n
for n ≥ 2, Note that logn is zero for n
= 1,O(logn)
f(n) n ≥ n0 = 2
= = 20n3 + 10nlogn + 5. Find Upper bound
Give f(n)
Ans:
20n3 + 10nlogn + 5 ≤ 35n3
≤ cn3 when n ≥ n0 =1 O(n3)
f(n) =
Give f(n), Find Lower bound
5n2

N3

5n2 =  (n2 ) or  (nlogn) or  (n) or  (logn) or  (1)

3
3 = 3 2 8
Asymptotic Notations
Given f(n), Write O, Ω,
Θ. f(n) = log n!
log(1*2*3*…..*n)
log(1*2*3*…..*n) ≤ log(n * n * n*
….. * n) log(1*1*1*…..*1) ≤
log(1*2*3*…..*n)
log(1*1*1*…..*1) ≤ log(1*2*3*…..*n) ≤ log(n * n *
n* ….. * n) 1 ≤ log n! ≤ log(nn)  1 ≤ log n! ≤
nlog(n)
f(n) = O(lognn) -- Upper
bound f(n) = Ω (1) – Lower
bound
Here we can’t find average
bound. No same function
on either side of log n!

3
Give f(n) = 3n +3 9
Asymptotic Notations

Little oh: The function f(n) = o (g(n)) if and only lim 𝑓(𝑛)
= 0 there
𝑛→∞ 𝑔(𝑛)
exist
if constants c, n0 such that f(n) < c*g(n), ∀n ≥ n0 positive
f(n) = n2 and g(n) = n3 then check whether f(n) = o(g(n)) or not.

The result is 0, and it satisfies the equation mentioned above. So we can say that f(n) =
o(g(n)).
lim
f(n) = 2n + 3
𝑛→
=0
𝑛2
2𝑛+3
∞
f(n) = o(n2)
or f(n) = o(nlogn) or f(n) = o(n log
logn) But f(n) ≠ o(n) 4
0
Asymptotic Notations
• Big-O is used as a tight upper-bound on the growth of an algorithm’s effort, Little-o”
(o()) notation is used to describe an upper-bound that cannot be tight.
• The set of functions f(n) are strictly smaller than cg(n), meaning that little-o notation is
a stronger upper bound than big-O notation.
• Little omega: The function f(n) = ω(g(n)) if and only if lim 𝑔(𝑛)
= 0 there exist
𝑛→∞ 𝑓(𝑛)
positive
constants c, n0 such that f(n) > c*g(n), ∀n ≥ n0
 “Big-Omega” (Ω()) is the tight lower bound notation, and “little-omega” (ω()) describes
the loose lower bound.

The relationships between the notations

41
Types of Analysis
Worst case
• Provides an upper bound on running time
• An absolute guarantee that the algorithm would not run longer, no matter
what the inputs are.
Best case
• Provides a lower bound on running time
• Input is the one for which the algorithm runs the fastest
Average case
• Provides a prediction about the running time
• Assumes that the input is random

42
Asymptotic Notations
Note: Mostly everyone will think that O for worst case, Ω for best case and
Θ for average case, completely wrong, any notation can be used to
represent the worst case, best case, and average case.
Basic Rules:
1. Drop constant factors
2. Drop lower order polynomials
3. Any polynomials dominate any logarithms
4. Exponential functions dominate polynomials
5. xn dominates yn if x > y
6. In general log means log10, but for this course log means log2

43
Types of Analysis
For each of the following pairs of functions,
either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) =
Θ(g(n)). Determine the relationship.
f(n) = log n2 g(n) f(n) Θ(g(n)
= log n+5 f(n) = n g(n) = Ω (g(n))
log n2 f(n) O(g(n))
f(n) = log log n g(n) = Ω (g(n))
= log n f(n) = n g(n) = f(n) Ω (g(n))
log2n = Θ(g(n)
f(n) = nlog n + n g(n) = f(n) Ω (g(n))
log n f(n) = 10 g(n) = log = O(g(n))
10 f(n)
f(n) = 2n g(n) = =
10n2 f(n) = 2n f(n)
g(n) = 3n = 44
Types of Analysis
• In algorithm analysis we often use the notation “log n” without
specifying the base

45
Summary
We have discussed about
• Introduction about Algorithms
• Types of Algorithms
• Experimental Analysis of
Algorithms

46