Complexity of conventional Sorting
Algorithms
Course Code: CSC2211 Course Title: Algorithms
Dept. of Computer Science
Faculty of Science and Technology
Lecturer No: 02 Week No: 02 Semester: Spring 22-23
Lecturer: Md. Faruk Abdullah Al Sohan;
[email protected] Lecture Outline
1. Sorting Algorithms
Merge Sort
Divide and Conquer
Recursive in structure
Divide the problem into independent sub-problems that are similar
to the original but smaller in size
Conquer the sub-problems by solving them recursively. If they are
small enough, just solve them in a straightforward manner. This can
be done by reducing the problem until it reaches the base case,
which is the solution.
Combine the solutions of the sub-problems to create a solution to
the original problem
Merge Sort
Sorting Problem: Sort a sequence of n elements into non-
decreasing order.
Divide: Divide the n-element sequence to be sorted into
two subsequences of n/2 elements each
Conquer: Sort the two subsequences recursively using
merge sort.
Combine: Merge the two sorted subsequences to produce
the sorted answer.
Merge Sort Algorithm
Step 1 − if it is only one element in the list it is already sorted, return.
Step 2 − divide the list recursively into two halves until it can no more be divided.
Step 3 − merge the smaller lists into new list in sorted order.
Merge Sort Algorithm
Merge(L,R,A) {
nL = length(L)
Mergesort(A) nR = length(R)
{ i= j = k = 0
n = length(A) while(i< nL && j< nR) {
if(n<2) if(L[i] <= R[j]) {
return A[k] = L[i]
mid = n/2 i= i+1
L = array of size(mid) }
R = array of size(n-mid) else {
for(i=0; i < mid; i++) A[k] = R[j]
L[i] = A[i] j= j+1
for(i=mid; i < n; i++) }
R[i-mid] = A[i] k= k+1
Mergesort(L) }
Mergesort(R) while( i<nL) {
Merge(L , R , A) A[k] = L[i]; i=i+1; k=k+1;
} }
while (j<nR) {
A[k] = R[i]; j=j+1; k=k+1;
}
}
Execution Example
Partition
7 2 9 43 8 6 1
7
Execution Example (cont.)
Recursive call, partition
7 2 9 43 8 6 1
7 29 4
8
Execution Example (cont.)
Recursive call, partition
7 2 9 43 8 6 1
7 29 4
72
9
Execution Example (cont.)
Recursive call, base case
7 2 9 43 8 6 1
7 29 4
72
77
10
Execution Example (cont.)
Recursive call, base case
7 2 9 43 8 6 1
7 29 4
72
77 22
11
Execution Example (cont.)
Merge
7 2 9 43 8 6 1
7 29 4
722 7
77 22
12
Execution Example (cont.)
Recursive call, …, base case, merge
7 2 9 43 8 6 1
7 29 4
722 7 9 4 4 9
77 22 99 44
13
Execution Example (cont.)
Merge
7 2 9 43 8 6 1
7 29 4 2 4 7 9
722 7 9 4 4 9
77 22 99 44
14
Execution Example (cont.)
Recursive call, …, merge, merge
7 2 9 43 8 6 1
7 29 4 2 4 7 9 3 8 6 1 1 3 6 8
722 7 9 4 4 9 3 8 3 8 6 1 1 6
77 22 99 44 33 88 66 11
15
Execution Example (cont.)
Merge
7 2 9 43 8 6 1 1 2 3 4 6 7 8 9
7 29 4 2 4 7 9 3 8 6 1 1 3 6 8
722 7 9 4 4 9 3 8 3 8 6 1 1 6
77 22 99 44 33 88 66 11
16
7 2 9 43 8 6 1 1 2 3 4 6 7 8 9
7 29 4 2 4 7 9 3 8 6 1 1 3 6 8
722 7 9 4 4 9 3 8 3 8 6 1 1 6
77 22 99 44 33 88 66 11
17
Merge(L,R,A) {
nL = length(L)
nR = length(R)
i= j = k = 0
while(i< nL && j< nR) {
if(L[i] <= R[j]) {
A[k] = L[i] n*c0+ n*c1+n*c2+ c4
i= i+1
}
=n*(c0+c1+c2) + c4
else { =n*c3 + c4
A[k] = R[j]
j= j+1
=n (Linear time)
} Because of having no nested loops
k= k+1
}
while( i<nL) {
A[k] = L[i]; i=i+1; k=k+1;
}
while (j<nR) {
A[k] = R[i]; j=j+1; k=k+1;
}
}
nL = length(L)
nR = length(R)
i= j = k = 0
i=0 j=0
L 14 23 45 98 R 6 33 42 67
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=0 j=0
L 14 23 45 98 R 6 33 42 67
A
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=0 j=1
L 14 23 45 98 R 6 33 42 67
A 6
k=0
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=1 j=1
L 14 23 45 98 R 6 33 42 67
A 6 14
k=1
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=1
L 14 23 45 98 R 6 33 42 67
A 6 14 23
k=2
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1;
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=2
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33
k=3
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=2 j=3
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42
k=4
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=3
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42 45
k=5
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=4
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42 45 67
k=6
while(i< nL && j< nR) {
If(L[i] <= R[j]) {
A[k] = L[i]
i= i+1
}
else {
A[k] = R[j]
j= j+1
}
k= k+1
}
i=3 j=4
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42 45 67
k=6
while( i<nL) {
A[k] = L[i];
i=i+1;
k=k+1;
}
while (j<nR) {
A[k] = R[i];
j=j+1;
k=k+1;
}
i=3 j=4
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42 45 67
k=6
while( i<nL) {
A[k] = L[i];
i=i+1;
k=k+1;
}
while (j<nR) {
A[k] = R[i];
j=j+1;
k=k+1;
}
i=4 j=4
L 14 23 45 98 R 6 33 42 67
A 6 14 23 33 42 45 67 98
k=7
Exercise
98 23 45 14 6 67 33 42
Merge Sort Analysis
1. The divide step takes constant time, regardless of the
subarray size. After all, the divide step just computes the
midpoint q of the indices p and r. Recall that in big-O
notation, we indicate constant time by O(1).
2. The conquer step, where we recursively sort two
subarrays of approximately n/2 elements each, takes
some amount of time, but we'll account for that time
when we consider the subproblems.
3. The combine step merges a total of n elements, taking
O(n) time.
Merge Sort Analysis
Substitution Method
Mergesort(A) T(n), to sort n elements
{
n = length(A) c1
if(n<2) return
mid = n/2
L = array of size(mid)
R = array of size(n-mid)
for(i=0; i < mid; i++) c2* (n/2) + c2* (n/2)
L[i] = A[i] =c2( n/2 + n/2)
for(i=mid; i < n; i++) =c2 * n
R[i-mid] = A[i]
Mergesort(L) T(n/2)
Mergesort(R) T(n/2)
Merge(L , R , A) c3.n+c4
}
So, T(n) = O(1) when n = 1, and
2T(n/2) + (c2+c3)*n + (c1+c4) when n > 1
So
Skipping the insignificant parts,
[]
[k= ]
[= n]
Merge Sort Analysis
Recursion Tree Method
Recursion-tree Method
Show successive expansions of recurrences using
trees.
Keep track of the time spent on the subproblems of a
divide and conquer algorithm.
Help organize the algebraic bookkeeping necessary to
solve a recurrence.
Recursion-tree Method for Merge Sort
For the original problem, Each of the size n/2 problems
we have a cost of cn, plus has a cost of cn/2 plus two
two subproblems each of subproblems, each costing
size (n/2) and running time T(n/4).
T(n/2).
cn
cn
Cost of divide
and merge.
cn/2 cn/2
T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
•Each level has total cost cn.
•Each time we go down one level, the number of
sub-problems doubles, but the cost per sub-
cn/2 cn/2
problem halves cost per level remains the
same.
cn/4 cn/4 cn/4 cn/4 •There are lg n + 1 levels, height is lg n.
(Assuming n is a power of 2.)
•Total cost = sum of costs at each level = (lg n +
1)cn = cnlgn + cn = O(n lgn).
c c c c c c
Recursion-tree Method for Merge Sort
Elements to sort /
Recursion Tree for Merge Sort
𝒏 𝒏 =c.n
𝟐 𝟐
𝒏 𝒏 𝒏 𝒏
𝟒 𝟒 𝟒 𝟒 =c.n
𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n
Recursion Tree for Merge Sort
n=8
𝒏 𝒏
c.n
𝟐 𝟐
𝒏 𝒏 𝒏 𝒏
=c.n
𝟒 𝟒 𝟒 𝟒
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖
=1 =1 =1 =1 =1 =1 =1 =1
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n
Continue expanding until the problem size reduces to 1.
So , if n=8 we need 3cn
n=8 3 levels we need 3.cn
n=16 4 levels we need 4.cn
n=32 5 levels we need 5.cn
n=64 6 levels we need 6.cn
n=8 n= 3 levels we need 3.cn
n=16 n= 4 levels we need 4.cn
n=32 n= 5 levels we need 5.cn
n=64 n= 6 levels we need 6.cn
n=8 n= 3 levels we need 3.cn
n=16 n= 4 levels we need 4.cn
n=32 n= 5 levels we need 5.cn
n=64 n= 6 levels we need 6.cn
n n= levels . cn
Merge Sort Analysis
Master Method
The master method applies to recurrences of the form
T(n) = a T(n/b) + f (n) ,
where a ≥ 1, b > 1, and f is asymptotically positive.
Examples:
Examples:
Examples:
Þ and
Examples:
Examples:
Best Case Time Complexity: 𝛀(n*log n)
Worst Case Time Complexity: O(n*log n)
Average Time Complexity: ϴ(n*log n)
The time complexity of MergeSort is O(n*Log n) in all the 3
cases (worst, average and best) as the mergesort always
divides the array into two halves and takes linear time to
merge two halves.
Books
Introduction to Algorithms, Thomas H. Cormen, Charle E.
Leiserson, Ronald L. Rivest, Clifford Stein (CLRS).
Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj
Sahni, Sanguthevar Rajasekaran (HSR)
References
https://www.google.com/search?q=bubble+sort+step+by+step&sxsrf=AL
eKk01uxzgfT3Oy6k1Q3WxVnSpiIN8_4g:1587999728942&tbm=isch&sourc
e=iu&ictx=1&fir=vRwFsGwVfJ6pJM%253A%252CSzhhze6MPQr4cM%252C
_&vet=1&usg=AI4_-kSrEEXqwRL-PkHhVUtn7jNfF9dB6g&sa=X&ved=2ahU
KEwje0Pz974jpAhXRAnIKHWhMD2UQ_h0wAXoECAcQBg#imgrc=EN4Sdu7
veOWVoM&imgdii=eOqvCu85p9-eBM
https://www.interviewcake.com/concept/java/counting-sort
https://www.geeksforgeeks.org/counting-sort/
https://www.hackerearth.com/practice/algorithms/sorting/quick-sort/tut
orial/
