LECTURE NOTES ON

SUBJECT: DESIGN AND ANALYSIS OF ALGORITHM

SUBJECT CODE: BCS-503
Unit-1
B.TECH
BRANCH: CSE & IT
YEAR- 3RD SEMESTER- 5th

(AKTU)

Rohit Mishra
(Assistant Professor)

Department of Computer Science & Engineering and Information

Technology
United Institute of Technology, Prayagraj

Syllabus Unit-1
Introduction: Algorithms, Analyzing Algorithms, Complexity of Algorithms, Growth of Functions, Performance
Measurements, Sorting and Order Statistics - Shell Sort, Quick Sort, Merge Sort, Heap Sort, Comparison of Sorting
Algorithms, Sorting in Linear Time.
 Unit-1
Introduction

1.1 Algorithm
An algorithm is any well-defined computational procedure that takes some value, or set of
values, as input and produces some value, or set of values, as output. An algorithm is thus a
sequence of computational steps that transform the input into the output.
For example, given the input sequence {31, 41, 59, 26, 41, 58), a sorting algorithm returns as
output the sequence {26, 31, 41, 41, 58, 59}. Such an input sequence is called an instance of the
sorting problem. ,
Instance: An instance of a problem consists of the input needed to compute a solution to the
problem.
An algorithm is said to be correct if, for every input instance, it halts with the correct output.
There are two aspects of algorithmic performance:
• Time
- Instructions take time.
- How fast does the algorithm perform?
- What affects its runtime?
• Space
- Data structures take space
- What kind of data structures can be used?
- How does choice of data structure affect the runtime?
1.1.1 Analysis of Algorithms
Analysis is performed with respect to a computational model
• We will usually use a generic uniprocessor random-access machine (RAM)
 All memory equally expensive to access
 No concurrent operations
 All reasonable instructions take unit time
o Except, of course, function calls
 Constant word size
Unless we are explicitly manipulating bits
Input Size:
• Time and space complexity
 This is generally a function of the input size
o E.g., sorting, multiplication
 How we characterize input size depends:
o Sorting: number of input items
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
1
 o Multiplication: total number of bits
o Graph algorithms: number of nodes & edges
o Etc
Running Time:
• Number of primitive steps that are executed
 Except for time of executing a function call most statements roughly require the
same amount of time
o y=m*x+b
o c = 5 / 9 * (t - 32 )
o z = f(x) + g(y)
• We can be more exact if need be
Analysis:
• Worst case
 Provides an upper bound on running time
 An absolute guarantee
• Average case
 Provides the expected running time
 Very useful, but treat with care: what is “average”?
o Random (equally likely) inputs
o Real-life inputs

1.1.2 Analyzing algorithm

Example: Insertion Sort:
InsertionSort(A, n)
{
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j=j-1
}
A[j+1] = key
}
}
Analysis

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
2
 Statement Effort
InsertionSort(A, n) {
for i = 2 to n { c1n
key = A[i] c2(n-1)
j = i - 1; c3(n-1)

while (j > 0) and (A[j] > key) { c4T

A[j+1] = A[j] c5(T-(n-1))
j=j-1 c6(T-(n-1))
} 0
A[j+1] = key c7(n-1)
} 0
}
T = t2 + t3 + … + tn where ti is number of while expression evaluations for the ith for loop
iteration
T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1)
= c8T + c9n + c10
Best case -- inner loop body never executed
ti = 1

= an-b = ϴ(n)
T(n) is a linear function
Worst case -- inner loop body executed for all previous elements
ti = i

= an2 +bn –c= O(n2)

T(n) is a quadratic function

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
3
Average case: The “average case” is often roughly as bad as the worst case.

1.1.3 Designing algorithms

The divide and conquer approach
The divide-and-conquer paradigm involves three steps at each level of the recursion:
Divide the problem into a number of subproblems that are smaller instances of the same
problem.
Conquer the subproblems by solving them recursively. If the subproblem sizes are small
enough, however, just solve the subproblems in a straightforward manner.
Combine the solutions to the subproblems into the solution for the original problem.

1.2 Merge sort

The merge sort algorithm closely follows the divide-and-conquer paradigm. Intuitively, it
operates as follows.
Divide: Divide the n-element sequence to be sorted into two subsequences of n=2 elements each.
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.
The key operation of the merge sort algorithm is the merging of two sorted sequences in the
“combine” step. We merge by calling an auxiliary procedure MERGE (A, p, q, r) where A is an
array and p, q, and r are indices into the array such that p≤ q < r. The procedure assumes that the
subarrays A (p…. q) and A (q+1……… r) are in sorted order. It merges them to form a single
sorted subarray that replaces the current subarray A (p …….r). Our MERGE procedure takes
time ϴ(n), where n = r- p+1 is the total number of elements being merged.

MERGE (A, p, q, r)
1 n1 ← q - p + 1
2 n2 ← r - q
3 create arrays L[1……. n1 + 1] and R[1……… n2 + 1]
4 for i ← 1 to n1
5 do L[i] ← A[p + i - 1]
6 for j ← 1 to n2
7 do R[j] ← A[q + j]
8 L[n1 + 1] ← ∞
9 R[n2 + 1] ← ∞
10 i ← 1
11 j ← 1

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
4
12 for k ← p to r
13 do if L[i] ≤ R[j]
14 then A[k] ← L[i]
15 i ← i + 1
16 else A[k] ← R[j]
17 j ← j + 1
Example:

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
5
Fig: The Merge procedure applies on given array and sort and combines the solution in recursive iteration.
MERGE-SORT (A, p, r)
1 if p < r
2 then q ← (p + r)/2
3 MERGE-SORT (A, p, q)
4 MERGE-SORT (A, q + 1, r)
5 MERGE (A, p, q, r)

1.2.1 Analysis of Merge-sort

When we have n > 1 elements, we break down the running time as follows.
Divide: The divide step just computes the middle of the subarray, which takes constant time.
Thus, D(n)=ϴ(1).
Conquer: We recursively solve two subproblems, each of size n/2, which contributes 2T(n/2) to
the running time.
Combine: We have already noted that the MERGE procedure on an n-element subarray takes
time ϴ(n) and so C(n)=ϴ(n).

The recurrence for the worst-case running time T(n) of merge sort:
The solution for above recurrence is ϴ (n log n).

1.3 Growth of functions

The notations we use to describe the asymptotic running time of an algorithm are defined in
terms of functions whose domains are the set of natural numbers N=(0, 1, 2……). Such
notations are convenient for describing the worst-case running-time function T(n), which usually
is defined only on integer input sizes.

1.3.1 Asymptotic notations

1.3.1.1 Upper Bound Notation or O-notation

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
6
 We say InsertionSort’s run time is O(n2). Properly we should say run time is in O(n2). Read O as
“Big-O” (you’ll also hear it as “order”)

In general a function f(n) is O(g(n)) if there exist positive constants c and n0 such that f(n)   

g(n) for all n  n0

Formally

O(g(n)) = { f(n):  positive constants c and n0 such that f(n)  c  g(n)  n  n0

Example.
1. functions in O(n2)
n2/ 1000, n1.9, n2, n2+n, 1000 n2+50n
2. Show 2n2 = O (n3)
0 ≤ h(n) ≤ cg(n) Definition of O(g(n))
0 ≤ 2n2 ≤ cn3 Substitute
0/n3 ≤ 2n2/n3 ≤ cn3/n3 Divide by n3
Determine c
0 ≤ 2/n ≤ c 2/n = 0
2/n maximum when n=1
0 ≤ 2/1 ≤ c = 2 Satisfied by c=2
Determine n0
0 ≤ 2/ n0 ≤ 2
0 ≤ 2/2 ≤ n0
0 ≤ 1 ≤ n0 = 1 Satisfied by n0=1
0 ≤ 2n2 ≤ 2n3 ∀ n ≥ n0=1

If f(n) ≤ cg(n), c > 0, ∀ n ≥ n0 then f(n) ∈ O(g(n))

3. 1000n2 + 50n = O(n2)
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
7
0 ≤ h(n) ≤ cg(n)
0 ≤ 1000n2 + 50n ≤ cn2 Substitute
0/n2 ≤ 1000 n2/ n2 + 50n/ n2 ≤ c n2/ n2 Divide by n2
0 ≤ 1000 + 50/n ≤ c Note that 50/n → 0 as n → ∞
Greatest when n = 1
0 ≤ 1000 + 50/1 = 1050 ≤ c = 1050 Satisfied by c=1050
0 ≤ 1000 + 50/n0 ≤ 1050 Find n0 ∀n ≥ n0
-1000 ≤ 50/ n0 ≤ 50
-20 ≤ 1/ n0 ≤ 1
-20 n0 ≤ 1 ≤ n0 = 1 Satisfied by n0=1
0 ≤ 1000 n2 + 50n ≤ 1050 n2 ∀n ≥ n0=1, c=1050
If f(n) ≤ cg(n), c > 0, ∀n ≥ n0 then f(n) ∈O(g(n))

Big O Fact
A polynomial of degree k is O(nk)
Proof:
Suppose f(n) = bknk + bk-1nk-1 + … + b1n + b0
Let ai = | bi |
f(n)  aknk + ak-1nk-1 + … + a1n + a0

 n k  a i nk n k  a 
i
cnk
n

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
8
 i

1.3.1.2 Lower Bound Notation or Ω-notation

We say Insertion Sort’s run time is (n). In general a function f(n) is (g(n)) if  positive
constants c and n0 such that 0  cg(n)  f(n)  n  n0

Proof: Suppose run time is an + b. Assume a and b are positive (what if b is

negative?). an  an + b

Example. 1. Functions in Ω(n2)

n2/ 1000, n1.999, n2+n, 1000 n2+50n
2. n3 = Ω(n2) with c=1 and n0=1
0 ≤ cg(n) ≤ h(n)
0 ≤ 1*12 = 1 ≤ 1 = 13
0 ≤ cg(n) ≤
h(n) 0 ≤ c n2
≤ n3
0/ n2 ≤ c n2/ n2 ≤
n3/ n2 0 ≤ c ≤ n
0 ≤ 1 ≤ 1 with c=1 and n0=1 since n increases.

1.3.1.3 Asymptotic Tight Bound or -notation

A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that
c1 g(n)  f(n)  c2 g(n)  n  n0

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
9
 Theorem: f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))

Example . 1. Show that ½n2 - 3n ∈ Θ(n2)

Determine ∃ c1, c2, n0 positive constants such that:
c1 n2 ≤ ½ n2 - 3n ≤ c2 n2
c1 ≤ ½ - 3/n ≤ c2 Divide
by n2 O: Determine c2 = ½
½ - 3/n ≤ c2
as n → ∞, 3/n → 0
½ - 3/n = ½
therefore ½ ≤ c2 or c2 = ½ ½ - 3/n maximum for as n → ∞ Ω:
Determine c1 = 1/14
0 < c1 ≤ ½ - 3/n ½ - 3/n > 0 minimum for n0
= 7 0 < c1 = ½ - 3/7 = 7/14 - 6/14 = 1/14
n0: Determine n0
= 7 c1 ≤ ½ - 3/ n0
≤ c2 1/14 ≤ ½ - 3/
n0 ≤ ½
½n2-3n = Θ(n2)
c1=1/14, c2=1/2
and n

Θ: ½n2 - 3n ∈Θ(n2) when c1 = 1/14, c2 = ½ and n0 = 7

1.3.1.4 Other Asymptotic Notations

o-notation: A function f(n) is o(g(n)) if  positive constants c and n0 such
that f(n) < c g(n)  n  n0
ω-notation: This notation is A function f(n) is (g(n)) if  positive constants c and n0 such that

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
10
 c g(n) < f(n)  n  n0
Intuitively,
• o() is like <
• () is like >
• () is like =
• () is like 
• O() is like 
1.4 Recurrences
A recurrence is an equation or inequality that describes a function in terms of its value on
smaller inputs. For example, the worst-case running time T (n) of the MERGE-SORT
procedure by the recurrence

Whose solution is to be T(n)=O(n lg n).

1.4.1 Solving Recurrences: following methods are used to solve recurrences and find its
asymptotic performance
1. Substitution method
2. Recursion Tree
3. Master method
4. Iteration Method

1.4.1.1 The substitution method

This method is called “making a good guess method” and solving recurrences comprises two
steps:
1. Guess the form of the solution.
2. Use mathematical induction to find the constants and show that the solution
We can use the substitution method to establish either upper or lower bounds on a recurrence.
As an example, let us determine an upper bound on the recurrence

T(n) = 2T(n/2˚) + n
We guess that the solution is T(n)=O(n lg n). The substitution method requires us to prove
T(n) ≤ c n lg n for an appropriate choice of the constant c > 0. We start by assuming that this
bound holds for all positive m < n, in particular for m = n/2˚, yielding T(n/2˚) =c(n/2˚) lg
(n/2˚).Substituting into the recurrence yields

T(n) ≤ 2 c(n/2˚) lg (n/2˚)+ n

≤ cn lg (n/2) +n
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
11
 = cn lg n _ cn lg 2 + n
≤ cn lg n _ cn + n
≤ cn lgn
where the last step holds as long as c ≥ 1.

1.4.1.2 Changing variables:

Sometimes, a little algebraic manipulation can make an unknown recurrence similar to one

you have seen before. As an example, consider the recurrence

which looks difficult. We can simplify this recurrence, though, with a change of variables. For
convenience Renaming m= lg n yields
T(2m) = T(2m/2)+m
We can now rename S(m) = T(2m) to produce the new
recurrence S(m)= S (m/2) +m
The solution is to be S (m)= O (m lg m)
After replacing m= log n the solution becomes
T(n) = O(lg n lg lg n)

1.4.1.3 The recursion-tree method:

Substitution method is not coming up with good guess. Drawing out a recursion tree provides
a method to devise a good guess. In a recursion tree, each node represents the cost of a single
subproblem somewhere in the set of recursive function invocations. We sum the costs within
each level of the tree to obtain a set of per-level costs, and then we sum all the per-level costs
to determine the total cost of all levels of the recursion.
For example, let us see how a recursion tree would provide a good guess for the
recurrence T(n)= 3T( n/4 ) + ϴ(n2)
we create a recursion tree for the recurrence T(n)= 3T(n/4) + cn2, having written out the implied
constant coefficient c > 0.

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
12
Fig: Constructing a recursion tree for the recurrence T(n)= 3T(n/4) + cn2. Part (a) shows T (n),
which progressively expands in (b)–(d) to form the recursion tree. The fully expanded tree in part
(d) has height log4n (it has log4n +1 levels).
The above fig shows how we derive the recursion tree for T(n)= 3T(n/4) + cn2 .For
convenience, we assume that n is an exact power of 4 so that all subproblem sizes are
integers. Part (a) of the figure shows T(n), which we expand in part (b) into an equivalent tree
representing the recurrence. The cn2 term at the root represents the cost at the top level of
recursion, and the three subtrees of the root represent the costs incurred by the subproblems
of size n=4. Part (c) shows this process carried one step further by expanding each node with
cost T(n/4) from part (b). The cost for each of the three children of the root is c(n/42). We
continue expanding each node in the tree by breaking it into its constituent parts as
determined by the recurrence.
Because subproblem sizes decrease by a factor of 4 each time the subproblem size for a node
at depth i is (n/4i). Thus for subproblem size at last level n=1 n/4i =1 then i = log4n and the
tree has log4n+1 levels. The cost of each node is derived by generalized term at depth i where
i= 0, 1, 2…….log4n-1 by c (n/4i)2 . the total cost over all nodes at depth i, for i= 0, 1,
2…….log4n-1 is 3i (n/4i )2 =(3/16)i cn2. The last level i.e. at depth log4n cost is 3log4n= nlog43
nodes each contributing cost T(1), for a total cost nlog43 T(1) which is ϴ(nlog43 ) since we
assume T(1) is constant.

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
13
Now we add up the costs over all levels to determine the cost for the entire tree:

Thus, we have derived a guess of T(n)= O(n2). Now we can use the substitution method to
verify that our guess was correct, that is, T(n)= O(n2) is an upper bound for the recurrence
T(n)= 3T( n/4 ) + ϴ(n2). We want to show that T(n)≤ dn2 for some constant d > 0. Using the
same constant c > 0 as before, we have

Where the last step holds as long as d = (16/13)c.

1.4.1.4 The master method

For divide and conquer algorithm, An algorithm that divides the problem of size n into a
subproblems, each of size n/b Let the cost of each stage (i.e., the work to divide the problem
+ combine solved subproblems) be described by the function f(n), then, the Master Theorem
gives us a cookbook for the algorithm’s running time in the form of recurrence given below
T(n) = a T(n/b) + f(n)
Where a≥1and b>1 are constants and f (n) is an asymptotically positive function.
1.4.1.4.1 The master theorem:
The master method depends on the following theorem.
Let a≥1and b>1 be constants, let f (n) be a function, and let T(n) be defined on the
nonnegative integers by the recurrence
T(n) = a T(n/b) + f(n)

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
14
 where we interpret n/b to mean either floor (n/b) or ceil (n/b). Then T(n) has the following
asymptotic bounds:

( ) ( )
1. If  ( ) =      − for some constant ϵ>0 then ()=      

( ) ( )
2. If  ( ) =      − for some constant ϵ>0 then ()=      

( )
3. If  ( ) =      + for some constant ϵ>0 and if a T (n/b) ≤c f(n) (regularity function
In each of the three cases, we compare the function f (n) with the function nlogba. Intuitively, the larger of the two
bba
functions determines the solution to the recurrence. In case 1the function nlog a is larger, solution is T(n)= n log .
In case 3 function f(n) is larger, solution is T(n)= ϴ(f (n)).
In case 2 both functions has same value, solution is T(n)= ϴ(nlog a logb n).
In the first case, not only must f(n) be smaller than nlog a, itb must be polynomially smaller. In the third
case, not only must f (n) be larger than nlog a, it also must
b be polynomially larger and in addition satisfy

the “regularity” condition that a T (n/b) ≤c f(n). An addition to that all three cases do not cover all
possibilities. Some function might be lies in between case 1 and 2 and some other lies in between case
2 and 3 because the comparison is not polynomial larger or smaller and in case 3 the regularity
condition fails.
Example. 1. . The given recurrence is
T(n) = 9T(n/3) + n
Sol: a=9, b=3, f(n) = n
nlogb a = nlog3 9
= (n2)
Since f(n) = O(nlog3 9 - ), where =1, case 1 applies:

( )
T (n) =  n log b a when f (n) = O n log b a − )
Thus the solution is T(n) = (n2)

2. T(n)= T(2n/3)+1
in which a = 1, b= 3/2, f(n)= 1, and nlogba = nlog3/2 =n0 = 1. Case 2
applies, since f (n)=  (nlogba )= (1) and thus the solution to the recurrence is T(n) =(lg n)

3. The master method does not apply to the recurrence:

T(n)= 2T(n/2) +n log n
Sol: a=2, b=2, f(n)= n log n , nlogba=nlog 2=n
Since f(n) is larger than nlogba you mistakenly apply case 3 but the f(n) is larger but not
polynomialy larger. The ratio f(n)/ nlogba= log n is asymptotically less than nϵ for any positive
constant ϵ. Consequently, the recurrence falls into the gap between case 2 and case 3.
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
15
1.4.1.5 Iteration method
Another option is the “iteration method” which expand the recurrence by using iterative
equations, work some algebra to express as a summation and finally evaluate the summation.
It’s a iterative procedure which recursively used and prorogate to a final single value.
For Example

1.5 Heap Sort

A heap can be seen as a nearly binary tree:

An array A that represents a heap is an object with two attributes: A.length, which (as usual)
gives the number of elements in the array, and A.heap-size, which represents how many
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
16
elements in the heap are stored within array A. That is, although A (1……..A.length)
may contain numbers, only the elements in A(1……….A.heap-size), where 0≤ A.heap-size≤
A.length, are valid elements of the heap. The root of the tree is A(1), and given the index i of
a node, we can easily compute the indices of its parent, left child, and right child:
Parent(i)
return floor (i/2)
Left(
i)
retur
n 2i
Right(i)
return
2i+1
There are two kinds of binary heaps: max-heaps and min-heaps. In both kinds, the values in
the nodes satisfy a heap property, the specifics of which depend on the kind of heap. In a
max-heap, the max-heap property is that for every node i other than the root,
A[parent(i)]≥A[i]
That is the value of a node is lesser or equal to parent.
A min-heap is organized in the opposite way; the min-heap property is that for every node i
other than the root,
A[parent(i)]≤A[i]
For the heapsort algorithm, we use max-heaps.

1.5.1 Maintaining the heap property:

1.5.1.1 MAX-HEAPIFY(A, i)
{
l = Left(i); r = Right(i);
if (l <= A.heap-size && A[l] >
A[i]) largest = l;
else
largest = i;
if (r <= A.heap-size && A[r] >
A[largest]) largest = r;
if (largest != i)
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
17
 exchange A[i] with
A[largest]; MAX-
HEAPIFY(A, largest);
}

Figure: The action of MAX-HEAPIFY(A, 2), where heap-size[A] = 10. (a) The initial
configuration, with A[2] at node i = 2 violating the max-heap property since it is not larger
than both children. The max-heap property is restored for node 2 in (b) by exchanging A[2]
with A[4], which destroys the max-heap property for node 4. The recursive call
MAXHEAPIFY (A, 4) now has i = 4. After swapping A[4] with A[9], as shown in (c), node 4
is fixed up, and the recursive call MAX-HEAPIFY(A, 9) yields no further change to the data
structure.

1.5.1.2 Complexity
We can describe the running time of MAX-HEAPIFY by the
recurrence T(n)≤2T(n/3)+ϴ(1)
The solution to this recurrence, by case 2 of the master theorem is T(n)= O(lg n)

1.5.2 Building a heap:

We can build a heap in a bottom-up manner by running Heapify() on successive subarrays.
For array of length n, all elements in range A[n/2˚ + 1 .. n] are heaps.

1.5.2.1 BUILD-MAX-HEAP(A)
1. A.heap-size = A.length
2. for i = length[A]/2˚ downto 1
3. MAX-HEAPIFY(A, i)
The time required by MAX-HEAPIFY when called on a node of height h is O(h), the running
time of above algorithm is O(n).

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
18
Figure: The operation of BUILD-MAX-HEAP, showing the data structure before the call to
MAX-HEAPIFY in line 3 of BUILD-MAX-HEAP. (a) A 10-element input array A and the
binary tree it represents. The figure shows that the loop index i refers to node 5 before the call
MAX-HEAPIFY(A, i). (b) The data structure that results. The loop index i for the next
iteration refers to node 4. (c)-(e) Subsequent iterations of the for loop in BUILD-MAXHEAP.
Observe that whenever MAX-HEAPIFY is called on a node, the two subtrees of that node are
both max- heaps. (f) The max-heap after BUILD-MAX-HEAP finishes.

1.5.3 The heapsort Algorithm

The heapsort algorithm starts to build max heap by using procedure BUILD-MAX-HEAP
and then picks the root element which has the higher value. Then remove root value from the
tree and built again it max heap. This process performs up to last value and sorted array is
formed.

1.5.3.1 HEAPSORT(A)
1. BUILD-MAX-HEAP(A)
2. for i = length(A) downto 2
3. Exchange (A[1] with A[i]);
4. A.heap-size= A.heap-size -1;
5. MAX-HEAPIFY (A, 1);

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
19
Figure: The operation of HEAPSORT. (a) The max-heap data structure just after it has been built by
BUILD-MAX-HEAP. (b)-(j) The max-heap just after each call of MAXHEAPIFY in line 5. The value
of i at that time is shown. Only lightly shaded nodes remain in the heap. (k) The resulting sorted array
A.

1.6 Quick sort

Quicksort is also a divide-and-conquer algorithm. An unsorted array A taken in which p and r
is the lower bound and upper bound of the elements respectivally.
Divide:The array A[p..r] is partitioned into two non-empty subarrays A[p..q] and A[q+1..r].
Invariant: All elements in A[p..q] are less than all elements in A[q+1..r].
Conquer: The subarrays are recursively sorted by calls to quicksort.
Combine: Unlike merge sort, no combining step: two subarrays form an already-sorted array.
The following procedure implements quicksort:
QUICKSORT(A, p, r)
1. if (p < r)
2. q = PARTITION(A, p, r)
3. QUICKSORT(A, p, q)
4. QUICKSORT(A, q+1, r)
To sort an entire array A, the initial call is QUICKSORT (A, 1, A.length).
Partitioning the array:

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
20
 The key to the algorithm is the PARTITION procedure, which rearranges the subarray
A(p….r) in place.
PARTITION(A, p, r)
1x←
A[r] 2 i
←p-
1
3 for j ← p to r
- 1 4 do if A[j]
≤x
5 then i ← i + 1
6 exchange A[i] ↔ A[j]
7 exchange A[i + 1] ↔
A[r] 8 return i + 1

Figure: The operation of PARTITION on a sample array. Lightly shaded array elements are
all in the first partition with values no greater than x. Heavily shaded elements are in the
second partition with values greater than x. The unshaded elements have not yet been put in
one of the first two partitions, and the final white element is the pivot. (a) The initial array
and variable settings. None of the elements have been placed in either of the first two
partitions. (b) The value 2 is "swapped with itself" and put in the partition of smaller values.
(c)-(d) The values 8 and 7 are added to the partition of larger values. (e) The values 1 and 8
are swapped, and the smaller partition Grows. (f) The values 3 and 8 are swapped, and the
smaller partition grows. (g)-(h) The larger partition grows to include 5 and 6 and the loop
Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
21
terminates. (i) In lines 7-8, the pivot element is swapped so that it lies between the two par
titions.

1.6.1 Performance of Quicksort:

The running time of quicksort depends on whether the partitioning is balanced or unbalanced,
which in turn depends on which elements are used for partitioning.

1.6.1.1 Worst Case Partitioning:

The worst-case behavior for quicksort occurs when the partitioning routine produces one
subproblem with n - 1 elements and one with 0 elements. The recurrence for the running time
is

T(n)=T(n-1)+T(0) + (1)= (n2) i.e T(n) = (n2)

1.6.1.2 Best case partitioning:
in the best case it partition the array into equal sub arrays. The recurrence for
balanced portioning is

T(n) = 2T(n/2) + (n)= (n lg n) i.e. T(n) = (n lg n)

1.7 Shell sort

Shellsort, also known as Shell sort or Shell's method, is an in-place comparison sort. It can be
seen as either a generalization of sorting by exchange (bubble sort) or sorting by insertion
(insertion sort).The method starts by sorting elements far apart from each other and

# Sort an array a[0...n-1].

gaps = [701, 301, 132, 57, 23, 10, 4, 1]
# Start with the largest gap and work down to a gap of 1
foreach (gap in gaps)
{
# Do a gapped insertion sort for this gap size.
# The first gap elements a[0..gap-1] are already in gapped order
# keep adding one more element until the entire array is gap sorted
for (i = gap; i < n; i += 1)
{
# add a[i] to the
elements that have been
gap sorted # save a[i] in
temp and make a hole at
progressivelyposition
reducingi temp = a[i]
the gap between them.
# shift earlier gap-sorted elements up until the correct location for
a[i] is found Prepared By:
for (j = i; j >= gap and a[j - gap]Rohit
Mr. Mishra
> temp; j -= gap)
Assistant Professor (UIT)
{ 22

a[j] = a[j - gap]

The first pass, 5-sorting, performs insertion sort on separate subarrays (a1, a6, a11),
(a2, a7, a12), (a3, a8), (a4, a9), (a5, a10). For instance, it changes the subarray (a1, a6,
a11) from (62, 17, 25) to (17, 25, 62). The next pass, 3-sorting, performs insertion
sort on the subarrays (a1, a4, a7, a10), (a2, a5, a8, a11), (a3, a6, a9, a12). The last pass,
1-sorting, is an ordinary insertion sort of the entire array (a1,..., a12).
The complexity of this algorithm is O (n1.25).

1.8 Sorting in linear time

The algorithms that can sort n numbers in O(n lg n) time is Merge sort and heap
sort and achieve this upper bound in the worst case; Quick sort achieves it on
average. These algorithms share an interesting property: the sorted order they
determine is based only on comparisons between the input elements. We call such
sorting algorithms comparison sorts. All the sorting algorithms introduced thus far
are comparison sorts.
We shall prove that any comparison sort must make Θ(n lg n) comparisons in the
worst case to sort n elements. Thus, merge sort and heap sort are asymptotically
optimal, and no comparison sort exists that is faster by more than a constant factor.
Here three sorting algorithms-counting sort, radix sort, and bucket sort-that run in
linear time. These algorithms use operations other than comparisons to determine
the sorted order. Consequently, the Θ(n lg n) lower bound does not apply to them

1.8.1 Counting sort

Counting sort assumes that each of the n input elements is an integer in the range 0
to k, for some integer k. When k = O(n), the sort runs in ‚ϴ (n) time.

1.8.1.1 The algorithm:

1. Input: A[1..n], where A[j] Œ {1, 2, 3, …, k}
2. Output: B[1..n], sorted (notice: not sorting in place)
3. Also: Array C[1..k] for auxiliary storage
COUNTING-SORT(A, B, k)
1 for i ← 0 to k
 2 do C[i] ← 0
3 for j ← 1 to length[A]
4 do C[A[j]] ← C[A[j]] + 1
5 //C[i] now contains the number of
elements equal to i. 6 for i ← 1 to k
7 do C[i] ← C[i] + C[i - 1]
8 //C[i] now contains the number of elements less than
or equal to i. 9 for j ← length[A] downto 1
10 do B[C[A[j]]] ← A[j]
11 C[A[j]] ← C[A[j]] – 1

1.8.1.2 Running time of Counting Sort

The for loop of lines 1-2 takes time Θ(k), the for loop of lines 3-4 takes time Θ(n),
the for loop of lines 6-7 takes time Θ(k), and the for loop of lines 9-11 takes time
Θ(n). Thus, the overall time is Θ(k+n). In practice, we usually use counting sort
when we have k = O(n), in which case the running time is Θ(n).
Example:

Figure: The operation of COUNTING-SORT on an input array A[1,… .. , 8], where each
element of A is a nonnegative integer no larger than k = 5. (a) The array A and the
auxiliary array C after line 4. (b) The array C after line 7. (c)-(e) The output array B and
the auxiliary array C after one, two, and three iterations of the loop in lines 9-11,
respectively. Only the lightly shaded elements of array B have been filled in. (f) The final
sorted output array B.

1.8.2 Radix Sort

Radix sort solves the problem of card sorting—by sorting on the least significant
digit first. The algorithm then combines the cards into a single deck, with the cards
in the 0 bin preceding the cards in the 1 bin preceding the cards in the 2 bin, and so
on. The process continues until the cards have been sorted on all d digits. Only d
passes through the deck are required to sort. The algorithm is

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
RadixSort(A, d)
for i=1 to d
StableSort(A) on digit i
Given n d-digit numbers in which each digit can take on up to k possible values,
RADIXSORT correctly sorts these numbers in Θ(d(n + k)) time.
Example:

Figure: The operation of radix sort on a list of seven 3-digit numbers. The leftmost
column is the input. The remaining columns show the list after successive sorts on
increasingly significant digit positions. Shading indicates the digit position sorted on to
produce each list from the previous one.

1.8.3 Bucket sort

Assumption: input is n reals from [0, 1)
Basic idea: Create n linked lists (buckets) to divide interval [0,1) into
subintervals of size 1/n. Add each input element to appropriate bucket and sort
buckets with insertion sort. Uniform input distribution ‡ O(1) bucket size.
Therefore the expected total time is O(n). These ideas wilreturn when we study
hash tables.
BUCKET-SORT(A)
1 n ← length[A]
2 for i ← 1 to n
3 do insert A[i] into list
4 for i ← 0 to n- 1
5 do sort list B[i] with insertion sort
6 concatenate the lists B[0], B[1], . . ., B[n - 1] together in order

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)
 Example:

Figure: The operation of BUCKET-SORT for n= 10. (a) The input array A(1… 10). (b) The
array B(0…… 9) of sorted lists (buckets) after line 8 of the algorithm. Bucket i holds values
in the half-open interval [i/10,.i + 1/10). The sorted output consists of a concatenation in
order of the lists B[0], B[1] B[9] To analyze the running time, observe that all lines except
line 5 take O (n) time in the worst case.

Prepared By:
Mr. Rohit Mishra
Assistant Professor (UIT)