Data Structures and Algorithms (3)

(CS F211)

BITS Pilani Dr.N.L.Bhanu Murthy

Hyderabad Campus
 Sorting Algorithms

Comparison Based
3 7 9 17 5 2 21 18 33 4  Bubble Sort
 Quick Sort
 Insertion Sort
 Merge Sort
 Heap Sort
17 9 7 5 3 33 21 18 4 2

? Non-Comp Based
 Radix Sort
 Bucket Sort
33 21 18 17 9 7 5 4 3 2

Lower bound on comparison based algorithms

BITS Pilani, Hyderabad Campus
Sorting Applications

Obvious Applications:
Sort a list of names.
Organize an MP3 library.
Display Google PageRank results.

Problems become easy once items are in sorted order

Find the median.
Find the closest pair.
Binary search in a database.
Identify statistical outliers.
Find duplicates in a mailing list.
Non-obvious applications
Data compression.
Computer graphics.
Computational biology.
Supply chain management.
Simulate a system of particles.
Book recommendations on Amazon.
Load balancing on a parallel computer.

BITS Pilani, Hyderabad Campus

Bubble Sort

4
BITS Pilani, Hyderabad Campus
 Bubble Sort

Complexity is O(1) + f(n) { O(1) + O(n) O(1) + O(1) } = f(n) . O(n)

O(1) time for operations

like i = 1, i < n and i++)

5
BITS Pilani, Hyderabad Campus
Hence complexity is f(n) . O(n) = O(n) * O(n) = O(n2)
6
BITS Pilani, Hyderabad Campus
• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

77 42 35 12 101 5

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 Swap 42
77 77 35 12 101 5

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 7735 Swap 35
77 12 101 5

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 35 12 Swap 12
77 77 101 5

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 101 5

No need to swap

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 5 Swap 101
101 5

BITS Pilani, Hyderabad Campus

• Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

BITS Pilani, Hyderabad Campus

• Notice that only the largest value is correctly
placed
• All other values are still out of order
• So we need to repeat this process

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

BITS Pilani, Hyderabad Campus

• If we have N elements…

• And if each time we bubble an element, we

place it in its correct location…

• Then we repeat the “bubble up” process N – 1

times.

• This guarantees we’ll correctly

place all N elements.

BITS Pilani, Hyderabad Campus

 1 2 3 4 5 6
42 35 12 77 5 101

1 2 3 4 5 6
35 12 42 5 77 101

1 2 3 4 5 6
N-1

12 35 5 42 77 101

1 2 3 4 5 6
12 5 35 42 77 101

1 2 3 4 5 6
5 12 35 42 77 101

BITS Pilani, Hyderabad Campus

7 3 1 4 6 2 5

3 1 4 6 2 5 7

1 3 4 2 5 6 7

1 3 2 4 5 6 7

17
BITS Pilani, Hyderabad Campus
Merge Sort

Divide and Conquer cuts the problem in half each time, but uses the
result of both halves:
 cut the problem in half until the problem is trivial
 solve for both halves
 combine the solutions

Merge Sort, a sorting algorithm exploiting Divide and Conquer Technique

BITS Pilani, Hyderabad Campus

Merge Sort

 Developed merge sort for EDVAC in 1945

John von Neumann

(1903-1957)

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

The key to Merge Sort is merging two sorted lists into one, such that
if you have two lists X (x1x2…xm) and Y(y1y2…yn) the resulting
list is Z(z1z2…zm+n)

Example: L1 = { 3 8 9 }
L2 = { 1 5 7 }
merge(L1, L2) = { 1 3 5 7 8 9 }

What is complexity of the below mentioned algorithm?

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

What is complexity of the below mentioned algorithm?

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 3 10 23 54 Y: 1 5 25 75

Result:

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 3 10 23 54 Y: 5 25 75

Result: 1

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 10 23 54 Y: 5 25 75

Result: 1 3

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 10 23 54 Y: 25 75

Result: 1 3 5

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 23 54 Y: 25 75

Result: 1 3 5 10

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 54 Y: 25 75

Result: 1 3 5 10 23

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: 54 Y: 75

Result: 1 3 5 10 23 25

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: Y: 75

Result: 1 3 5 10 23 25 54

BITS Pilani, Hyderabad Campus

Merge two sorted lists into a single sorted list

X: Y:

Result: 1 3 5 10 23 25 54 75

BITS Pilani, Hyderabad Campus

Merge Sort

A divide-and-conquer algorithm:
• Divide the unsorted array into 2 halves until the sub-arrays only
contain one element
• Merge the sub-problem solutions together:
– Compare the sub-array’s first elements
– Remove the smallest element and put it into the result array
– Continue the process until all elements have been put into
the result array

37 23 6 89 15 12 2 19

BITS Pilani, Hyderabad Campus

Merge Sort Algorithm

Mergesort(Pass an array)
if array size > 1
Divide array in half
Call Mergesort on first half.
Call Mergesort on second half.
Merge two halves.

Merge(Pass two arrays)

Compare leading element in each array
Select lower and place in new array.
(If one input array is empty then place
remainder of other array in output array)

BITS Pilani, Hyderabad Campus

Merge Sort Algorithm

MergeSort(A, left, right) {

if (left < right) {
mid = floor((left + right) / 2);
MergeSort(A, left, mid);
MergeSort(A, mid+1, right);
Merge(A, left, mid, right);
}
}

// Merge() takes two sorted subarrays of A and

// merges them into a single sorted subarray of A
// (how long should this take?)

BITS Pilani, Hyderabad Campus

Analysis of Merge Sort

Statement Effort

MergeSort(A, left, right) { T(n)

if (left < right) { (1)
mid = floor((left + right) / 2); (1)
MergeSort(A, left, mid); T(n/2)
MergeSort(A, mid+1, right); T(n/2)
Merge(A, left, mid, right); (n)
}
}
So T(n) = (1) when n = 1, and
2T(n/2) + (n) when n > 1
 So what (more succinctly) is T(n)?

BITS Pilani, Hyderabad Campus

98 23 45 14 6 67 33 42

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23 98

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45 98

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

23 98 14 45

14 23 45 98

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6 67

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge

BITS Pilani, Hyderabad Campus

 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

Merge
BITS Pilani, Hyderabad Campus
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

BITS Pilani, Hyderabad Campus

 Assume n = 2k
98 23 45 14 6 67 33 42
1 2 k
Number of operations = (2 + 2 + … + 2 ) O(1) = O(n)
2 O(1)
1
98 23 45 14 6 67 33 42
log n
22 O(1) 98 23 45 14 6 67 33 42

23 O(1) 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 O(n)
Number of operations = (log n) (O(n)) = O(n logn) log n
14 23 45 98 6 33 42 67 O(n)

6 14 23 33 42 45 67 98 O(n)

BITS Pilani, Hyderabad Campus

98 23 45 14 6 67 33 42

6 14 23 33 42 45 67 98

BITS Pilani, Hyderabad Campus

Merge Sort, another example

179 285 285

310 254 310 351
179 310 652 423 450
652 351 861 520
254 652
450 861
520
310 285 310
179 179 351 351 423
652 652 423 254
254 861 450 450
520 520
861
310
285
310 310
285
285 351
179 351 423
652
652 652
351 861 254 450 520
652 351 450 520

BITS Pilani, Hyderabad Campus

Thank You!!

BITS Pilani, Hyderabad Campus