0% found this document useful (0 votes)

24 views208 pages

Lecture 15 - Sorting

The document provides an overview of various sorting algorithms, including Bubble Sort, Insertion Sort, Selection Sort, and Shell Sort, detailing their processes and complexities. Each algorithm is explained with examples and their respective time complexities, highlighting their efficiency in different scenarios. The content is part of a lecture from the Faculty of Engineering at Assumption University, focusing on data structures and algorithms.

Uploaded by

Siwakorn Kheangtong

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

24 views208 pages

Lecture 15 - Sorting

Uploaded by

Siwakorn Kheangtong

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 208

Faculty of Engineering, Assumption University

CE2102: Data Lecture 15: Sorting

Structure & Algorithms
Sorting
❖ Sort a list of real number in ascending order
❖ Sorting algorithms:
❖ Bubble sort
❖ Insertion sort
❖ Selection sort
❖ Shell sort
❖ Heap sort
❖ Merge sort
❖ Quick sort
Bubble Sort

❖ Start from the beginning of a list

❖ Compare each pair of adjacent elements, swap them if
they are in reverse order
❖ Repeat step 1 if at least one swap has been done
Example: Bubble Sort

5 1 9 7 2 4 8
Example: Bubble Sort

Swap = False

5 1 9 7 2 4 8
Example: Bubble Sort

Swap = False

5 1 9 7 2 4 8
Example: Bubble Sort

Swap = True

1 5 9 7 2 4 8
Example: Bubble Sort

Swap = True

1 5 9 7 2 4 8
Example: Bubble Sort

Swap = True

1 5 9 7 2 4 8
Example: Bubble Sort

Swap = True

1 5 7 9 2 4 8
Example: Bubble Sort

Swap = True

1 5 7 9 2 4 8
Example: Bubble Sort

Swap = True

1 5 7 2 9 4 8
Example: Bubble Sort

Swap = True

1 5 7 2 9 4 8
Example: Bubble Sort

Swap = True

1 5 7 2 4 9 8
Example: Bubble Sort

Swap = True

1 5 7 2 4 9 8
Example: Bubble Sort

Swap = True

1 5 7 2 4 8 9
Example: Bubble Sort

Swap = False

1 5 7 2 4 8 9
Example: Bubble Sort

Swap = False

1 5 7 2 4 8 9
Example: Bubble Sort

Swap = False

1 5 7 2 4 8 9
Example: Bubble Sort

Swap = False

1 5 7 2 4 8 9
Example: Bubble Sort

Swap = False

1 5 2 7 4 8 9
Example: Bubble Sort

Swap = False

1 5 2 7 4 8 9
Example: Bubble Sort

Swap = True

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = True

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = True

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = True

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = False

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = False

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = False

1 5 2 4 7 8 9
Example: Bubble Sort

Swap = True

1 2 5 4 7 8 9
Example: Bubble Sort

Swap = True

1 2 5 4 7 8 9
Example: Bubble Sort

Swap = True

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = True

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = True

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = True

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Swap = False

1 2 4 5 7 8 9
Example: Bubble Sort

Finished

1 2 4 5 7 8 9
Complexity: Bubble Sort
❖ Worst case: Array is sorted in reverse order
❖ For an array of N numbers
❖ First iteration: N-1 swaps
❖ Second iteration: N-2 swaps
❖ Last iteration: 0 swap
❖ Total number of swaps = (N-1) + (N-2) + … + 1 =
N(N-1)/2 = O(N2)
Complexity: Bubble Sort

❖ However, for almost sorted array it gives O(N)

estimation, very good
❖ Running time heavily depends on the initial order of the
elements
❖ Big elements (rabbits) go up fast, while small ones
(turtles) go down very slow
Insertion Sort

❖ Divide array into two parts, sorted one and unsorted one
❖ At the beginning, sorted part contains first element of the
array and unsorted one contains the rest
❖ At every step, take the first element in the unsorted
part and insert it to the right place of the sorted one
❖ Stop when unsorted part becomes empty
Insertion Sort
Insertion Sort

sorted part x unsorted part

Example: Insertion Sort

5 1 9 7 2 4 8
Example: Insertion Sort

1 5 9 7 2 4 8
Example: Insertion Sort

1 5 7 9 2 4 8
Example: Insertion Sort

1 5 7 2 9 4 8
Example: Insertion Sort

1 5 2 7 9 4 8
Example: Insertion Sort

1 2 5 7 9 4 8
Example: Insertion Sort

1 2 5 7 4 9 8
Example: Insertion Sort

1 2 5 4 7 9 8
Example: Insertion Sort

1 2 4 5 7 9 8
Example: Insertion Sort

1 2 4 5 7 8 9
Example: Insertion Sort

1 2 4 5 7 8 9
Complexity: Insertion Sort
❖ Worst case: Array is sorted in reverse order
❖ For an array of N numbers
❖ First iteration: 1 swap
❖ Second iteration: 2 swaps
❖ Last iteration: N-1 swaps
❖ Total number of swaps = 1 + 2 + … + (N-2) + (N-1) =
N(N-1)/2 = O(N2)
Complexity: Insertion Sort

❖ However, for almost sorted array it gives O(N)

estimation, very good
❖ Normally used for sorting small arrays of data
❖ In practice, insertion sort outperforms most of the O(N2)
algorithms, like bubble sort or selection sort
Selection Sort

❖ Divide array into two parts, sorted one and unsorted one
❖ At the beginning, sorted part is empty, and unsorted
one contains the whole array
❖ At every step, select the minimum element in
the unsorted part and add it to the end of the sorted one
by swapping with the first number in the unsorted part
❖ Stop when unsorted part becomes empty
Selection Sort

sorted part unsorted part min unsorted part

Example: Selection Sort

5 1 9 7 2 4 8
Example: Selection Sort

1 5 9 7 2 4 8
Example: Selection Sort

1 2 9 7 5 4 8
Example: Selection Sort

1 2 4 7 5 9 8
Example: Selection Sort

1 2 4 5 7 9 8
Example: Selection Sort

1 2 4 5 7 8 9
Example: Selection Sort

1 2 4 5 7 8 9
Complexity: Selection Sort
❖ Worst case: Array is sorted in reverse order
❖ For an array of N numbers
❖ First iteration: selecting the minimum requires N-1 comparisons,
then making 1 swap
❖ Second iteration: selecting the minimum requires N-2 comparisons,
then making 1 swap
❖ Last iteration: selecting the minimum requires 0 comparison, then
making 0 swap
❖ Total number of comparisons = (N-1) + (N-2) + … + 1 = N(N-1)/2 =
O(N2)
Complexity: Selection Sort

❖ Selection sort requires at most N-1 swaps

❖ Very efficient when moving data is a slow process
Shell Sort

❖ A generalization of insertion sort that allows the

exchange of elements that are far apart
❖ Choose a sequence of positive integers known as
the increment sequence or gap sequence (H)
❖ Perform gap insertion sort with the gap being the largest
number in the increment sequence
❖ Continue gap insertion sort until finish with a gap of 1
Shell Sort
❖ Shell’s sequence:
❖ 1, 2, 4, 8, 16, …
❖ O(N2)
❖ Knuth’s sequence: ai+1=3*ai+1, a1=1
❖ 1, 4, 13, 40, 121, …
❖ O(N3/2)
❖ Segewick’s sequence:
❖ 1, 8, 23, 77, 281, …
❖ 1, 5, 19, 41, 109, …
❖ O(N4/3)
Example: Shell Sort

Gap sequence = 1, 3, 5

62 95 18 53 7 17 80 86 47 69
Example: Shell Sort

Gap = 5

62 95 18 53 7 17 80 86 47 69
Example: Shell Sort

Gap = 5

17 95 18 53 7 62 80 86 47 69
Example: Shell Sort

Gap = 5

17 95 18 53 7 62 80 86 47 69
Example: Shell Sort

Gap = 5

17 80 18 53 7 62 95 86 47 69
Example: Shell Sort

Gap = 5

17 80 18 53 7 62 95 86 47 69
Example: Shell Sort

Gap = 5

17 80 18 53 7 62 95 86 47 69
Example: Shell Sort

Gap = 5

17 80 18 47 7 62 95 86 53 69
Example: Shell Sort

Gap = 5

17 80 18 47 7 62 95 86 53 69
Example: Shell Sort

Gap = 3

17 80 18 47 7 62 95 86 53 69
Example: Shell Sort

Gap = 3

17 80 18 47 7 62 95 86 53 69
Example: Shell Sort

Gap = 3

17 80 18 47 7 62 69 86 53 95
Example: Shell Sort

Gap = 3

17 80 18 47 7 62 69 86 53 95
Example: Shell Sort

Gap = 3

17 7 18 47 80 62 69 86 53 95
Example: Shell Sort

Gap = 3

17 7 18 47 80 62 69 86 53 95
Example: Shell Sort

Gap = 3

17 7 18 47 80 53 69 86 62 95
Example: Shell Sort

Gap = 1

17 7 18 47 80 53 69 86 62 95
Example: Shell Sort

Gap = 1

7 17 18 47 53 62 69 80 86 95
Complexity: Shell Sort

❖ Complexity depends on the gap sequence

❖ Sub-quadratic time complexity
❖ Alternative to the O(N.log N) when the data is not very
large
Heap Sort

❖ Build heap
❖ Repeatedly remove the smallest element from the heap,
and insert it into the temporary array
Example: Heap Sort

6 5 3 1 8 7 2 4
Example: Heap Sort
6 5 3 1 8 7 2 4

5 3

1 8 7 2

4
Example: Heap Sort
6 5 3 1 8 7 2 4

5 3

1 8 7 2

4
Example: Heap Sort
6 5 3 1 8 7 2 4

5 3

1 8 7 2

4
Example: Heap Sort
6 5 2 1 8 7 3 4

5 2

1 8 7 3

4
Example: Heap Sort
6 5 2 1 8 7 3 4

5 2

1 8 7 3

4
Example: Heap Sort
6 1 2 5 8 7 3 4

1 2

5 8 7 3

4
Example: Heap Sort
6 1 2 4 8 7 3 5

1 2

4 8 7 3

5
Example: Heap Sort
6 1 2 4 8 7 3 5

1 2

4 8 7 3

5
Example: Heap Sort
1 6 2 4 8 7 3 5

6 2

4 8 7 3

5
Example: Heap Sort
1 4 2 6 8 7 3 5

4 2

6 8 7 3

5
Example: Heap Sort
1 4 2 5 8 7 3 6

4 2

5 8 7 3

6
Example: Heap Sort
1 4 2 5 8 7 3 6

4 2

5 8 7 3

6
Example: Heap Sort
1 4 2 5 8 7 3 6

4 2

5 8 7 3

6
Example: Heap Sort
4 2 5 8 7 3 6

4 2

5 8 7 3

6
1
Example: Heap Sort
6 4 2 5 8 7 3

4 2

5 8 7 3

1
Example: Heap Sort
2 4 6 5 8 7 3

4 6

5 8 7 3

1
Example: Heap Sort
2 4 3 5 8 7 6

4 3

5 8 7 6

1
Example: Heap Sort
2 4 3 5 8 7 6

4 3

5 8 7 6

1
Example: Heap Sort
2 4 3 5 8 7 6

4 3

5 8 7 6

1
Example: Heap Sort
4 3 5 8 7 6

4 3

5 8 7 6

1 2
Example: Heap Sort
6 4 3 5 8 7

4 3

5 8 7

1 2
Example: Heap Sort
3 4 6 5 8 7

4 6

5 8 7

1 2
Example: Heap Sort
3 4 6 5 8 7

4 6

5 8 7

1 2
Example: Heap Sort
3 4 6 5 8 7

4 6

5 8 7

1 2
Example: Heap Sort
4 6 5 8 7

4 6

5 8 7

1 2 3
Example: Heap Sort
7 4 6 5 8

4 6

5 8

1 2 3
Example: Heap Sort
4 7 6 5 8

7 6

5 8

1 2 3
Example: Heap Sort
4 5 6 7 8

5 6

7 8

1 2 3
Example: Heap Sort
4 5 6 7 8

5 6

7 8

1 2 3
Example: Heap Sort
4 5 6 7 8

5 6

7 8

1 2 3
Example: Heap Sort
5 6 7 8

5 6

7 8

1 2 3 4
Example: Heap Sort
8 5 6 7

5 6

1 2 3 4
Example: Heap Sort
5 8 6 7

8 6

1 2 3 4
Example: Heap Sort
5 7 6 8

7 6

1 2 3 4
Example: Heap Sort
5 7 6 8

7 6

1 2 3 4
Example: Heap Sort
5 7 6 8

7 6

1 2 3 4
Example: Heap Sort
7 6 8

7 6

1 2 3 4 5
Example: Heap Sort
8 7 6

7 6

1 2 3 4 5
Example: Heap Sort
6 7 8

7 8

1 2 3 4 5
Example: Heap Sort
6 7 8

7 8

1 2 3 4 5
Example: Heap Sort
6 7 8

7 8

1 2 3 4 5
Example: Heap Sort
7 8

7 8

1 2 3 4 5 6
Example: Heap Sort
8 7

1 2 3 4 5 6
Example: Heap Sort
7 8

1 2 3 4 5 6
Example: Heap Sort
8

1 2 3 4 5 6 7
Example: Heap Sort
8

1 2 3 4 5 6 7
Example: Heap Sort

1 2 3 4 5 6 7 8
Complexity: Heap Sort

❖ Build heap is O(N)

❖ Each delete min is O(log N)
❖ Complexity is O(N + N.log N)=O(N.log N)
❖ Slower than quick sort except in the worst case
Merge Sort

❖ Divide and conquer algorithm

❖ Recursively divide the unsorted list into N sublists, each
containing 1 element
❖ Recursively merge sublists to produce new sublists until
there is only 1 sublist remaining
❖ This will be the sorted list
Example: Merge Sort

6 5 3 1 8 7 2 4 9
Example: Merge Sort

5 6 3 1 8 7 2 4 9
Example: Merge Sort

3 5 6 1 8 2 7 4 9
Example: Merge Sort

1 3 5 6 8 2 4 7 9
Example: Merge Sort

1 2 3 4 5 6 7 8 9
Example: Merge Sort
Complexity: Merge Sort

❖ Merge sort has 2 steps

❖ Sort two arrays of size N/2, each taking time T(N/2)
❖ Combine those two sorted arrays of size N/2 in N
time steps
❖ T(N) = 2T(N/2) + N
❖ O(N.log N)
Quick Sort

❖ Divide and conquer algorithm

❖ Pick an element, called a pivot, from the list
❖ Reorder the list so that all elements with values less than
the pivot come before the pivot, while all elements with
values greater than the pivot come after it (equal values
can go either way)
❖ Recursively apply the above steps to both sublists
Example: Quick Sort

31 12 55 26 17 14 43 22
Example: Quick Sort

pivot

31 12 55 26 17 14 43 22
Example: Quick Sort

pivot

31 12 55 26 17 14 43 22