Data Structures & Algorithms
Sorting
Lecturer: Duc Q. Nguyen
October 3rd, 2023
Motivations
Outline Taxonomies of sorting algorithms
“Failure is the opportunity to begin again Insertion
more intelligently.”
Henry Ford Selection
Exchange
Divide-and-conquer
Learning outcomes
● L.O.6.1 - Depict the working steps of sorting algorithms step-by-steps.
● L.O.6.2 - Describe sorting algorithms by using pseudocode.
● L.O.6.3 - Implement sorting algorithms using C/C++ .
● L.O.6.4 - Analyze the complexity and develop experiment (program) to evaluate sorting
algorithms.
● L.O.6.5 - Use sorting algorithms for problems in real-life.
● L.O.8.4 - Develop recursive implementations for methods supplied for the following
structures: list, tree, heap, searching, and graphs.
● L.O.1.2 - Analyze algorithms and use Big-O notation to characterize the computational
complexity of algorithms composed by using the following control structures: sequence,
branching, and iteration (not recursion).
Motivations
● Goal: Given some data points, arrange those data points into
ascending/descending order by some quantity
○ E.g. sort cards by face value or suit
Motivations
● Advantages:
○ Sorted data is often easier to work with
○ Sorted data can allow for faster insert/retrieval/deletion
Motivations
● Questions:
○ What are the different ways we can sort data?
○ What is the “best” strategy?
Taxonomies of sorting algorithms
Sorting
Internal External
Insertion Selection Exchange Divide-and-
conquer
- Insertion sort - Selection sort - Bubble sort
- Shell sort - Heap sort - Quick sort - Quick sort
- Merge sort
Insertion
● Insertion sort idea:
○ Split the array into 2 parts: sorted and unsorted
○ Insert the first element of the unsorted into the right position in the sorted
Insert
Sorted Unsorted
Insertion
● Insertion sort
23 78 45 8 32 56 12 64
23 78 45 8 32 56 12 64
23 78 45 8 32 56 12 64
23 45 78 8 32 56 12 64
8 23 45 78 32 56 12 64
Insertion
● Insertion sort
8 23 32 45 78 56 12 64
8 23 32 45 56 78 12 64
8 12 23 32 45 56 78 64
8 12 23 32 45 56 64 78
Insertion
● Shell sort idea:
○ For K reducing to 1:
■ Split the array into K segments.
■ Sort each segment using Insertion sort
Insertion
● Shell sort
23 78 45 8 32 56 12 64
8 32 45 12 64 56 23 78
23 78 45 8 32 56 12 64
12 8 23 56 32 64 45 78
Insertion
● Shell sort
12 8 23 56 32 64 45 78
8 12 23 32 45 56 64 78
Insertion
● Complexity:
○ Insertion sort:
■ f (n) = n(n+1)/2 = O(n2)
○ Shell sort
■ O(n1.25)
Selection
● Selection sort idea:
○ Split the array into 2 parts: sorted and unsorted
○ Select the candidate and swap it with the first element in the unsorted
Swap
Sorted Unsorted
Selection
● Selection sort
23 78 45 8 32 56 12 64
8 78 45 23 32 56 12 64
8 12 45 23 32 56 78 64
8 12 23 45 32 56 78 64
8 12 23 32 45 56 78 64
Selection
● Selection sort
8 12 23 32 45 56 78 64
8 12 23 32 45 56 78 64
8 12 23 32 45 56 64 78
8 12 23 32 45 56 64 78
Selection
● Heap sort idea:
○ The unsorted part is organized as a heap (the minimal value is on the root)
○ Select the root and exchange it with the beginning of the unsorted
Unsorted Sorted
Selection
● Heap sort
23 78 45 8 32 56 12 64
23
78 45
8 32 56 12
64
Selection
● Heap sort
78 64 56 23 32 45 12 8
78
64 56
23 32 45 12
8
Selection
● Heap sort
8 64 56 23 32 45 12 78
64 56
23 32 45 12
Selection
● Heap sort
64 32 56 23 8 45 12 78
64
32 56
23 8 45 12
Selection
● Heap sort
12 32 56 23 8 45 64 78
12
32 56
23 8 45
Selection
● Heap sort
56 32 45 23 8 12 64 78
56
32 45
23 8 12
Selection
● Heap sort
12 32 45 23 8 56 64 78
12
32 45
23 8
Selection
● Heap sort
45 32 12 23 8 56 64 78
45
32 12
23 8
Selection
● Heap sort
8 32 12 23 45 56 64 78
32 12
23
Selection
● Heap sort
32 23 12 8 45 56 64 78
32
23 12
8
Selection
● Heap sort
8 23 12 32 45 56 64 78
23 12
Selection
● Heap sort
23 8 12 32 45 56 64 78
23
8 12
Selection
● Heap sort
12 8 23 32 45 56 64 78
8
Selection
● Heap sort
8 12 23 32 45 56 64 78
8
Selection
● Heap sort
8 12 23 32 45 56 64 78
Selection
● Complexity:
○ Selection sort:
■ f (n) = n(n+1)/2 = O(n2)
○ Heap sort:
■ f (n) = O(nlog2n)
Exchange
● Bubble sort idea:
○ Split the array into 2 parts: sorted and unsorted
○ Sort the sized-2 array from the end to the beginning of the unsorted
Sorted Unsorted
Exchange
● Bubble sort
23 78 45 8 32 56 12 64
8 23 78 45 12 32 56 64
8 12 23 78 45 32 56 64
8 12 23 32 78 45 56 64
8 12 23 32 45 78 56 64
Exchange
● Bubble sort
8 12 23 32 45 78 56 64
8 12 23 32 45 56 78 64
8 12 23 32 45 56 64 78
8 12 23 32 45 56 64 78
Exchange
● Complexity:
○ Bubble sort:
■ f (n) = n(n+1)/2 = O(n2)
Divide-and-conquer
● Quick sort idea:
○ Choosing pivot
○ Divide array into 2 sub-arrays:
■ 1 less than & 1 greater than the pivot
○ Sort the sub-arrays by recursion
Divide-and-conquer
● Quick sort
23 78 45 8 32 56 12 64
8 12 23 78 45 32 56 64
8 12 23 78 45 32 56 64
8 12 23 45 32 56 64 78
Divide-and-conquer
● Quick sort
8 12 23 45 32 56 64 78
8 12 23 32 45 56 64 78
8 12 23 32 45 56 64 78
8 12 23 32 45 56 64 78
Divide-and-conquer
● Merge sort idea:
○ Split array into 2 half sub-arrays
○ Recursively sort the sub-arrays
Divide-and-conquer
● Merge sort
23 78 45 8 32 56 12 64
23 78 45 8 32 56 12 64
23 78 45 8 32 56 12 64
23 78 8 45 32 56 12 64
Divide-and-conquer
● Merge sort
23 78 8 45 32 56 12 64
8 23 45 78 12 32 56 64
8 12 23 32 45 56 64 78
Divide-and-conquer
● Complexity:
○ Quick sort:
■ Average: f (n) = O(nlog2n)
■ Worst: f (n) = O(n2)
○ Merge sort:
■ f (n) = O(nlog2n)
