DSAL-211 LECT 5: Sorting
Algorithms
By
W.G. Gondwe
Ag. ICT Manager/Lecturer, Computer Networks
CS&IT Department
Malawi University of Science & Technology
Overview
• Understanding the Sorting Problem
• Sorting Terminology
• Sorting Algorithms
• Bubble Sort
• Selection Sort
• Insertion Sort
• Quick Sort
• Merge Sort (recursive)
The Sorting Problem
• Sorting: putting elements (of a list/document/chart/graph) in a
certain desired order
• One of the most important and ubiquitous computational
problems
• Widely used in applications/operation systems
• Used to pre-process input for other algorithms (e.g. binary search)
• Sorting criteria can be numeric or lexicographical
• Example sorting algorithms: bubble, selection, insertion,
quick, heap, comb, merge, bucket, shell
Sorting Terminology/Definitions
• Swap – switching positions of a pair of elements
• In-place sorting - elements of a list are sorted in their place and
minimal auxiliary space is required (input directly overwritten by
output)
• Internal/External sorting – Internal sorting utilizes available memory,
external sorting requires storage of items on secondary storage
• Stability – a sorting algorithm is stable if the relative positions of
elements with the same value (sorting key) is maintained i.e. any two
items are not swapped if they are equal
• Comparison-based sorting – examines elements only by using a
comparison operator e.g. <, >, ≥, ≤ (compare with search engine
sorting)
Bubble Sort
• A simple, in-place, comparison-based algorithm
• It is an iterative sorting algorithm that compares a pair of
elements at a time
• A pair out of order is swapped, otherwise the algorithm
proceeds to compare the next pair
• The algorithm continues to pass through the list until no swaps
are required
• Effectively, each out-of-order element “bubbles” to its rightful
position on each pass
Bubble Sort cont..
• Simulation: sort the list (1,5,8,3,4,7) in ascending order
1st iteration: (1,5,8,3,4,7) : 1 < 5 (no swap)
2nd iteration: (1,5,8,3,4,7) : 5 < 8 (no swap)
3rd iteration: (1,5,8,3,4,7) -> (1,5,3,8,4,7) : 8 > 3 (swap(8,3))
4th iteration: (1,5,3,8,4,7) -> (1,5,3,4,8,7) : 4 < 8 (swap(8,4))
5th iteration: (1,5,3,4,8,7) -> (1,5,3,4,7,8) : 7 < 8 (swap(8,7))
End of 1st pass
etc…
i.e. on first pass, largest item in the list bubbles to the end of the list
(ascending order)
sub swap(A[i],A[k])
Bubble Sort cont…
temp := A[i]
A[i] := A[j]
A[j] := temp
end sub
Algorithm:
procedure bubbleSort(list: A) 1
n := size(A) 2
for i:=0 to n-1 3
for j:=1 to n-1 4
if(A[j-1] > A[j]) then 5
swap(A[j-1], A[j]) 6
end if 7
end for 8
end for 9
end procedure
10
Bubble Sort (variation)
Source: https://en.wikipedia.org/wiki/Bubble_sort
Bubble Sort cont…
• Performance/Complexity
• Worst-case: O(n2)
• Avg. case: O(n2)
• Best case: O(n)
Worst case complexity reasoning:
In the worst case (reverse-sorted list), we get (n-1) + (n-2) + (n-3) + …+1
swaps
Hence quadratic
Selection Sort
• In-place, comparison-based algorithm
• It divides the list into two, sorted and unsorted sub-lists (sorted
list has size 0 at the onset)
• Sorts by finding the smallest (or largest in the case of
descending order) item in the list and inserting it into the sorted
sub-list
• Since each selection is the desired element for the position, a
sorted list is produced on the last iteration
Selection Sort cont..
• Simulation: sort the list (1,5,8,3,4,7) in ascending order
1st pass: (1,5,8,3,4,7) -> (1,5,8,3,4,7) : 1 selected, right position
2nd pass: (1,5,8,3,4,7) -> (1,3,8,5,4,7) : 3 selected, swapped with 5
3rd pass: (1,3,8,5,4,7) -> (1,3,4,5,8,7) : 4 selected, swapped with 8
4th pass: (1,3,4,5,8,7) -> (1,3,4,5,8,7) : 5 selected, right position
5th pass: (1,3,4,5,8,7) -> (1,3,4,5,7,8) : 7 selected, swapped with 8
List sorted!
Selection sort (animated)
Animation courtesy of https://en.wikipedia.org/wiki/Selection_sort
sub swap(A[i],A[k])
Selection Sort cont…
temp := A[i]
A[i] := A[j]
A[j] := temp
end sub
Algorithm (ascending order):
procedure selectionSort(list: A) 1
n := size(A) 2
for j:=1 to n-1 3
min_index := j 4
for i := j+1 to n-1 5
if(A[i] < A[min_index]) then 6
min_index = i 7
end if 8
end for 9
if(min_index != j) then 10
swap(A[i],A[min_index]) 11
end if 12
end for 13
end procedure 14
Selection Sort cont…
• Performance/complexity
• Worst-case: O(n2)
• Avg. case: O(n2)
• Best case: O(n2)
• Worst case complexity reasoning:
In the worst case (reverse-sorted list), we get (n-1) + (n-2) + (n-3) + …+1
comparisons
Hence quadratic
Insertion Sort
• In-place, iterative sorting algorithm
• Divides input into two sub-lists (sorted and unsorted)
• Sorted sub-list initially has the first element
• Elements are fetched from the unsorted sub-list and ‘inserted’
into the right position in the sorted sub-list
• Similar to sorting playing cards (by inserting each card into its
place)
• Final iteration produces a sorted list (all unsorted items inserted)
Insertion Sort (animation)
Starting with the first
element as the sorted
sublist, insert the next item
in its rightful position
Animation courtesy of https://en.wikipedia.org/wiki/Insertion_sort
Assuming a
Insertion Sort cont.. list index
starts from 1
• Algorithm (ascending order):
procedure insertionSort(list: A) 1
i := 2 2
n := size(A) 3
while i < n 4
j:= i 5
while j>1 && A[j-1] > A[j]) 6
swap(A[j], A[j-1]) 7
j := j-1 8
end while 9
i := i+1 10
end while 11
end procedure 12
Divide & Conquer Sorting
• Recap: Divide and conquer is an algorithm design paradigm
that solves a problem by recursively solving smaller versions of
the problem
• Smaller solutions are then combined to form a full solution
• Some sorting algorithms use the divide and conquer approach
• We take a closer look at Quick Sort and Merge Sort
Quick Sort
• A divide-and-conquer sorting algorithm
• Works by identifying an arbitrary pivot element and partitioning
the list around the pivot
• Elements smaller than pivot are put to the left and elements
greater than pivot are put on the right (assuming ASC order)
• The partitioning step is the most critical (conquering)
Quicksort cont..
Illustration:
Diagram courtesy of https://www.geeksforgeeks.org/quick-sort/
Quick Sort
Algorithm:
procedure quicksort(A, lo, hi)
if(lo < hi) then
p := partition(A, lo, hi)
quicksort(A, lo, p - 1 )
quicksort(A, p + 1, hi)
end if
end procedure
Hints:
Quick Sort 1. Partition scheme searches for a
new pivot position within the list
2. Index i keeps track of the right
Algorithm (partitioning): position to insert pivot element
3. Upon final iteration of j, the pivot
procedure partition(A, lo, hi) element is in its correct position
pivot := A[hi]
i := lo
for j := lo to hi
if(A[j] < pivot) then
swap(A[i],A[j])
i := i + 1
end if
end for
swap(A[i],A[hi])
return i // return new pivot index
end procedure
Quicksort cont…
Complexity:
• Best-case: O(n log n)
• Average case: O(n log n)
• Worst-case: O(n2)
• On average, divide step complexity is log n and conquer (partitioning) complexity
is n (linear), giving a log-linear complexity
• Best case complexity can degenerate to O(n) for some partitioning schemes.
• Choice of pivot: can be first/last element, median or random element
• Choice of partitioning scheme has a bearing on overall complexity
Merge Sort
• Another divide-and-conquer sorting algorithm
• Recursively partitions a list into two (divide) and merges them
(conquer)
• Merge: combine two lists such that the resultant list is sorted
• The merging step is the most critical (conquering)
Merge sort cont..
Illustration:
Diagram courtesy of https://en.wikipedia.org/wiki/Merge_sort
Merge sort cont..
Algorithm:
procedure merge_sort(Arr[], iBegin, iEnd, A[])
if(iEnd - iBegin < 2) then
return
end if
iMiddle := (iEnd + iBegin) / 2
merge_sort(A, iBegin, iMiddle, Arr[])
merge_sort(A, iMiddle, iEnd, Arr[])
TopDownMerge(Arr[], iBegin, iMiddle, iEnd, A)
End procedure
Merge sort cont..
Algorithm (merge):
procedure merge(Arr[], iBegin, iMiddle, iEnd, B[])
i = iBegin, j = iMiddle
for (k = iBegin; k < iEnd; k++)
if (i < iMiddle && (j >= iEnd || Arr[i] <= Arr[j])) then
B[k] := Arr[i]
i := i + 1
else
B[k] := Arr[j]
j := j + 1
End if
End for
End procedure
Merge sort cont…
Complexity:
• Best-case: O(n log n)
• Average case: O(n log n)
• Worst-case: O(n log n)
• On average, divide step complexity is log n and conquer (partitioning) complexity
is n (linear), giving a log-linear complexity
• Best case complexity can degenerate to O(n) for variants of merge schemes
Merge sort cont
• Theoretically, search complexity cannot get better than n log n
in the average case
• i.e. Merge sort presents one of the most efficient search
algorithms
Questions?
