CS 216
Design and Analysis of
Algorithms
Sorting
How fast can we sort ?
Selection Sort, Insertion Sort: O (n2)
Heap Sort, Merge sort: O (n log n)
Quicksort: O (n log n) [ best and average case] ,
O (n2) [worst case]
What is common to all these algorithms?
the sorted order they determine is based only on
comparisons between the input elements.
ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj
2
Lower-Bound for Sorting
• Theorem: to sort n elements, comparison sorts must
make (n log n) comparisons in the worst case.
• We can do better with linear sorting algorithms
Counting Sort
Radix Sort
• Make certain assumptions about the data.
• Linear sorts are NOT “comparison sorts”.
3
Counting Sort
Counting Sort
• Counting sort assumes that each of the
n input elements is an integer in the
range 0 to k, for some integer k.
• It assumes that the input consists of
integers in a small range.
• Counting sort determines, for each input
element x, the number of elements less
than x.
• It uses this information to place element
x directly into its position in the output
array. 5
Counting Sort
• To sort Array A [1..n], it requires two
other arrays:
the array B [1..n] holds the sorted
output,
the array C [0..k] provides temporary
working storage.
• C [i] holds the number of input elements
equal to i for each integer i = 0,1,2,…,
k.
6
Counting sort
algorithm
for i 1 to k
do C[i] 0
for j 1 to n
do C[A[ j]] ⊳ C[i] = |{key =
C[A[ j]] i}|
+1
for i 2 to k ⊳ C[i] = |{key
do C[i] i}|
C[i] + C[i–1]
for j n downto 1
September 26, 2005
j]]]
do B[C[A[ Copyright A[ j]
© 2001-5 Erik D. Demaine and Charles E. Leiserson L5.12
Counting Sort - example
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C:
B:
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.13
Loop
1
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 00 00 00 00
B:
for i 1 to k
do C[i] 0
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.14
Loop
2
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 00 00 00 11
B:
for j 1 to n
do C[A[ j]] C[A[ j]] + 1 ⊳ C[i] = |{key =
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.15
Loop
2
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 00 11
B:
for j 1 to n
do C[A[ j]] C[A[ j]] + 1 ⊳ C[i] = |{key =
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.16
Loop
2
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 11 11
B:
for j 1 to n
do C[A[ j]] C[A[ j]] + 1 ⊳ C[i] = |{key =
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.17
Loop
2
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 11 22
B:
for j 1 to n
do C[A[ j]] C[A[ j]] + 1 ⊳ C[i] = |{key =
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.18
Loop
2
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 22 22
B:
for j 1 to n
do C[A[ j]] C[A[ j]] + 1 ⊳ C[i] = |{key =
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.19
Loop
3
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 22 22
B: C': 11 11 22 22
for i 2 to k
do C[i] C[i] + C[i–1] ⊳ C[i] = |{key
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.20
Loop
3
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 22 22
B: C': 11 11 33 22
for i 2 to k
do C[i] C[i] + C[i–1] ⊳ C[i] = |{key
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.21
Loop
3
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 00 22 22
B: C': 11 11 33 55
for i 2 to k
do C[i] C[i] + C[i–1] ⊳ C[i] = |{key
i}|
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.22
Loop
4
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 11 33 55
B: 33 C': 11 11 22 55
for j n downto 1
do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.23
Loop
4
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 11 22 55
B: 33 44 C': 11 11 22 44
for j n downto 1
do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.24
Loop
4
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 11 22 44
B: 33 33 44 C': 11 11 11 44
for j n downto 1
do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.25
Loop
4
1 2 3 4 5 1 2 3 4
A: 44 11 33 44 33 C: 11 11 11 44
B: 11 33 33 44 C': 00 11 11 44
for j n downto 1
do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.26
Loop
4
1 2 3 4 5 1 2 3 4
A: 44 11 3 44 33 C: 00 11 11 44
3
B: 11 33 33 44 44 C': 00 11 11 33
for j n downto 1
do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] –
1
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.27
Analysi
s
for i 1 to k
(k) do C[i] 0
for j 1 to n
(n)
do C[A[ j]] C[A[ j]] + 1
for i 2 to k
(k) do C[i] C[i] + C[i–1]
for j n downto 1
(n) do B[C[A[ j]]] A[ j]
C[A[ j]] C[A[ j]] – 1
(n + k)
September 26, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L5.28
Counting Sort – running
time
• Thus, the overall time is
• In practice, we usually use counting sort
when we have
the running time is
24
Radix Sort
Radix Sort
• It is a sorting algorithm that is used to
sort numbers.
• The numbers are sorted from least
significant digit to the most significant
digit.
• If we are dealing with unsorted numbers
then we know that there are 10 digits
from 0 to 9 that are used to form any
number
we will need 10 bucket 0 – 9 to sort
26
numbers
Radix Sort
To sort numbers using Radix sort :
• Find the number of digit in the biggest
number
• If there are d number of digit in the
biggest number, it will need to perform d
number of passes.
• The remaining numbers must be padded
with zeros so they all have d digits
• Then we will take 10 buckets labeled
from 0 – 9 .
27
Radix Sort
• Sort the numbers from least significant
digit.
• After the sorting is complete, remove
the leading zeros.
• Radix sort algorithm :
The following procedure assumes that each
element
in the n-element array A has d digits
28
Radix Sort
What does “stable” means ?
• A sorting algorithm is said to be stable if
two objects with equal keys appear in
the same order in sorted output as they
appear in the input array to be sorted.
• Insertion sort, Merge Sort, Bubble Sort
are stable.
• Heap Sort, Quick Sort are not stable
29
Radix Sort
30
Radix Sort – example
10, 15, 1, 60, 5, 100 , 25, 50
• The biggest number of digits is 3 in 100
• We will do 3 passes to sort numbers
• Pad the remaining numbers:
010, 015, 001, 060 , 005, 100, 025, 050
• Use 10 bucket labelled : 0 – 9
0 1 2 3 4 5 6 7 8 9 31
Radix Sort – example
010, 015, 001, 060, 005, 100 , 025,
050
• Sort
050
numbers using the first digit :
100 025
060 005
010 001 015
0 1 2 3 4 5 6 7 8 9
010, 060, 100, 050, 001, 015, 005,
025
Pass 1 is completed 32
Radix Sort – example
010, 060, 100, 050, 001, 015, 005,
025
• Sort numbers using the second digit :
005
001 015
100 010 025 050 060
0 1 2 3 4 5 6 7 8 9
100, 001, 005, 010, 015, 025,
050 ,060
Pass 2 is completed 33
Radix Sort – example
100, 001, 005, 010, 015, 025,
050 ,060
Sort numbers using the third digit :
060
050
025
015
010
005 100
001
0 1 2 3 4 5 6 7 8 9
001, 005, 010, 015, 025, 050, 060,
100
Pass 3 is completed 34
Radix Sort – example
• stop after third pass
• remove the leading zeros :
1, 5, 10, 15, 25, 50, 60, 100
numbers are sorted !
35
Radix Sort – example 2
36
Radix Sort – running time
Given n numbers of d digits each, where each digit
may take up to k possible values, RADIX-SORT
correctly sorts the numbers in
One pass of sorting per digit takes assuming that
we use counting sort.
There are d passes (for each digit)
Assuming d=O(1) and k=O(n), running time is O(n)
37
Choosing a Sorting
Algorithm
Running time (best, worst, and
average).
Implementation
Stable or not
Space complexity (in-place or not)
Input size
Format of the input list
Nearly sorted list
38
