Radix and Bucket Sort
Instructor: Ashok Singh Sairam
Lecture Plan
• Radix Sort
▪ Example
▪ Algorithm, Correctness and complexity
• Bucket Sort
▪ Example
▪ Algorithm, Correctness and complexity
MA512: Data Structures and Algorithms
2
Radix
• Radix - The base of a number system
• Represents keys as d-digit numbers in some base-k
key = x1x2...xd where 0≤xi≤k-1
• Example: key=15
key10 = 15, d=2, k=10 where 0≤xi≤9
key2 = 1111, d=4, k=2 where 0≤xi≤1
MA512: Data Structures and Algorithms
3
Radix Sort Analogy
• Sort a deck of 52 cards
• Intuitive Approach
▪ Sort by suit, then sort each pile and
then combine them
• Dis: When one pile is sorted the other
piles have to be kept aside and kept track
• Counter intuitive approach
▪ Sort by value
▪ The four separate piles are combined in order
and then sorted by suit.
MA512: Data Structures and Algorithms
4
Radix Sort: The idea
• Assumptions
d=Θ(1) and k =O(n)
• Sorting looks at one column at a
time
▪ For a d digit number, sort the least
significant digit first
▪ Continue sorting on the next least
significant digit, until all digits have
been sorted
▪ Requires only d passes through the list
MA512: Data Structures and Algorithms
5
RADIX-SORT
RADIX-SORT(A, d)
for i ← 1 to d
use a stable sort to sort array A on digit i
• 1 is the lowest order digit, d is the highest-order digit
6
Analysis of Radix Sort
• Given n d-digit numbers, where each digit may take up to k
possible values, RADIX-SORT correctly sorts the numbers in
Θ(d n + k ) if the stable sort it uses takes Θ(𝑛 + 𝑘) time
Proof
• One pass of sorting per digit takes Θ(𝑛 + 𝑘) assuming that
we use counting sort
• There are d passes (for each digit)
Θ(d n + k )
• Assuming d=O(1) and k=O(n), running time is O(n)
MA512: Data Structures and Algorithms
7
Correctness of Radix sort
• Claim: After ith Radix sort, least significant i digits are
sorted.
Proof:
• We use induction on number of passes through each digit
• Basis: If d = 1, there’s only one digit, trivial
• Inductive step: assume digits 1, 2, . . . , d-1 are sorted
MA512: Data Structures and Algorithms
8
Correctness of Radix sort – contd.
▪ Now sort on the d-th digit
▪ If ad < bd, sort will put a before b:
correct a < b regardless of the low-order digits
▪ If ad > bd, sort will put a after b:
correct a > b regardless of the low-order digits
▪ If ad = bd, sort will leave a and b in the
same order (stable!) and a and b are
already sorted on the low-order d-1 digits
MA512: Data Structures and Algorithms
9
Bucket Sort
• Assumption: the input is generated by a random process that
distributes elements uniformly over [0, 1)
• Idea:
▪ Divide [0, 1) into n equal-sized buckets
▪ Distribute the n input values into the buckets
▪ Sort each bucket (e.g., using quicksort)
▪ Go through the buckets in order, listing elements in each
one
• Input: A[1 . . n], where 0 ≤ A[i] < 1 for all i
• Output: elements A[i] sorted
• Auxiliary array: B[0 . . n - 1] of linked lists, each list initially
empty
10
Example - Bucket Sort
A 1 .78 B 0 Step 1: Initialize B[i] into
an empty list
2 .17 1
3 .39 2
4 .26 3
5 .72 4
6 .94 5
7 .21 6
8 .12 7
9 .23 8
10 .68 9
11
Example - Bucket Sort
A 1 .78 B 0 Step 1: Initialize B[i] into
an empty list
2 .17 1
3 .39 2 Step 2: Insert A[i] into
list B[ 𝑛𝐴[𝑖] ]
4 .26 3
5 .72 4
6 .94 5
7 .21 6
8 .12 7
9 .23 8
10 .68 9
12
Example - Bucket Sort
A 1 .78 B 0 /
2 .17 1 .17 .12 /
3 .39 2 .26 .21 .23 /
4 .26 3 .39 /
5 .72 4 /
6 .94 5 /
7 .21 6 .68 /
8 .12 7 .78 .72 /
9 .23 8 /
10 .68 9 .94 /
13
Example - Bucket Sort
.12 .17 .21 .23 .26 .39 .68 .72 .78 .94 /
0 /
1 .12 .17 / Step 1: Initialize B[i] into
an empty list
2 .21 .23 .26 /
Step 2: Insert A[i] into
.39 /
3 list B[ 𝑛𝐴[𝑖] ]
4 /
5 / Step 3: Sort each list
6 .68 /
.72 .78 /
Step 4: Concatenate the
7
lists from 0 to n – 1
8 / together, in order
9 .94 /
14
Bucket Sort: Algorithm
MA512: Data Structures and Algorithms
15
Correctness of Bucket sort
• Consider two elements A[i], A[ j]
• Assume without loss of generality that A[i] ≤ A[j]
• Then nA[i] ≤ nA[j]
▪ A[i] belongs to the same bucket as A[j] or to a bucket with a lower
index than that of A[j]
• If A[i], A[j] belong to the same bucket:
▪ sorting puts them in the proper order
• If A[i], A[j] are put in different buckets:
▪ concatenation of the lists puts them in the proper order
MA512: Data Structures and Algorithms
16
Analysis of Bucket Sort
O(n)
(n)
O(n)
(n)
(n)
MA512: Data Structures and Algorithms
17
Analysis of step 8 of the Algorithm
• Assume the elements are distributed uniformly
(i.e. n/k elements in each bucket)
𝑛 𝑛 𝑛 𝑛 [Since the n elements are
• 𝑛−1
σ𝑖=1 𝑐 𝑙𝑜𝑔
𝑘 𝑘
𝑘
= σ𝑖=1 𝑐 𝑙𝑜𝑔
𝑘 𝑘 distributed over k buckets]
𝑛
= 𝑐𝑛𝑙𝑜𝑔 [If k is O(n)]
𝑘
= 𝑐𝑛
MA512: Data Structures and Algorithms
18
Problems
• You are given 5 distinct numbers to sort. Describe
an algorithm which sorts them using at most 6
comparisons, or argue that no such algorithm
exists.
19
Problems
• Show how you can sort n integers in the range 1 to
n2 in O(n) time.
20
Conclusion
• Any comparison sort will take at least nlgn to sort an array
of n numbers
• We can achieve a better running time for sorting if we can
make certain assumptions on the input data:
▪ Counting sort: each of the n input elements is an integer
in the range [0 ... r] and r=O(n)
▪ Radix sort: the elements in the input are integers
represented with d digits in base-k, where d=Θ(1) and k
=O(n)
▪ Bucket sort: the numbers in the input are uniformly
distributed over the interval [0, 1)
21
Acknowledgement
• Dr George Bebis, Foundation Professor, Dept of
Computer Science and Engineering, University of
Nevada Reno
MA512: Data Structures and Algorithms
22
