Sorting in Linear Time

Heejin Park
Hanyang University
 Contents

Lower bounds for sorting

Counting sort

Radix sort

2
 Lower bounds for sorting

Comparison sorts
 Sorting algorithms using only comparisons to determine the
sorted order of the input elements.
 Use tests such as ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj.
 Heapsort, Mergesort, Insertion sort, Selection sort, Quicksort

Lower bounds for (comparison) sorting

 Any comparison sort must make Ω(n lg n) comparisons in
the worst case to sort n elements.

3
 Lower bounds for sorting

Comparison sort
 we assume without loss of generality that all of the input
elements are distinct.
 The comparisons ai ≤ aj, ai ≥ aj, ai > aj , and ai < aj are all
equivalent.
 We assume that all comparisions have the form ai ≤ aj

4
 The decision-tree model

Comparison sorts can be viewed in terms of decision trees.

 A full binary tree.

 Each leaf is a permutation of input elements.

 Each internal node i:j indicates a comparison ai ≤ aj .

5
A decision tree for insertion sort
 The decision-tree model

The left subtree of the node i:j includes all permutations for ai ≤ aj .
The right subtree includes all permutations for ai > aj .

A decision tree for insertion sort

6
 The decision-tree model

The execution of the sorting algorithm corresponds to tracing a

path from the root of the decision tree to a leaf.

A decision tree for insertion sort

7
 The decision-tree model

the worst-case number of comparisons

= the height of its decision tree.

A decision tree for insertion sort

8
 The decision-tree model

Theorem 8.1: Any comparison sort algorithm requires

Ω(n lg n) comparisons in the worst case.

Proof:
 Height: h, Number of element: n
 The number of leaves: n!
 Each permutations for n input elements should appear as leaves.
 n! ≤ 2h
 lg(n!) ≤ h
 Ω(n lg n) (by equation (3.18) : lg(n!) = Θ(nlgn)).

9
 Self-study

Exercise 8.1-1
 The smallest depth of a leaf in a decision tree

Exercise 8.1-3
 Decision tree existence

Exercise 8.1-4
 Lower bound of a decision tree

10
 Counting sort

Counting sort
 A sorting algorithm using counting.

A 0 1 1 0 1 1 0 1

B 0 0 0 1 1 1 1 1

 Each input element x should be located in the ith place

after sorting if the number of elements less than x is i-1.
11
 Counting sort

1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3

0 1 2 3 4 5
C 0
1
2 0 0
1
2 0
1
2
3 0 0
1

B 0 0 2 2 3 3 3 5
12
 Counting sort

Counting sort

 Stable
 Same values in the input array appear in the same order
in the output array.

A 2 5 3 0 2 3 0 3

B 0 0 2 2 3 3 3 5

13
 Counting sort

1 2 3 4 5 6 7 8
A 2 5 3 0 2 3 0 3

0 1 2 3 4 5
C 2 0 2 3 0 1

C’ 2 2 4 7 7 8
14
 Counting sort

1 2 3 4 5 6 7 8 0 1 2 3 4 5
A 2 5 3 0 2 3 0 3 C’ 2 2 4 7 7 8

1 2 3 4 5 6 7 8 0 1 2 3 4 5

B 3 C’ 2 2 4 6 7 8

1 2 3 4 5 6 7 8 0 1 2 3 4 5
B 0 3 C’ 1 2 4 6 7 8

1 2 3 4 5 6 7 8 0 1 2 3 4 5
B 0 3 3 C’ 1 2 4 5 7 8

15
 Counting sort
COUNTING-SORT(A, B, k)
1 for i = 0 to k
Θ(k)
2 C[i] = 0
Θ(n) 3 for j = 1 to A.length
4 C[A[j]] = C[A[j]] + 1
5 ▹ C[i] contains the number of elements equal to i.
6 for i = 1 to k
Θ(k)
7 C[i] = C[i] + C[i - 1]
8 ▹ C[i] contains the number of elements less than or equal to i.
9 for j = A.length downto 1
Θ(n) 10 B[C[A[j]]] = A[j]
11 C[A[j]] = C[A[j]] - 1

16
 Counting sort

The overall time is Θ(k+n) where k is the range of

input integers.

If k = O(n), the running time is Θ(n).

17
 Self-study

Exercise 8.2-1
 A counting-sort example

Exercise 8.2-3
 Counting-sort stability

Exercise 8.2-4
 A counting-sort application

18
 Radix sort

Radix sort (MSD  LSD)

326 326 326 326

453 453 435 435
608 435 453 453
835 608 608 608
751 690 690 690
435 751 704 704
704 704 751 751
690 835 835 835

19
 Radix sort

Radix sort (MSD  LSD)

326 690 704 326

453 751 608 435
608 453 326 453
835 704 835 608
751 835 435 690
435 435 751 704
704 326 453 751
690 608 690 835

20
 Radix sort

RADIX-SORT(A, d)
1 for i = 1 to d
2 use a stable sort to sort array A on digit i

RADIXSORT sorts in Θ(d(n + k)) time when n d-digit

numbers are given and each digit can take on up to k possible
values.

When d is constant and k = O(n), radix sort runs in linear time.

21
 Radix sort

Changing d and k

1326
4534 d=?
6018 k=?
8135

1326
4534 d=?
6018 k=?
8135
22
 Radix sort

Lemma 8.4 (Self-study)

Given n b-bit numbers and any positive integer r ≤
b, RADIX-SORT correctly sorts these numbers in
Θ((b/r)(n + 2r)) time.
b
1 0 1    1 0 1 1
0 1 1 0 0 1
     n
0 1 0 1 0 0
1 0 0    1 0 0 1
23
r  r r
 Radix sort

Computing optimal r minimizing (b/r)(n + 2r).

1. b < ⌊lg n⌋
for any value of r, (n + 2r) = Θ(n) because r ≤ b.
So choosing r = b yields a running time : (b/b)(n + 2b) = Θ(n),
which is asymptotically optimal.

24
 Radix sort

Computing optimal r minimizing (b/r)(n + 2r).

2. b ≥ ⌊lg n⌋
choosing r = ⌊lg n⌋ gives the best time to within a constant
factor, (b/lgn)(n+2lgn) = (b/lgn)(2n) = Θ(bn/ lg n).

As we increase r above ⌊lg n⌋, the 2r in the numerator

increases faster than the r in the dominator.
As we decrease r below ⌊lg n⌋, then the b/r term increases and
the n + 2r term remains at Θ(n).

25
 Radix sort

Compare radix sort with other sorting algorithms.

If b = O(lg n), we choose r ≈ lg n.
Radix sort: Θ(n)
Quicksort: Θ(n lg n)

26
 Radix sort

The constant factors hidden in the Θ-notation differ.

1. Radix sort may make fewer passes than quicksort over the
n keys, each pass of radix sort may take significantly longer.
2. Radix sort does not sort in place.

27
 Self-study

Exercise 8.3-1
 Radix sort example

Exercise 8.3-2
 Stability

Exercise 8.3-4
 Radix sort application

28