Quick Sort

Fastest known sorting algorithm in practice

Average case: O(N log N)
Worst case: O(N2)
 But the worst case can be made exponentially
unlikely.
Another divide-and-conquer recursive algorithm,
like merge sort.

1
Quick Sort: Main Idea
1. If the number of elements in S is 0 or 1, then
return (base case).
2. Pick any element v in S (called the pivot).
3. Partition the elements in S except v into two
disjoint groups:
1. S1 = {x S – {v} | x  v}
2. S2 = {x S – {v} | x  v}
4. Return {QuickSort(S1) + v + QuickSort(S2)}

2
Quick Sort: Example

3
Example of Quick Sort...

4
Issues To Consider
How to pick the pivot?
 Many methods (discussed later)
How to partition?
 Several methods exist.
 The one we consider is known to give good results and
to be easy and efficient.
 We discuss the partition strategy first.

5
Partitioning Strategy
 For now, assume that pivot = A[(left+right)/2].
 We want to partition array A[left .. right].
 First, get the pivot element out of the way by
swapping it with the last element (swap pivot and
A[right]).
 Let i start at the first element and j start at the next-to-
last element (i = left, j = right – 1)
swap
5 7 4 6 3 12 19 5 7 4 19 3 12 6

pivot i j
6
Partitioning Strategy
 Want to have
 A[k]  pivot, for k < i  pivot  pivot
 A[k]  pivot, for k > j
 When i < j i j
 Move i right, skipping over elements smaller than the pivot
 Move j left, skipping over elements greater than the pivot
 When both i and j have stopped
A[i]  pivot
A[j]  pivot  A[i] and A[j] should now be swapped

5 7 4 19 3 12 6 5 7 4 19 3 12 6

i j i j 7
Partitioning Strategy (2)
 When i and j have stopped and i is to the left of j (thus
legal)
 Swap A[i] and A[j]
The large element is pushed to the right and the small element
is pushed to the left
 After swapping
A[i]  pivot
A[j]  pivot
 Repeat the process until i and j cross

swap
5 7 4 19 3 12 6 5 3 4 19 7 12 6

i j i j 8
Partitioning Strategy (3)
5 3 4 19 7 12 6
When i and j have crossed
 swap A[i] and pivot
i j
Result:
 A[k]  pivot, for k < i 5 3 4 19 7 12 6

 A[k]  pivot, for k > i

j i
swap A[i] and pivot

5 3 4 6 7 12 19

Break! 9
j i
Picking the Pivot
There are several ways to pick a pivot.

Objective: Choose a pivot so that we will

get 2 partitions of (almost) equal size.
Picking the Pivot (2)
Use the first element as pivot
 if the input is random, ok.
 if the input is presorted (or in reverse order)
all the elements go into S2 (or S1).
this happens consistently throughout the recursive calls.
results in O(N2) behavior (we analyze this case later).
Choose the pivot randomly
 generally safe,
 but random number generation can be expensive and
does not reduce the running time of the algorithm.

11
Picking the Pivot (3)
Use the median of the array (ideal pivot)
 The N/2 th largest element
 Partitioning always cuts the array into roughly half
 An optimal quick sort (O(N log N))
 However, hard to find the exact median

Median-of-three partitioning
 eliminates the bad case for sorted input.
 reduces the number of comparisons by 14%.

12
Median of Three Method
 Compare just three elements: the leftmost, rightmost and center
 Swap these elements if necessary so that
A[left] = Smallest
A[right] = Largest
A[center] = Median of three
 Pick A[center] as the pivot.
 Swap A[center] and A[right – 1] so that the pivot is at the second last
position (why?)

13
Median of Three: Example
A[left] = 2, A[center] = 13,
2 5 6 4 13 3 12 19 6
A[right] = 6

2 5 6 4 6 3 12 19 13 Swap A[center] and A[right]

2 5 6 4 6 3 12 19 13 Choose A[center] as pivot

pivot

2 5 6 4 19 3 12 6 13 Swap pivot and A[right – 1]

pivot
We only need to partition A[ left + 1, …, right – 2 ]. Why? 14
Quick Sort Summary
Recursive case: QuickSort( a, left, right )
pivot = median3( a, left, right );
Partition a[left … right] into a[left … i-1], i, a[i+1 … right];
QuickSort( a, left, i-1 );
QuickSort( a, i+1, right );

Base case: when do we stop the recursion?

 In theory, when left >= right.
 In practice, …
Small Arrays
For very small arrays, quick sort does not
perform as well as insertion sort
Do not use quick sort recursively for small arrays
 Use a sorting algorithm that is efficient for small
arrays, such as insertion sort.
When using quick sort recursively, switch to
insertion sort when the sub-arrays have between
5 to 20 elements (10 is usually good).
 saves about 15% in the running time.
 avoids taking the median of three when the sub-array
has only 1 or 2 elements.
16
Quick Sort: Pseudo-code

Choose pivot

Partitioning

Recursion

For small arrays 17

 Partitioning Part
 The partitioning code we just
saw works only if pivot is picked
as median-of-three.
 A[left]  pivot and A[right]  pivot
 Need to partition only
A[left + 1, …, right – 2]

 j will not run past the beginning

 because A[left]  pivot

 i will not run past the end

 because A[right-1] = pivot
18
 Quick Sort Faster Than Merge Sort
 Both quick sort and merge sort take O(N log N) in the
average case.
 But quick sort is faster in the average case:
 The inner loop consists of an increment/decrement (by
1, which is fast), a test and a jump.
 There is no extra juggling as in merge sort.

inner loop

19
Analysis
Assumptions:
 A random pivot (no median-of-three partitioning)
 No cutoff for small arrays ( to make it simple)

1. If the number of elements in S is 0 or 1, then return (base

case).
2. Pick an element v in S (called the pivot).
3. Partition the elements in S except v into two disjoint
groups:
1. S1 = {x S – {v} | x  v}
2. S2 = {x S – {v} | x  v}
4. Return {QuickSort(S1) + v + QuickSort(S2)}
20
Analysis (2)
 Running time
 pivot selection: constant time, i.e. O(1)
 partitioning: linear time, i.e. O(N)
 running time of the two recursive calls

 T(N)= T(i) + T(N – i – 1) + cN

 i: number of elements in S1
 c is a constant

21
Worst-Case Scenario
What will be the worst case?
 The pivot is the smallest element, all the time
 Partition is always unbalanced

22
Best-Case Scenario
What will be the best case?
 Partition is perfectly balanced.
 Pivot is always in the middle (median of the array).

T(N) = T(N/2) + T(N/2) + cN = 2T(N/2) + cN

 This recurrence is similar to the merge sort

recurrence.
 The result is O(NlogN).

23
Average-Case Analysis
Assume that each of the sizes for S 1 is equally
likely  has probability 1/N.
This assumption is valid for the pivoting and
partitioning strategy just discussed (but may not
be for some others),
On average, the running time is O(N log N).
Proof: pp 272–273, Data Structures and
Algorithm Analysis by M. A. Weiss, 2nd edition

24