COMP171
Spring 2007
Priority Queue (Heap) &
Heapsort
Sorting III / Slide 2
Motivating Example
3 jobs have been submitted to a printer in the order A, B, C.
Sizes: Job A – 100 pages
Job B – 10 pages
Job C -- 1 page
Average waiting time with FIFO service:
(100+110+111) / 3 = 107 time units
Average waiting time for shortest-job-first service:
(1+11+111) / 3 = 41 time units
A queue be capable to insert and deletemin?
Priority Queue
Sorting III / Slide 3
Priority Queue
Priority queue is a data structure which allows at
least two operations
insert
deleteMin: finds, returns and removes the minimum
elements in the priority queue
deleteMin Priority Queue insert
Applications: external sorting, greedy algorithms
Sorting III / Slide 4
Possible Implementations
Linked list
Insert in O(1)
Find the minimum element in O(n), thus deleteMin is
O(n)
Binary search tree (AVL tree, to be covered later)
Insert in O(log n)
Delete in O(log n)
Search tree is an overkill as it does many other
operations
Eerr, neither fit quite well…
Sorting III / Slide 5
Background: Binary Trees
root
Has a root at the topmost
level Parent(x)
Each node has zero, one or
two children
x
A node that has no child is leaf leaf
called a leaf
For a node x, we denote the left(x)right(x)
left child, right child and the
parent of x as left(x), leaf
right(x) and parent(x),
respectively.
Sorting III / Slide 6
Height (Depth) of a Binary Tree
The number of edges on the longest path
from the root to a leaf.
Height = 4
Sorting III / Slide 7
Background: Complete Binary Trees
A complete binary tree is the tree
Where a node can have 0 (for the leaves) or 2 children and
All leaves are at the same depth
height no. of nodes
0 1
1 2
2 4
3 8
d 2d
No. of nodes and height
A complete binary tree with N nodes has height O(logN)
A complete binary tree with height h has 2d+1-1 nodes
Sorting III / Slide 8
Proof: O(logN) Height
Proof: a complete binary tree with N nodes
has height of O(logN)
1. Prove by induction that number of nodes at depth
d is 2d
2. Total number of nodes of a complete binary tree
of depth d is 1 + 2 + 4 +…… 2d = 2d+1 - 1
3. Thus 2d+1 - 1 = N
4. d = log(N+1)-1 = O(logN)
Side notes: the largest depth of a binary
tree of N nodes is O(N)
Sorting III / Slide 9
Binary Heap
Heaps are “almost complete binary trees”
All levels are full except possibly the lowest level
If the lowest level is not full, then nodes must be packed to
the left
Pack to the left
Structure properties
Has 2h to 2h+1-1 nodes with height h
The structure is so regular, it can be represented in an array
and no links are necessary !!!
Sorting III / Slide 10
Array Implementation of Binary Heap
A
B C A B C D E F G H I J
0 1 2 3 4 5 6 7 8 …
D E F G
H I J
For any element in array position i
The left child is in position 2i
The right child is in position 2i+1
The parent is in position floor(i/2)
A possible problem: an estimate of the maximum heap size is required
in advance (but normally we can resize if needed)
Note: we will draw the heaps as trees, with the implication that an actual
implementation will use simple arrays
Side notes: it’s not wise to store normal binary trees in arrays, coz it
may generate many holes
Sorting III / Slide 11
Back to Priority Queues
1 4
2 5 2 5
4 3 6 1 3 6
A heap Not a heap
Heap-order property: the value at each node is less
than or equal to the values at both its descendants
It is easy (both conceptually and practically) to perform
insert and deleteMin in heap if the heap-order
property is maintained
Use of binary heap is so common for priority queue implemen-
tations, thus the word heap is usually assumed to be the
implementation of the data structure
Sorting III / Slide 12
Heap Properties
Heap supports the following operations
efficiently
Insert in O(logN) time
Locate the current minimum in O(1) time
Delete the current minimum in O(log N) time
Sorting III / Slide 13
Insertion
Algorithm
1. Add the new element to the next available
position at the lowest level
2. Restore the min-heap property if violated
General strategy is percolate up (or bubble up): if the
parent of the element is larger than the element, then
interchange the parent and child.
1 1 1
2 5 2 5 2 2.5
swap
4 3 6 4 3 6 2.5 4 3 6 5
Insert 2.5 Percolate up to maintain
the heap property
Sorting III / Slide 14
An Implementation Trick
Implementation of percolation in the insert
routine
by performing repeated swaps: 3 assignment state-
ments for a swap. 3d assignments if an element is
percolated up d levels
An enhancement: Hole digging with d+1 assignments
7 7 7 4
9 8 9 4 8 8
17 16 4 14 10 17 14 10 17 9 14 10
20 18 20 18 16 20 18 16
Dig a hole Compare 4 with 9 Compare 4 with 7
Compare 4 with 16
Sorting III / Slide 15
Insertion Complexity
7
9 8
A heap!
17 16 14 10
20 18
Time Complexity = O(height) = O(logN)
Sorting III / Slide 16
Insertion PseudoCode
void insert(const Comparable &x)
{
//resize the array if needed
if (currentSize == array.size()-1
array.resize(array.size()*2)
//percolate up
int hole = ++currentSize;
for (; hole>1 && x<array[hole/2];
hole/=2)
array[hole] = array[hole/2];
array[hole]= x;
}
Sorting III / Slide 17
deleteMin: First Attempt
Algorithm
1. Delete the root.
2. Compare the two children of the root
3. Make the lesser of the two the root.
4. An empty spot is created.
5. Bring the lesser of the two children of the empty
spot to the empty spot.
6. A new empty spot is created.
7. Continue
Sorting III / Slide 18
Example for First Attempt
1
2 5 2 5
4 3 6 4 3 6
2 1
5 3 5
4 3 6 4 6
Heap property is preserved, but completeness is not preserved!
Sorting III / Slide 19
deleteMin
1. Copy the last number to the root (i.e.
overwrite the minimum element stored there)
2. Restore the min-heap property by percolate
down (or bubble down)
Sorting III / Slide 20
The same ‘hole’ trick used in insertion can be used here too
Sorting III / Slide 21
deleteMin with ‘Hole Trick’
2
2 5 5
4 3 6 4 3 6
1. create hole 2. Compare children and tmp
tmp = 6 (last element) bubble down if necessary
2 2
3 5 3 5
4 6 4 6
3. Continue step 2 until 4. Fill the hole
reaches lowest level
Sorting III / Slide 22
deleteMin PseudoCode
void deleteMin()
{
if (isEmpty()) throw UnderflowException();
//copy the last number to the root, decrease array size by 1
array[1] = array[currentSize--]
percolateDown(1); //percolateDown from root
}
void percolateDown(int hole) //int hole is the root position
{
int child;
Comparable tmp = array[hole]; //create a hole at root
for( ; hold*2 <= currentSize; hole=child){ //identify child position
child = hole*2;
//compare left and right child, select the smaller one
if (child != currentSize && array[child+1] <array[child]
child++;
if(array[child]<tmp) //compare the smaller child with tmp
array[hole] = array[child]; //bubble down if child is smaller
else
break; //bubble stops movement
}
array[hole] = tmp; //fill the hole
}
Sorting III / Slide 23
Heapsort
(1) Build a binary heap of N elements
the minimum element is at the top of the heap
(2) Perform N DeleteMin operations
the elements are extracted in sorted order
(3) Record these elements in a second array and then
copy the array back
Sorting III / Slide 24
Build Heap
Input: N elements
Output: A heap with heap-order property
Method 1: obviously, N successive insertions
Complexity: O(NlogN) worst case
Sorting III / Slide 25
Heapsort – Running Time Analysis
(1) Build a binary heap of N elements
repeatedly insert N elements O(N log N) time
(there is a more efficient way, check textbook p223 if interested)
(2) Perform N DeleteMin operations
Each DeleteMin operation takes O(log N) O(N log N)
(3) Record these elements in a second array and then copy
the array back
O(N)
Total time complexity: O(N log N)
Memory requirement: uses an extra array, O(N)
Sorting III / Slide 26
Heapsort: No Extra Storage
Observation: after each deleteMin, the size of heap
shrinks by 1
We can use the last cell just freed up to store the element
that was just deleted
after the last deleteMin, the array will contain the elements
in decreasing sorted order
To sort the elements in the decreasing order, use a
min heap
To sort the elements in the increasing order, use a
max heap
the parent has a larger element than the child
Sorting III / Slide 27
Heapsort Example: No Extra Storage
Sort in increasing order: use max heap
Delete 97
Sorting III / Slide 28
Heapsort Example
Delete 16 Delete 14
Delete 10 Delete 9 Delete 8
Sorting III / Slide 29
Example (cont’d)
Sorting III / Slide 30
Preliminary Heap ADT
Template <typename Comparable>
class BinaryHeap
{
public:
BinaryHeap(int capacity=100);
explicit BinaryHeap(const vector<comparable> &items);
bool isEmpty() const;
void insert(const Comparable &x);
void deleteMin();
void deleteMin(Comparable &minItem);
void makeEmpty();
private:
int currentSize; //number of elements in heap
vector<Comparable> array; //the heap array
void buildHeap();
void percolateDown(int hole);
