ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
ECE750 Lecture 3: Divide and conquer,
Abstract Data Types, Lists, Iterators, Heaps
Todd Veldhuizen
[email protected]
Electrical & Computer Engineering
University of Waterloo
Canada
Sept. 28, 2007
1 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Part I
Divide and conquer
2 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Divide And Conquer
BinarySearch is an (arguably degenerate) instance of a
basic algorithm design pattern:
Divide And Conquer
1. If the problem is of trivial size, solve it and return.
2. Otherwise:
2.1 Divide the problem into several problems of smaller size.
2.2 Conquer these smaller problems by recursively applying
the divide and conquer pattern.
2.3 Combine the answers to the smaller problems into an
answer to the whole problem.
3 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Divide and Conquer
Binary Search as Divide-and-Conquer:
1. If the problem is of size n = 1, answer is obvious
return.
2. Otherwise:
Split the array into two halves. Principle:
(x in A[l ..h]) (x in A[l ..i ]) (x in A[i + 1..j ])
Search the two halves: for one half, call self recursively;
for other half, answer is false.
Combine the two answers: since one answer is always
false, and x or false is just x, simply return the
answer from the half we searched.
4 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Divide-and-Conquer Recurrences
If
1. The base cases (trivially small problems) require time
O(1), and
2. a problem of size n is split into k subproblems of size
s(n), and
3. splitting the problem into subproblems and combining
the answers takes time f (n)
then the general form of the time recurrence is
T(n) = c + kT(s(n)) + f (n)
e.g. for binary search we had k = 1 (we only had to search
one half), s(n) = n/2, and f (n) = 0.
5 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Strassen Matrix Multiplication
Recall that our MatrixMultiply routine took (n
3
)
time.
Can we do better? No one thought so until...
A landmark paper:
Volker Strassen, Gaussian Elimination is not optimal
(1969). [1]
An o(n
3
) divide-and-conquer approach to matrix
multiplication.
6 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
Strassen: If A, B are matrices of order m2
k+1
to be
multiplied, write
A =
A
11
A
12
A
21
A
22
B =
B
11
B
12
B
21
B
22
C =
C
11
C
12
C
21
C
22
where the A
ik
, B
ik
, C
ik
matrices are of order m2
k
...
7 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
Then compute
(7 subproblems)
I = (A
11
+ A
22
)(B
11
+ B
22
)
II = (A
21
+ A
22
)B
11
III = A
11
(B
12
B
22
)
IV = A
22
(B
11
+ B
21
)
V = (A
11
+ A
12
)B
22
VI = (A
11
+ A
21
)(B
11
+ B
12
)
VII = (A
12
A
22
)(B
21
+ B
22
)
(and combine)
C
11
= I + IV V + VII
C
21
= II + IV
C
12
= III + V
C
22
= I + III II + VI
8 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
1. Subproblems: compute 7 matrix multiplications of size
n/2.
2. Constructing the subproblems and combining the
answers is done with matrix additions/subtractions,
taking (n
2
) time.
3. Apply general divide-and-conquer recurrence with
k = 7, s(n) = n/2, f (n) = (n
2
):
T(n) = c + 7T(n/2) + (n
2
)
9 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
Recall that (n
2
) means some function f (n) for which
f (n)
n
2
is eventually restricted to some nite positive interval
[c
1
, c
2
].
Pick a value c > c
2
; then eventually f (n) cn
2
.
Solve recurrence:
T(n) = c + 7T(n/2) + cn
2
This will give an asymptotically correct bound, but possibly
T(n) is less than the actual time required for small n.
10 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
Let r () = T(2
), and = log
2
n. Recurrence becomes
r () = c + 7r ( 1) + c(2
)
2
= c + 7r ( 1) + c(4
Z-transform:
Z [c] =
c
1 z
1
Z [7r ( 1)] = 7z
1
R(z)
Z [c(4
)] =
c
1 4z
1
Z-transform version of recurrence is:
R(z) =
c
1 z
1
+ 7z
1
R(z) +
c
1 4z
1
Solve:
R(z) =
c2(1
5
2
z
1
)
(1 z
1
)(1 4z
1
)(1 7z
1
)
11 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Strassens method
Look at singularities: zero at z =
5
2
, poles at z = 1, 4, 7.
Pole at z = 7 is asymptotically dominant:
r () c
1
7
Change variables back: = log
2
n:
T(n) c
1
7
log
2
n
= c
1
n
log
2
7
= c
1
n
2.807
Strassen matrix multiplication takes (n
2.807
) time.
12 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Divide-and-conquer recurrences I
Our analysis of Strassens algorithm is easily generalized.
Consider a divide-and-conquer recurrence of the form
T(n) = kT(n/s) + (n
d
) (1)
To obtain an asymptotic upper bound we can solve the
related recurrence
T
(n) = kT
(n/s) + cn
d
(2)
It can be shown that T(n) T
(n).
Analyzing for the case when n = s
with N, we can
change variables to turn Eqn. (2) into:
r () = kr ( 1) + c(s
d
)
13 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Divide-and-conquer recurrences II
Z-transform:
R(z) = kz
1
R(z) +
c
1 s
d
z
1
Solve:
R(z) =
c
(1 kz
1
)(1 s
d
z
1
)
Which is the dominant pole? Depends on the values of
k, s, d.
Three cases: s
d
< k, s
d
= k, s
d
> k.
1. s
d
< k. Then the dominant pole is z = k. Get
r () k
Since n = s
, use = log
s
n:
T
(n) k
log
s
n
= (2
log k
)
log n
log s
= (2
log n
)
log k
log s
= n
log k
log s
14 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Divide-and-conquer recurrences III
2. s
d
= k. Then get a double pole at z = k. Recall that
Z
1
kz
1
(1 kz
1
)
2
= k
End up with
r () k
(n) (log
s
n)k
log
s
n
(log n)2
log klog n
log s
= n
log k
log s
log n
= n
log s
d
log s
log n = n
d
log n
3. s
d
> k. Then dominant singularity is z = s
d
. Get
r () (s
d
)
(n) (s
d
)
log
s
n
= (2
d log s
)
log n
log s
= n
d
15 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Master Theorem
Theorem (Master)
The solution to a divide-and-conquer recurrence of the form
T(n) = kT(n/s|) + (n
d
)
where s > 1, is
T(n) =
n
log k
log s
if s
d
< k
n
d
log n
if s
d
= k
(n
d
) if s
d
> k
Examples:
Binary search: k=1, s=2, d=0: second case,
log k
log s
= 0,
so T(n) = (n
0
log n) = (log n).
Strassen: k = 7, s = 2, d = 2: rst case,
log k
log s
2.807,
so T(n) = (n
2.807
).
16 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Part II
Abstract Data Types (ADTs)
17 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Abstract Data Types
A basic software engineering principle:
Separate the interface (what you can do) from
the implementation (how it is done.)
An abstract data type is a an interface to a collection of
data.
There may be numerous ways to implement an ADT,
each with dierent performance characteristics.
An ADT consists of
1. Some types that may be required to provide operations,
relations, and satisfy properties. e.g. a totally ordered
set.
2. An interface: the capabilities provided by the ADT.
18 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Abstract Data Types
Notation for types: T, f
1
, f
2
, , R
1
, R
2
, ) indicates
a set T together with
Some operators or functions f
1
, f
2
, . . .
Some relations R
1
, R
2
, .
This is the standard notation for a structure in logic:
could be
an algebra (functions but no relations) e.g. a eld
F, +, , ,
1
, 0, 1);
a relational structure (relations but no functions) e.g. a
total order T, );
some structure with both functions and relations e.g. an
ordered eld F, +, , ,
1
, 0, 1, )
Often a type is required to satisfy certain axioms, or
belong to a specied class of structures, e.g. (a eld, a
total order, a poset).
19 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: Dictionary[K, V]
Dictionary[K,V]
Stores a set of pairs (key, value), and nds values by
key. At most one value per key is permitted.
Types:
K): key type (e.g. a word).
V, 0): value type (e.g., a denition of a word). The
value 0 is a special value used to indicate absence of a
dictionary entry.
Operations:
insert(k, v): insert a key-value pair into the dictionary.
If an entry for the key is already present, it is replaced.
V nd(k): if (k, v) is in the dictionary, returns v;
otherwise returns 0.
remove(k): deletes the entry for key k, if present.
20 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Abstract Data Types
ADT
Implementation
Data structure
Linked list
Dictionary
55
k
k
k
k
k
k
k
k
k
//
))
S
S
S
S
S
S
S
S
S
Sorted array
Search tree
Implementation insert time nd time remove time
Linked list O(1) O(n) O(n)
Sorted array O(n) O(log n) O(n)
Search tree O(log n) O(log n) O(log n)
21 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]The role of ADTs in choosing data structures
Questions to consider when choosing a data structure:
1. What type of data do you need to store?
Does it have a natural total order? e.g. integers, strings
under lexicographic ordering, database keys, etc.
Does it bear some natural partial order?
Do elements represent points or regions in some metric
space, e.g., R
n
? (e.g., screen regions, boxes in a
three-dimensional space, etc.)
The order relation(s) or geometric organization of your
data may allow the use of ADTs that exploit those
properties to allow ecient access.
22 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]The role of ADTs in choosing data structures
2. What operations are required? Do you need to
determine whether a value is in the data set (search)?
insert, modify, delete elements?
iterate through the elements (order doesnt matter)?
iterate through the elements in some sorted order?
nd the biggest or smallest element?
nd elements that are close to some value?
These requirements can be compared with the
interfaces provided by ADTs, to decide what ADTs
might be suitable.
23 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]The role of ADTs in choosing data structures
3. What is the typical mixture of operations you will
perform? Will you
insert frequently?
delete frequently?
search frequently?
Dierent ADT implementations may oer dierent
performance characteristics. By understanding the
typical mixture of operations you can choose an
implementation with the most suitable performance
characteristics.
24 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
ADT: Array[V]
Array[V]
A nite sequence of cells, each containing a value.
Types :
V, 0): a value type, with a default value 0 used to
initialize cells.
Operations :
Array(n): create an array of length n, where n N is a
positive integer. Cells are initialized to 0.
integer length(): returns the length of the array
get(i ): returns the value of cell i . It is required that
0 i length() 1.
set(i , v): sets the value of cell i to v. It is required that
0 i length() 1.
25 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: Set[V]
Set[V]
Stores a set of values. Permits inserting, removing, and
testing whether a value is in the set. At most one
instance of a value can be in the set.
Types:
V): a value type
Operations:
insert(v): adds v to the set, if absent. Inserting an
element already in the set causes no change.
remove(v): removes v from the set, if present;
boolean contains(v): returns true if and only if v is in
the set.
26 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: Multiset[V]
Multiset[V]
Stores a multiset of values (i.e., with duplicate elements
permitted.) Permits inserting, removing, and testing
whether a value is in the set. Sometimes called a bag.
Types:
V): a value type
Operations:
insert(v): adds v to the multiset.
remove(v): removes an instance of v from the multiset,
if present;
boolean contains(v): returns true if and only if v is in
the set.
27 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Stacks, Queues, and Priority Queues
Informally, a queue is a collection of objects awaiting
their turn. e.g., customers queueing at a grocery store.
The queueing policy governs who goes next.
First In, First Out (FIFO): like a line at the grocery
store: the element that was added least recently goes
next.
Last In, First Out (LIFO): the item added most recently
goes next. A stack: like an in-box of work where new
items are placed on the top, and what ever is on the top
of the stack gets processed next.
Priority Queueing: items are associated with priorities;
the item with the highest priority goes next.
28 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: Queue[V]
Queue[V]
A FIFO queue (rst in, rst out) of objects.
Types :
V): a value type
Operations :
insert(v): adds the object v to the end of the queue
boolean isEmpty(): returns true just when the queue is
empty
V next(): returns and removes the value at the front of
the queue. It is an error to perform this operation when
the queue is empty.
29 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: Stack[V]
Stack[V]
A LIFO (last in, rst out) stack of objects.
Types :
V): a value type
Operations :
push(v): adds a value to the top of the stack.
boolean isEmpty(): returns true just when the stack
contains no elements.
V pop(): returns the value at the top of the stack. It is
an error to perform this operation when the stack is
empty.
30 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]ADT: PriorityQueue[P,V]
PriorityQueue[P,V]
A queue of objects, each with an associated priority, in
which an object with maximal priority is always chosen
next.
Types :
P, ): a priority type, with a total order .
V): a value type
Operations :
insert(p, v): insert a pair (p, v) where p P is a
priority, and v V is a value;
boolean isEmpty(): returns true just when the queue is
empty;
(P, V) next(): returns and removes the object at the
front of the queue, which is guaranteed to have
maximal priority. It is an error to perform this operation
on an empty queue.
31 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Part III
Linked Lists
32 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Data Structure: Linked List
Realizes a list of items, e.g., (2, 3, 5, 7, 11)
Inserting and removing elements at the front of the list
requires O(1) time.
Searching requires iterating through the list: O(n) time
List can be iterated through from front to back.
Basic building block is a node that contains:
data: A piece of data;
next: A pointer to the next node, or a null pointer if
the node is the end of the list.
33 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Data Structure: Linked List
public class LinkedList <T>
Node<T> head; // Pointer to the rst node in the list
...
class Node<T>
T data; // Data item
Node<T> next; // Next node in the list , if any
Node(T data, Node<T> next)
data = data;
next = next;
34 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Data Structure: Linked List I
Insert a new data element:
public void insert (T data)
head = new Node<T>(data, head);
Remove the front element, if any:
public T removeFirst()
if (head == null)
throw new RuntimeException(List is empty.);
else
T data = head.data;
head = head.next;
return data;
35 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Data Structure: Linked List II
Check if the list is empty:
public boolean isEmpty()
return (head == null);
36 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Implementing Stack[V] with a Linked List
The ADT Stack[V] is naturally implemented by a
linked list.
public class Stack<V>
LinkedList <V> list;
public void push(V v) list . insert (v);
public boolean isEmpty() return list . isEmpty();
public V pop() return list . removeFirst ();
push(v), isEmpty(), and pop() require O(1) time.
37 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Iterators
Iterator[V]
An iterator is an ADT that provides traversal through
some container. It abstracts away from the details of
how the data are stored, and presents a simple interface
for retrieving the elements of the container one at a
time.
Types:
V: the value type stored in the container
Operations:
Iterator(C): initialize the iterator to point to the rst
element in the container C;
boolean hasNext(): returns true just when there is
another element in the container;
V next(): returns the next element in the container,
and advances the iterator.
38 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]An Iterator for a linked list
public class ListIterator <T>
Node<T> node;
ListIterator ( LinkedList <T> list)
node = list . head;
boolean hasNext()
return (node == null);
T next()
if (node == null)
throw new RuntimeException(Tried to iterate past end of list );
T data = node.data;
node = node.next;
return data;
39 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Iterators
Example usage of an iterator:
ListIterator <Integer> iter = new ListIterator <Integer>( list );
while ( iter . hasNext())
System.out. println (The next element is + iter . next ());
40 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Implementing Queue[V] with a Linked List
Recall that a Queue[V] ADT requires rst-in rst-out
queueing. But, the list as we have shown it only
supports inserting and removing at one end.
Simple variant of a linked list: each list item contains a
pointer to the next and previous items.
41 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List
class BiNode<T>
T data; // Data item
BiNode<T> next; // Next node in the list , if any
BiNode<T> prev; // Previous node in the list , if any
BiNode(T data, BiNode<T> next)
data = data;
next = next;
next. prev = this;
42 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List
43 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List
public class BiLinkedList <T>
BiNode<T> head; // Pointer to the rst node in the list
BiNode<T> tail; // Pointer to the last node in the list
public void insert (T data)
head = new BiNode<T>(data, head);
if ( tail == null)
tail = head;
44 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List
public T removeLast()
if ( tail == null)
throw new RuntimeException(List is empty.);
else
T data = tail . data;
if ( tail . prev == null)
head = null;
tail = null;
else
tail = tail . prev;
tail . next = null;
return data;
45 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Implementing a Queue < V > with a
Bidirectional Linked List
public class Queue<V>
BiLinkedList <V> list;
public void insert (V v) list . insert (v);
public boolean isEmpty() return list . isEmpty();
public V next() return list . removeLast();
insert(v), isEmpty() and next() all take O(1) time.
46 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List
Bidirectional Linked List
Each node has a link to both the next and previous
items in the list.
We maintain a pointer to both the front and the back.
We can insert and remove items at both the front and
back of the list.
We will use Bidirectional Linked Lists to illustrate two
basic, but extremely useful principles:
1. Maintaining invariants of data structures;
2. Symmetry.
47 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List
We have encountered invariants already, in the
correctness sketch of binary search. That was an
invariant for a recursive algorithm, and was required to
be true of each recursive invokation of the function.
Here we discuss data structure invariants.
An invariant of a data structure is a property
that is required to be always true, except
when we are performing some transient
update operation.
Invariants help us to implement data structures
correctly: many basic operations can be viewed as
disruptions of the invariant (e.g., inserting an element)
after which we need to repair or maintain the invariant.
48 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List
As in a linked list, the basic building block of a
bidirectional linked list is a node.
class BiNode<T>
T data; / Data item /
BiNode<T> next; / Next node in the list , if any /
BiNode<T> prev; / Previous node in the list , if any /
BiNode(T data)
data = data;
next = null;
prev = null;
49 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List: Invariants
Lets look at a few examples to see what invariants
suggest themselves.
50 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Invariants
Here are the invariants we will use:
1. (front ,= null) implies (front.prev = null). (If the list is
nonempty, there is no element before the front element.)
2. (back ,= null) implies (back.next = null). (If the list is
nonempty, there is no element after the last element.)
3. (front = null) if and only if (back = null).
4. For any node x,
4.1 (x.next = null) implies x.next.prev = x;
4.2 (x.prev = null) implies x.prev.next = x.
51 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Symmetry
Bidirectional linked lists have a natural symmetry: if we
reverse the list by swapping the front back
pointers and each of the next prev pointers, we get
another bidirectional linked list.
Well call this the dual list.
The symmetry extends to operations on the list:
insertFront() is dual to insertBack(), and
removeFront() is dual to removeBack().
This kind of duality has the following nice property:
Carrying out a sequence of operations op
1
, . . . , op
k
gives the same result as
Taking the dual list, carrying out the dual sequence of
operations op
1
, . . . , op
2
, and taking the dual list.
52 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Symmetry
Example: starting from the list [1, 2], we can
insertBack(3) to give [1, 2, 3];
removeFront() to give [2, 3].
The dual version:
take the dual list [2, 1];
insertFront(3) to give [3, 2, 1];
removeBack() to give [3, 2];
take the dual list [2, 3].
Get the same answer both ways.
53 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Symmetry
Why we care about symmetry: if our implementation is
correct,
We can obtain the routine insertBack() by taking the
code for insertFront() and swapping front/back and
prev/next;
Ditto for removeBack() and removeFront();
The set of invariants should not change under swapping
front/back and prev/next. Example: For any node x,
1. (x.next = null) implies x.next.prev = x;
2. (x.prev = null) implies x.prev.next = x.
If we swap next/prev, we get
1. (x.prev = null) implies x.prev.next = x;
2. (x.next = null) implies x.next.prev = x.
54 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Implementation
For operations at the front of the list, there are three
cases to consider:
1. front=null (empty list)
2. front ,= null and front.next = null (one-element list)
3. front ,= null and front.next ,= null (multi-element list)
We need to consider each of these cases when we
implement, and ensure that in each case, the invariants
are maintained.
55 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bidirectional Linked List: Implementation
public void insertFront (T data)
BiNode<T> node = new BiNode<T>(data);
if ( front == null) / Case 1 /
front = node; / Both made nonnull for Inv. 3 /
back = node;
else / Case 2,3 /
front . prev = node; / Inv 4.1 /
node.next = front; / Inv 4.2 /
front = node;
56 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List: Implementation
public void insertBack(T data)
BiNode<T> node = new BiNode<T>(data);
if (back == null) / Case 1 /
back = node; / Both made nonnull for Inv. 3 /
front = node;
else / Case 2,3 /
back.next = node; / Inv 4.1 /
node.prev = back; / Inv 4.2 /
back = node;
57 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List: Implementation
public T removeFront()
if ( front == null) / Case 1 /
throw new RuntimeException(Empty list.);
else
T data = front.data;
front = front. next;
if ( front == null) / Case 2 /
back = null;
else
front . prev = null; / Case 3 /
return data;
58 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bidirectional Linked List: Implementation
public T removeBack()
if (back == null) / Case 1 /
throw new RuntimeException(Empty list.);
else
T data = back.data;
back = back.prev;
if (back == null) / Case 2 /
front = null;
else
back.next = null; / Case 3 /
return data;
59 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bibliography
Part IV
Heaps
60 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bibliography
Heaps
A heap dynamically maintains the maximum element in
a collection (or, dually, the minimum element). A
binary heap can:
Obtain the maximum element in O(1) time;
Remove the maximum element in O(log n) time;
Insert new element in O(log n) time.
Heaps are a natural implementation of the
PriorityQueue ADT.
There are several avours of heaps: binary heaps,
binomial heaps, bonacci heaps, pairing heaps. The
more sophisticated of these support merging (melding)
two heaps.
We will look at binary heaps.
61 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Binary Heap Invariants
1. A binary heap is a complete binary tree of height h 1,
plus a possibly incomplete level of height h lled from
left to right.
2. The key stored at each node is the key(s) stored in its
children (if used, you get a min heap.)
62 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Binary Heap
A binary heap may be stored as a (1-based) array,
where
Parent(j ) = j /2|
LeftChild(i ) = 2 i
RightChild(i ) = 2 i + 1
e.g., [17, 11, 13, 9, 6, 2, 12, 4, 3, 1] is an array
representation of the heap:
63 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Heap operations
To insert a key k into the heap:
Place k at the next available position.
Swap k with its parent(s) until the heap invariant is
satised. (Takes O(log n) time.)
The maximum element is just the key stored at the
root, which can be read o in O(1) time.
To delete the maximum element:
Place the key at the last heap position at the root
(overwriting the current maximum), and decrease the
size of the heap by one.
Choose the largest of the root and its two children, and
make this the root; perform this procedure recursively
until the heap invariant is satised.
64 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]
Bibliography
Heap: insert example
Example: insert 23 into the heap and restore the heap
invariant.
65 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Heap: delete-max example
To delete the max element, move the element from the
last position (2) to the root;
To restore heap invariant, swap root with the largest
child greater than it, if any, and repeat down the heap.
66 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Heapsort
Heaps can be used to implement a simple sorting
algorithm that runs in (n log n) worst case: given an
unordered array A[0..n 1],
1. Create an empty min heap.
2. Insert elements A[0], . . . , A[n 1] into the heap.
3. For i = 0 . . . n 1, remove the min element from the
heap and store it at A[i ].
67 / 68
ECE750 Lecture 3:
Divide and
conquer, Abstract
Data Types, Lists,
Iterators, Heaps
Todd Veldhuizen
[email protected]Bibliography
Bibliography I
[1] V. Strassen.
Gaussian elimination is not optimal.
Numerische Mathematik, 13:354356, 1969.
68 / 68
