Trees

Zahra Farahi
[email protected]
Trees
A rooted tree data structure stores information in nodes
• Similar to linked lists:
• There is a first node, or root
• Each node has variable number of references to successors
• Each node, other than the root, has exactly one node pointing to it
Terminology
All nodes will have zero or more child nodes or children
• I has three children: J, K and L

For all nodes other than the root node, there is one
parent node
• H is the parent I
Terminology
The degree of a node is defined as the number of its children:
deg(I) = 3

Nodes with the same parent are siblings

• J, K, and L are siblings
 Terminology
Nodes with degree zero are also called leaf nodes

All other nodes are said to be internal nodes, that is, they are internal
to the tree
 Terminology
These trees are equal if the order of the children is ignored
• unordered trees

They are different if order is relevant (ordered trees)

• We will usually examine ordered trees (linear orders)
• In a hierarchical ordering, order is not relevant
Terminology
• The shape of a rooted tree gives a natural flow from the root
node, or just root
 Terminology
A path is a sequence of nodes
(a0, a1, ..., an)
where ak + 1 is a child of ak

The length of this path is n

E.g., the path (B, E, G) has length 2

 Terminology
Paths of length 10 (11 nodes) and 4 (5 nodes)
 Terminology
For each node in a tree, there exists a unique path from the root node to
that node

The length of this path is the depth of the node, e.g.,

• E has depth 2
• L has depth 3
 Terminology
• Nodes of depth up to 17
 Terminology
The height of a tree is defined as the maximum depth of any node
within the tree

The height of a tree with one node is 0

• Just the root node

For convenience, we define the height of the empty tree to be –1

 Terminology
The height of this tree is 17
Terminology
If a path exists from node a to node b:
• a is an ancestor of b
• b is a descendent of a

Thus, a node is both an ancestor and a descendant of itself

• We can add the adjective strict to exclude equality: a is a strict descendent of b
if a is a descendant of b but a ≠ b

The root node is an ancestor of all nodes

Terminology
The descendants of node B are B, C, D, E, F, and G:

The ancestors of node I are I, H, and A:

 Terminology
All descendants (including itself) of the indicated node
 Terminology
All ancestors (including itself) of the indicated node
 Terminology
Another approach to a tree is to define the tree recursively:
• A degree-0 node is a tree
• A node with degree n is a tree if it has n children and all of its children are
disjoint trees (i.e., with no intersecting nodes)

Given any node a within a tree

with root r, the collection of a and
all of its descendants is said to
be a subtree of the tree with
root a
 Tree traversals
Types of Traversals:

The breadth-first traversal visits all nodes at depth k before proceeding

onto depth k + 1
 The depth-first traversals: visit always go as deep as possible before
visiting other siblings:
Breadth-First Traversal
Breadth-first traversals visit all nodes at a given depth
• Can be implemented using a queue
• Run time is Q(n)
• Memory is potentially expensive: maximum nodes at a given depth
• Order: A B H C D G I E F J K
Breadth-First Traversal
The implementation was already discussed:
• Create a queue and push the root node onto the queue
• While the queue is not empty:
• Push all of its children of the front node onto the queue
• Pop the front node
Backtracking
To discuss depth-first traversals, we will define a backtracking
algorithm for stepping through a tree:
• At any node, we proceed to the first child that has not yet been visited
• Or, if we have visited all the children (of which a leaf node is a special case),
we backtrack to the parent and repeat this decision making process
We end once all the children
of the root are visited
Depth-first Traversal
We define such a path as a depth-first traversal

We note that each node could be visited twice in such a scheme

• The first time the node is approached (before any children)
• The last time it is approached (after all children)
Definition
This is not a binary tree:
Definition
The arbitrary number of children in general trees is often
unnecessary—many real-life trees are restricted to two
branches
• Expression trees using binary operators
• An ancestral tree of an individual, parents, grandparents, etc.
• Phylogenetic trees
• Lossless encoding algorithms

There are also issues with general trees:

• There is no natural order between a node and its children
Definition
A binary tree is a restriction where each node has exactly two
children:
• Each child is either empty or another
binary tree
• This restriction allows us to label the
children as left and right subtrees

At this point, recall that lg(n) = Q(logb(n)) for any b

Definition
We will also refer to the two sub-trees as
• The left-hand sub-tree, and
• The right-hand sub-tree
Definition
Sample variations on binary trees with five nodes:
Definition
A full node is a node where both the left and right sub-trees are
non-empty trees

Legend: full nodes neither leaf nodes

Definition
An empty node or a null sub-tree is any location
where a new leaf node could be appended
Definition
A full binary tree is where each node is:
• A full node, or
• A leaf node

These have applications in

• Expression trees
• Huffman encoding
Binary tree- class Node
Size
The recursive size function runs in Q(n) time and
Q(h) memory
• These can be implemented to run in Q(1)
Height
The recursive height function also runs in Q(n) time
and Q(h) memory
• Later we will implement this in Q(1) time
Height
Definition
Standard definition:
• A perfect binary tree of height h is a binary tree where
• All leaf nodes have the same depth h
• All other nodes are full
Definition
Recursive definition:
• A binary tree of height h = 0 is perfect
• A binary tree with height h > 0 is a perfect if both sub-trees are prefect
binary trees of height h – 1
Examples
Perfect binary trees of height h = 0, 1, 2, 3 and 4
Examples
Perfect binary trees of height h = 3 and h = 4
• Note they’re the wrong-way up…
Theorems
We will now look at four theorems that describe the properties
of perfect binary trees:
• A perfect tree has 2h + 1 – 1 nodes
• The height is Q(ln(n))
• There are 2h leaf nodes
• The average depth of a node is Q(ln(n))

The results of these theorems will allow us to determine the

optimal run-time properties of operations on binary trees
Definition

A complete binary tree filled at each depth from left to right:

Definition

The order is identical to that of a breadth-first traversal

 Recursive Definition

Recursive definition: a binary tree with a single node is a

complete binary tree of height h = 0 and a complete binary
tree of height h is a tree where either:
• The left sub-tree is a complete tree of height h – 1 and the right
sub-tree is a perfect tree of height h – 2, or
• The left sub-tree is perfect tree with height h – 1 and the right sub-
tree is complete tree with height h – 1
Array storage

We are able to store a complete tree as an array

• Traverse the tree in breadth-first order, placing the entries into the array
Array storage

We can store this in an array after a quick traversal:

Array storage

To insert another node while maintaining the complete-binary-

tree structure, we must insert into the next array location
Array storage

To remove a node while keeping the complete-tree structure,

we must remove the last element in the array
Array storage

Leaving the first entry blank yields a bonus:

– The children of the node with index k are in 2k and 2k + 1
– The parent of node with index k is in k ÷ 2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Array storage

Leaving the first entry blank yields a bonus:

– In C++, this simplifies the calculations: parent = k >> 1;
left_child = k << 1;
right_child = left_child | 1;

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Array storage

For example, node 10 has index 5:

• Its children 13 and 23 have indices 10 and 11, respectively

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Array storage
For example, node 10 has index 5:
• Its children 13 and 23 have indices 10 and 11, respectively
• Its parent is node 9 with index 5/2 = 2

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Array storage
Question: why not store any tree as an array using breadth-first
traversals?
• There is a significant potential for a lot of wasted memory

Consider this tree with 12 nodes would require an array of size

32
• Adding a child to node K doubles the required memory
Array storage

In the worst case, an exponential

amount of memory is required

These nodes would be stored

in entries 1, 3, 6, 13, 26, 52, 105
N-ary Trees
One generalization of binary trees are a class of trees termed
N-ary trees:
• A tree where each node had N sub-trees, any of which may be may be
empty trees
Ternary Trees
Examples of a ternary (3-ary) trees
• We don’t usually indicate empty sub-trees
Quaternary Trees
Example of a perfect quaternary (4-ary) tree

Under certain conditions, the number of operations required

may be slightly fewer with quaternary trees (e.g., with heaps)
• The asymptotic behaviour is similar
Perfect N-ary Trees
Each node can have N children
The number of nodes in a perfect N-ary tree of height h is

1 + N + N2 + N3 ⋅⋅⋅ + Nh

This is a geometric sum, and therefore, the number of nodes is

h
N h 1  1
n N  k

k 0 N 1
Perfect N-ary Trees
Solving this equation for h, a perfect N-ary tree with n nodes has
a height given by

h  log N n( N  1)  1  1
 Complete N-ary Trees
A complete N-ary tree has height
h  log N  N  1 n 

Like complete binary trees, complete N-ary trees can also be stored
efficiently using an array:
• Assume the root is at index 0  k  1
 N 
• The parent of a node with index k is at location  

• The children of a node with index k are at locations

kN + j
for j = 1, …, N
 Applications
One application of an 26-ary trees is a trie where the root represents the start of
each valid word, and the different sub-trees represent next letters in valid words
• Consider the words in the phrase
“The fable then faded from
my thoughts and memory.”
• All 26 sub-trees are only
shown for the root node, but
all nodes have 26 sub-trees
• Some nodes are marked as
terminal indicating the end of a
valid word
• These terminal points could be used to store a list
of all places in a document where the word occurs
• Consider the ultimate index to a book
Requirement for Balance
We want to ensure that the run times never fall into w(ln(n))

Requirement:
• We must maintain a height which is Q(ln(n))

To do this, we will define an idea of balance

Examples
For a perfect tree, all nodes have the same number of
descendants on each side

Perfect binary trees are balanced while linked lists are not
Examples
This binary tree would also probably not be considered to be
“balanced” at the root node
Examples
How about this example?
• The root seems balanced, but what about the left sub-tree?
Definition for Balance
We must develop a quantitative definition of balance which can be
applied

Balanced may be defined by:

• Height balancing: comparing the heights of the two sub trees
• Null-path-length balancing: comparing the null-path-length of each of the two sub-
trees (the length to the closest null sub-tree/empty node)
• Weight balancing: comparing the number of null sub-trees in each of the two sub
trees

We will have to mathematically prove that if a tree satisfies the definition

of balance, its height is Q(ln(n))
Definition for Balance
We will see one definition of height balancing:
• AVL trees

We will also look at B+-trees

• Balanced trees, but not binary trees
Red-Black Trees
Red-black trees maintain balance by
• All nodes are colored red or black (0 or 1)

Requirements:
• The root must be black
• All children of a red node
must be black
• Any path from the root
to an empty node must
have the same number
of black nodes
Red-Black Trees
Red-black trees are null-path-length balanced in that the null-
path length going through one sub-tree must not be greater
than twice the null-path length going through the other
• A perfect tree of height h has a null-path length of h + 1
• Any other tree of height h must have a null-path-length less than h + 1
Weight-Balanced Trees
Recall: an empty node/null subtree is any position within a
binary tree that could be filled with the next insertion:
• This tree has 9 nodes and 10 empty nodes:
Weight-Balanced Trees
The ratios of the empty nodes at the root node are 5/10 and
5/10
Weight-Balanced Trees
The ratios of the empty nodes at this node are 2/5 and 3/5
Weight-Balanced Trees
The ratios of the empty nodes at this node, however, are 4/5
and 1/5
Weight-Balanced Trees
BB(a) trees (0 < a ≤ 1/3) maintain weight balance requiring that
neither side has less than a a proportion of the empty nodes,
i.e., both proportions fall in [a, 1 – a]
• With one node, both are 0.5

• With two, the proportions are 1/3 and 2/3

Abstract Sorted Lists
Previously, we discussed Abstract Lists: the objects are
explicitly linearly ordered by the programmer

We will now discuss the Abstract Sorted List:

• The relation is based on an implicit linear ordering

Certain operations no longer make sense:

• push_front and push_back are replaced by a generic insert
Abstract Sorted Lists
Queries that may be made about data stored in a Sorted List
ADT include:
• Finding the smallest and largest values
• Finding the kth largest value
• Find the next larger and previous smaller objects of a given object
which may or may not be in the container
• Iterate through those objects that fall on an interval [a, b]
Implementation
If we implement an Abstract Sorted List using an array or a
linked list, we will have operations which are O(n)
• As an insertion could occur anywhere in a linked list or array, we must
either traverse or copy, on average, O(n) objects
Background
Recall that with a binary tree, we can dictate an order on the
two children

We will exploit this order:

• Require all objects in the left sub-tree to be less than the object stored
in the root node, and
• Require all objects in the right sub-tree to be greater than the object in
the root object
Binary Search Trees
Graphically, we may relationship

• Each of the two sub-trees will themselves be binary search trees

Binary Search Trees
Notice that we can already use this structure for searching:
examine the root node and if we have not found what we are
looking for:
• If the object is less than what is stored in the root node, continue
searching in the left sub-tree
• Otherwise, continue searching the right sub-tree

With a linear order, one of the following three must be true:

a<b a=b a>b
Definition
Thus, we define a non-empty binary search tree as a binary tree
with the following properties:
• The left sub-tree (if any) is a binary search tree and all values are less
than the root value, and
• The right sub-tree (if any) is a binary search tree and all values are
greater than the root value
Examples
Here are other examples of binary search trees:
Examples
Unfortunately, it is possible to construct degenerate binary search trees

• This is equivalent to a linked list, i.e., O(n)

Examples
All these binary search trees store the same data
Duplicate values
We will assume that in any binary tree, we are not storing
duplicate values unless otherwise stated
• In reality, it is seldom the case where duplicate values in a container
must be stored as separate entities

You can always consider duplicate values with modifications to

the algorithms we will cover
 Finding the Minimum Object

• The run time O(h)

 Finding the Maximum Object

• The extreme values are not necessarily leaf nodes

Find
To determine membership, traverse the tree based on the linear
relationship:
• If a node containing the value is found, e.g., 81, return 1

• If an empty node is reached, e.g., 36, the object is not in the tree:
Find

• The run time is O(h)

Insert
Recall that a Sorted List is implicitly ordered
• It does not make sense to have member functions such as push_front
and push_back
• Insertion will be performed by a single insert member function which
places the object into the correct location
 Insert
An insertion will be performed at a leaf node:
• Any empty node is a possible location for an insertion

The values which may be inserted at any empty node depend

on the surrounding nodes
Insert
For example, this node may hold 48, 49, or 50
Insert
An insertion at this location must be 35, 36, 37, or 38
Insert
This empty node may hold values from 71 to 74
Insert
Like find, we will step through the tree
• If we find the object already in the tree, we will return
• The object is already in the binary search tree (no duplicates)
• Otherwise, we will arrive at an empty node
• The object will be inserted into that location
• The run time is O(h)
Insert
In inserting the value 52, we traverse the tree until we reach an
empty node
• The left sub-tree of 54 is an empty node
Insert
A new leaf node is created and assigned to the member
variable left_tree
Insert
In inserting 40, we determine the right sub-tree of 39 is an
empty node
Insert
A new leaf node storing 40 is created and assigned to the
member variable right_tree
Insert
Insert
It is assumed that if neither of the conditions:
obj < value()
obj > value()

then obj == value() and therefore we do nothing

• The object is already in the binary search tree
Insert
Blackboard example:
• In the given order, insert these objects into an initially empty binary
search tree:
31 45 36 14 52 42 6 21 73 47 26 37 33 8
• What values could be placed:
• To the left of 21?
• To the right of 26?
• To the left of 47?
• How would we determine if 40 is in this binary search tree?
• Which values could be inserted to increase the height of the tree?
Erase
A node being erased is not always going to be a leaf node
There are three possible scenarios:
• The node is a leaf node,
• It has exactly one child, or
• It has two children (it is a full node)
Erase
A leaf node simply must be removed and the appropriate
member variable of the parent is set to nullptr
• Consider removing 75
Erase
The node is deleted and left_tree of 81 is set to nullptr
Erase
Erasing the node containing 40 is similar
Erase
The node is deleted and right_tree of 39 is set to nullptr
Erase
If a node has only one child, we can simply promote the sub-
tree associated with the child
• Consider removing 8 which has one left child
Erase
The node 8 is deleted and the left_tree of 11
is updated to point to 3
Erase
There is no difference in promoting a single node or a sub-tree
• To remove 39, it has a single child 11
Erase
The node containing 39 is deleted and left_node of 42 is
updated to point to 11
• Notice that order is still maintained
Erase
Consider erasing the node containing 99
Erase
The node is deleted and the left sub-tree is promoted:
• The member variable right_tree of 70 is set to point to 92
• Again, the order of the tree is maintained
Erase
Finally, we will consider the problem of erasing a full node, e.g.,
42
We will perform two operations:
• Replace 42 with the minimum object in the right sub-tree
• Erase that object from the right sub-tree
Erase
In this case, we replace 42 with 47
• We temporarily have two copies of 47 in the tree
Erase
We now recursively erase 47 from the right sub-tree
• We note that 47 is a leaf node in the right sub-tree
Erase
Leaf nodes are simply removed and left_tree of
51 is set to nullptr
• Notice that the tree is still sorted:
47 was the least object in the right sub-tree
Erase
Suppose we want to erase the root 47 again:
• We must copy the minimum of the right sub-tree
• We could promote the maximum object in the left sub-tree and achieve
similar results
Erase
We copy 51 from the right sub-tree
Erase
We must proceed by delete 51 from the right sub-tree
Erase
In this case, the node storing 51 has just a single child
Erase
We delete the node containing 51 and assign the member
variable left_tree of 70 to point to 59
Erase
Note that after seven removals, the remaining tree is still
correctly sorted
Erase
In the two examples of removing a full node, we promoted:
• A node with no children
• A node with right child
Is it possible, in removing a full node, to promote a child with
two children?
Erase
Recall that we promoted the minimum value in the right sub-tree
• If that node had a left sub-tree, that sub-tree would contain a smaller
value
Erase
In order to properly remove a node, we will have to change the
member variable pointing to the node
• To do this, we will pass that member variable by reference

Additionally: We will return 1 if the object is removed and 1 if

the object was not found
Erase
Blackboard example:
• In the binary search tree generated previously:
• Erase 47
• Erase 21
• Erase 45
• Erase 31
• Erase 36
Binary Search Tree
We have defined binary search nodes
• Similar to the Single_node in Project 1

We must now introduce a container which stores the root

• A Binary_search_tree class

Most operations will be simply passed to the root node

Implementation
template <typename Type>
class Binary_search_tree {
private:
Binary_search_node<Type> *root_node;
Binary_search_node<Type> *root() const;
public:
Binary_search_tree();
~Binary_search_tree();

bool empty() const;

int size() const;
int height() const;
Type front() const;
Type back() const;
int count( Type const &obj ) const;

void clear();
bool insert( Type const &obj );
bool erase( Type const &obj );
Constructor, Destructor, and Clear
template <typename Type>
Binary_search_tree<Type>::Binary_search_tree():
root_node( nullptr ) {
// does nothing
}

template <typename Type>

Binary_search_tree<Type>::~Binary_search_tree() {
clear();
}

template <typename Type>

void Binary_search_tree<Type>::clear() {
root()->clear( root_node );
}
Constructor, Destructor, and Clear
template <typename Type>
Binary_search_tree<Type> *Binary_search_tree<Type>::root() const {
return tree_root;
}

template <typename Type>

bool Binary_search_tree<Type>::empty() const {
return root()->empty();
}

template <typename Type>

int Binary_search_tree<Type>::size() const {
return root()->size();
}
Empty, Size, Height and Count
template <typename Type>
int Binary_search_tree<Type>::height() const {
return root()->height();
}

template <typename Type>

bool Binary_search_tree<Type>::find( Type const &obj ) const {
return root()->find( obj );
}
Front and Back
// If root() is nullptr, 'front' will throw an underflow exception
template <typename Type>
Type Binary_search_tree<Type>::front() const {
return root()->front();
}

// If root() is nullptr, 'back' will throw an underflow exception

template <typename Type>
Type Binary_search_tree<Type>::back() const {
return root()->back();
}
Insert and Erase
template <typename Type>
bool Binary_search_tree<Type>::insert( Type const &obj ) {
return root()->insert( obj, root_node );
}

template <typename Type>

bool Binary_search_tree<Type>::erase( Type const &obj ) {
return root()->erase( obj, root_node );
}
Other Relation-based Operations
We will quickly consider two other relation-based queries that
are very quick to calculate with an array of sorted objects:
• Finding the previous and next values, and
• Finding the kth value
Previous and Next Objects
All the operations up to now have been operations which work on any
container: count, insert, etc.
• If these are the only relevant operations, use a hash table

Operations specific to linearly ordered data include:

• Find the next larger and previous smaller objects of a given object which may or
may not be in the container
• Find the kth value of the container
• Iterate through those objects that fall on an interval [a, b]

We will focus on finding the next largest object

• The others will follow
Previous and Next Objects
To find the next largest object:
• If the node has a right sub-tree, the minimum object in that sub-tree is
the next-largest object
Previous and Next Objects
If, however, there is no right sub-tree:
• It is the next largest object (if any) that exists in the path from the root
to the node
Previous and Next Objects
More generally: what is the next largest value of an arbitrary object?
• This can be found with a single search from the root node to one of the leaves—an O(h)
operation
• This function returns the object if it did not find something greater than it
template <typename Type>
Type Binary_search_node<Type>::next( Type const &obj ) const {
if ( empty() ) {
return obj;
} else if ( value() == obj ) {
return ( right()->empty() ) ? obj : right()->front();
} else if ( value() > obj ) {
Type tmp = left()->next( obj );

return ( tmp == obj ) ? value() : tmp;

} else {
return right()->next( obj );
}
}
Finding the kth Object
Another operation on sorted lists may be finding the kth largest object
• Recall that k goes from 0 to n – 1
• If the left-sub-tree has ℓ = k values, return the current node,
• If the left sub-tree has ℓ < k values, return the kth value of the left sub-tree,
• Otherwise, the left sub-tree has ℓ > k values, so return the (k – ℓ – 1)th value
of the right sub-tree
18
7 10
1 5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 141516 17
Finding the kth Object
template <typename Type>
Type Binary_search_tree<Type>::at( int k ) const {
return ( k < 0 || k >= size() ) ? Type() : root()->at( k );
// Need to go from 0, ..., n - 1
}

template <typename Type>

Type Binary_search_node<Type>::at( int k ) const {
if ( left()->size() == k ) {
return value();
} else if ( left()->size() > k ) {
return left()->at( k );
} else {
return right()->at( k - left()->size() – 1 );
}
}
Finding the kth Object
This requires that size() returns in Q(1) time
• We must have a member variable
int tree_size;
which stores the number of descendants of this node
• This requires Q(n) additional memory
template <typename Type>
bool Binary_search_tree<Type>::size() const {
return root()->size();
}
• We can implement this in the Binary_node class, if we want
• The constructor will set the size to 1
Finding the kth Object
We must now update insert(…) and erase(…) to
update it
template <typename Type>
bool Binary_search_node<Type>::insert( Type const &obj,
Binary_search_node *&ptr_to_this ) {
if ( empty() ) {
ptr_to_this = new Binary_search_node<Type>( obj );
return true;
} else if ( obj < value() ) {
return left()->insert( obj, left_tree ) ? ++tree_size : false;
} else if ( obj > value() ) {
return right()->insert( obj, right_tree ) ? ++tree_size : false;
} else {
return false;
}
}
Clever trick: in C and C++, any non-zero value is interpreted as true
Run Time: O(h)
Almost all of the relevant operations on a binary search tree are O(h)
• If the tree is close to a linked list, the run times is O(n)
• Insert 1, 2, 3, 4, 5, 6, 7, ..., n into a empty binary search tree
• The best we can do is if the tree is perfect: O(ln(n))
• Our goal will be to find tree structures where we can maintain a height of Q(ln(n))

We will look at
• AVL trees
• B+ trees
both of which ensure that the height remains Q(ln(n))

Others exist, too

Summary
In this topic, we covered binary search trees
• Described Abstract Sorted Lists
• Problems using arrays and linked lists
• Definition a binary search tree
• Looked at the implementation of:
• Empty, size, height, count
• Front, back, insert, erase
• Previous smaller and next larger objects
Usage Notes
• These slides are made publicly available on the web for anyone to use
• If you choose to use them, or a part thereof, for a course at another
institution, I ask only three things:
• that you inform me that you are using the slides,
• that you acknowledge my work, and
• that you alert me of any mistakes which I made or changes which you make,
and allow me the option of incorporating such changes (with an
acknowledgment) in my set of slides

Sincerely,
Douglas Wilhelm Harder, MMath
[email protected]