MH1403 Algorithms and Computing

Lecture 6 Binary Search and

Binary Search Tree

1
Outline

• Binary search
• Binary search tree

2
Binary Search

3
Binary Search

• Suppose that there is a sorted list containing

the names of students and their grades.
The list is sorted according to the names.
Now given the name of a student, find the
grade of the student
• You can find the student name in the list in a
sequential way (by checking the names in the list
one by one), but it is inefficient:
average running time O(n)
• We can use binary search to speed up the search

4
Binary Search

• Binary Search (also called half-interval search)

• Keep reducing the search space by about half
• Complexity: Assume that there are n elements in the
sorted list (or array),
• Worst case: accessing at most floor(log2n) + 1 elements
• very efficient
suppose that there are around 1 million elements,
complexity: access the array for about 21 times

worst case running time: O(log(n))

5
Binary Search
Example: sorted array of integers.
We want to find the element with value 7

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

6
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint

7
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Is 7 = the element at the midpoint? NO.

8
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Is 7 < the element at the midpoint key? YES.

9
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Search for the target in the area before midpoint.

10
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint

11
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target = the element at the midpoint? NO.

12
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target < the element at the midpoint? NO.

13
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target > the element at the midpoint? YES.

14
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Search for the target in the area after midpoint.

15
Binary Search

Example: sorted array of integers. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint.

Is target = the element at the midpoint? YES.
16
Binary Search Algorithm (Iterative)
• Suppose that there is a sorted array A, we want to find the
index of the element with value x in array A. Let the
search interval be [left, right]
def binarySearch(A, x):
left = 0, right = len(A)-1
while left <= right:
mid = (left+right) // 2
if x == A[mid], return mid # find the index of x
if x < A[mid], right = mid-1
if x > A[mid], left = mid+1

return None # cannot find x

17
Binary Search Algorithm (Recursive)
• Alternatively, we can implement the binary search with the
recursive approach:
def _binary_search(A, left, right, x):
if left > right, return None # base condition 1: cannot find x

mid = (right + left) // 2

if x == A[mid], return mid # base condition 2: find x
if x < A[mid], return _binary_search(A, left, mid-1, x)
if x > A[mid], return _binary_search(A, mid+1, right, x)

def binary_search(A, x):

_binary_search(A, 0, len(A)-1, x)

18
Implementation of binary search
• We can use array (or Python list) to store the
sorted data, then apply the binary search
• Efficient for static data
• Inefficient when data need to be inserted or deleted
frequently
• Linked list
• Binary search cannot be implemented efficiently with
linked list. For example, we cannot reach the middle
node quickly.
• When the data need to be inserted or deleted
frequently, binary search can be implemented
efficiently using the binary search tree
• Fast to search
• Fast to insert or delete data
19
Binary Search Tree (BST)

20
Binary Search Tree (BST)

• We mentioned that if we need fast search, fast

insertion and deletion, we can use the data
structure binary search tree (BST) to store the
data
• We will learn the following topics on binary
search tree
• Definition and examples
• Operations on binary search tree

21
Binary Search Tree (BST)
• Binary Search Tree:
In a binary tree, the value stored at any node is greater
than all the values in its left subtree and less than the
values in its right subtree, then the tree is a binary search
tree.

22
Examples

More example of binary search trees:

23
Examples
• Unfortunately, it is a binary search tree:
• This is equivalent to a linked list.

24
Examples
All these binary search trees store the same data

25
Duplicate values

We will assume that in any binary tree, we are not

storing duplicate values unless otherwise stated
• In reality, it is seldom the case where duplicate values in a
container must be stored as separate entities

26
Nodes of binary search tree

• In Python, we implement the node using the following

Python class
• For example, the data in a node may be the student name
together with the student’s exam scores.

class Node:
def __init__(self, data):
self.data = data
self.leftChild = None
self.rightChild = None

27
Operations on Binary Search Tree

28
Operations on Binary Search Tree

• We will learn the following operations on binary

search tree
• Find
• Find a given value (search)
• Find the smallest and largest values
• Find the next larger node of a given node
• Find the next smaller node of a given node
• Find the kth smallest value
• Traverse
• Insert
• Delete

29
Find a Given Value (search)

30
Find a given value (Search)

• Finding a given value in binary search tree is

fast. Running time is O(h) (h is height)
• For example, to find a node containing the value 81
• Check root 42. Since 81 > 42, go to the right child.
• Check 70. Since 81 > 70, go to the right child
• Check 99. Since 81 < 99, go to the left child
• Check 92. Since 81 < 92, go to the left child
• Check 81. Since 81 == 81, done.

31
Find a given value (Search)
• Another example, find a node containing the value
36
• Check root 42. Since 36 < 42, go to the left child.
• Check 39. Since 36 < 39, go to the left child
• Check 11. Since 36 > 11, go to the right child
• Check 29. Since 36 > 29, go to the right child
• Check 34. Since 36 > 34, go to the right child
• The node with value 34 does not have right child. So the
value 36 is not in this binary search tree.

32
Find the Smallest Node and the Largest Node

33
Find the Smallest Value

• From the root, go to the left child, then go the left child,
… , until reaching a node with no left child.
The running time is O(h)
• For example,
• From root 42, go to the left child with value 39, then go to the left
child with value 11, then go to the left child with value 8, then go to
the left child with value 3. The node with value 3 has no left child, so
the smallest value is 3.

34
Find the Largest Value

• From the root, go to the right child, then go the right

child, … , until reaching a node with no right child.
The running time is O(h)
• For example, from root 42, go to the right child with value 70, then go
to the right child with value 99. The node with value 99 has no right
child, so the largest value is 99.

35
Find the Next Larger Node
and the Next Smaller Node

36
Find the Next Larger Node

• To find the next larger node:

• If the node has a right subtree, its next larger node is the
smallest node in its right subtree (we just learned how to find
the smallest node of a binary search tree)
• For example, for node 11, it has a right subtree, the smallest
node in that subtree is 16, so the next larger node of the node
11 is the node 16 (similarly, 19  24, 29  34, 51  52, …)

37
Find the Next Larger Node

• To find the next larger node (cont.):

• If the node has no right sub-tree, its next larger node (if any)
exists on the path from the root to the node
• For example, for node 24, it has no right subtree, from the root
to 24, the next larger node is 29.
For node 34, it has no right subtree, from the root to 24, the
next larger node is 51.

38
Find the Next Smaller Node

• To find the next smaller node:

• If the node has a left subtree, its next smaller node is the largest
node in its left subtree (we just learned how to find the largest
node of a binary search tree)
• For example, for node with value 70, it has a left subtree, the
smallest node in that subtree is 68, so the next smaller node of
the node 70 is the node 68

39
Find the Next Smaller Node

• To find the next smaller node (cont.):

• If the node has no left subtree, its next smaller node (if any)
exists on the path from the root to the node
• For example, for node 52, it has no left subtree, from the root to
node 52, the next smaller node is 51.
For node 81, it has no left subtree, from the root to node 81, the
next smaller node is 70.

40
Find the kth Smallest Node of BST

41
Find the kth Smallest Node

• Suppose that we have a Python list (or array)

sorted in ascending order, accessing the kth
element is straightforward
• Let the Python list be A. A[k] gives the kth element.
(n elements in total, k is between 0 and n-1)
• To find the kth smallest node in BST, it is not as
fast as finding the kth element in a sorted
Python list
• We will treat the smallest node as the 0th smallest
node, the largest node as the n-1th smallest node

42
Find the kth Smallest Node

• A simple way to find the kth smallest node in BST

is to start from the smallest node, then go to the
next larger node, then go to the next larger node,
…, until reach the kth smallest node
• This method is efficient only for small k
• If (n-k) is small, similar method is also efficient: find
the largest node, then the next smaller node, ….
• The above method require O(n) operations to find
the kth smallest node  inefficient

43
Find the kth Smallest Node
• To find the kth smallest node in BST efficiently, in
each node v, we will store a value nv which is the
number of nodes in the subtree rooted at node v

44
Find the kth Smallest Node

• The following recursive algorithm is used to find

the kth smallest node in BST efficiently: O(h)

• Denote the nv value of the left child of root as t (i.e., t

is the size of the left subtree of root)
• If t = k, the target is the root. Return the root node
• If t > k, the target is in its left subtree. Apply the
algorithm to find the kth smallest node in left subtree
• If t < k, the target is in its right subtree. Apply the
algorithm to find the (k – t – 1)th smallest node in its
right subtree

45
Find the kth Smallest Node
• Example: find the 7th smallest node
• For root 51, the nv value of its left child is 7, so the
7th smallest node is the node 51.

46
Find the kth Smallest Node
• Example: find the 0th smallest node
• For the root , the counter value of its left child is 7,
so apply the algorithm to find the 0th smallest node of
its left subtree rooted at node 11

47
Find the kth Smallest Node
• Example: find the 0th smallest node (cont.)
• For the subtree rooted at node 11, the nv value of its
left child is 1, so apply the algorithm to find the 0th
smallest node of its left subtree rooted at node 3

48
Find the kth Smallest Node
• Example: find the 0th smallest node (cont.)
• For the root of the subtree rooted at node 3, the nv
value of its left child (empty) is 0, so return the node
3 (i.e., the node 3 is the 0th smallest node)

49
Find the kth Smallest Node
• Example: find the 5th smallest node
• For the root, the nv value of its left child is 7, so
apply the algorithm to find the 5th smallest node of its
left subtree rooted at node 11

50
Find the kth Smallest Node
• Example: find the 5th smallest node (cont.)
• For the subtree rooted at node 11, the nv value of its
left child is 1, so apply the algorithm to find the (5-1-
1)th smallest node of its right subtree rooted at node 29

51
Find the kth Smallest Node
• Example: find the 5th smallest node (cont.)
• For the subtree rooted at node 29, the nv value of its
left child is 3, so return the node 29 (i.e., the 5th
smallest node is node 29)

52
Traversals of BST

53
Traverse

• The four traversals of binary tree can be applied

to the binary search tree
• The inorder traversal of a binary search tree visits
the nodes from the smallest value to the largest
value in order.

54
Traverse

55
Insertion Operation on BST

56
Insert

An insertion will be performed at an empty leaf node:

• All the empty nodes are indicated as small blue circles in the
following tree.

57
Insert

• The values which may be inserted at any empty

leaf node depends on the surrounding nodes
• For example, this node may hold 48, 49, or 50

58
Insert

An insertion at this location may be 35, 36, 37, or 38

59
Insert

This empty node may hold values from 71 to 74

60
Insert

• Inserting a value is similar to finding a value in

a tree, we will step through the tree
• If we find the value already in the tree, we will return
• The value is already in the binary search tree (no duplicates)
• Otherwise, we will arrive at an empty node, the value
will be inserted into that location
• The run time is O(h) (h is the height of the tree)

61
• Example: To insert the value 52, we step
through the tree until we reach an empty node
• Start at root. 52 > 42, go to right child
• At the node with value 70: 52 < 70, go to left child
• At the node with value 51: 52 > 51, go to right child
• At the node with value 59: 52 < 59, go to left child
• At the node with value 54: 52 < 54, go to left child
• The left child with value 54 is empty, we should insert
the value at this location.

62
• Example (cont.): after inserting the value 52,
the tree becomes

63
Insert
• Example: In the given order, insert the following values into an
empty binary search tree:

64
Insert

• Suppose that each node v contains the value nv

(the number of nodes in the subtree rooted at node
v).
• When we insert a new node, we need to increase
the nv values of all the nodes on the path from the
root to the new node by 1 (including the new node)

65
Insert
• Exercise: In the given order, insert the following
values into an empty binary search tree, and
update the nv values when inserting a node

66
Python code of Insertion operation
class BSTNode:
def __init__(self, data):
self.data = data
self.leftChild = None
self.rightChild = None

class BST:
def __init__(self):
self.root = None

def insert(self,data):
# create a new node
newNode = BSTNode(data)

# inser the new node into the binary search tree

# if the tree is empty, the new node is set as root.
if self.root == None:
self.root=newNode
return
# the insert method is continued in the next slide 67
curr=self.root
while curr:
# if data exists in the tree already, do nothing, return
if newNode.data == curr.data:
return
# if data is larger than the current node data,
# go to its right child, if its right child is empty, insert node.
elif newNode.data > curr.data:
if curr.rightChild != None:
curr = curr.rightChild
else:
curr.rightChild = newNode
return
# if data is smaller than the current node data,
# go to left child, if left child is empty, insert node.
else:
if curr.leftChild != None:
curr = curr.leftChild
else:
curr.leftChild = newNode
return

68
# Driver code
if __name__=='__main__':
mybst = BST()
mylist = [5, 15, 75, 81, 77, 30]
# insert the data in mylist into an empty binary search tree
for x in mylist:
mybst.insert(x)

69
Deletion Operation on BST

70
Delete
• There are three possible scenarios to delete a
node from a binary search tree:
• The node is a leaf node
• The node has exactly one child, or
• The node has two children

71
Delete a leaf node
• A leaf node can simply be removed
• The appropriate instant variable of the parent is set to NULL
• Example: removing node containing value 75
We find the node 75, set the instance variable leftChild of node
81 to NULL (None in Python)

72
Delete a leaf node

After removing the node containing value 75, the binary

search tree now becomes:

73
Delete a node with one child

If a node has only one child, we can simply promote the

sub-tree associated with the child
• Consider removing 8 which has one left child

74
Delete a node with one child

• The node 8 is deleted and the leftChild of node 11

is updated to point to node 3

75
Delete a node with one child

There is no difference in promoting a single node or a

subtree
• To remove 39, it has a single child 11

76
Delete a node with one child

The node containing 39 is deleted and leftChild of node 42 is

updated to point to 11
• Notice that order is still maintained

77
Delete a node with one child

Consider erasing the node containing 99

78
Delete a node with one child

The node is deleted and the left sub-tree is promoted:

• The instance variable rightChild of node 70 is set to point to 92
• Again, the order of the tree is maintained

79
Delete a node with two children

• Finally, we will consider the problem of deleting a node

with two children.
• For example, removing node containing value 42. We
will perform two operations:
• Replace 42 with the smallest node in its right subtree
• Delete the smallest node from its right subtree

80
Delete a node with two children

In this case, we replace 42 with 47

• We temporarily have two copies of 47 in the tree

81
Delete a node with two children

We now delete 47 from the right subtree

82
Delete a node with two children
Node 47 is simply removed by setting leftChild of 51 to
NULL
• Note that the tree is still sorted

83
Delete a node with two children

Suppose we want to delete the root 47 again:

• We will copy the smallest node in its right subtree

84
Delete a node with two children

We copy 51 from its right subtree

85
Delete a node with two children

We must proceed by deleting 51 from the right sub-tree

86
Delete a node with two children

In this case, the node storing 51 has just a single child

87
Delete a node with two children

We delete the node containing 51 and assign the member

variable leftChild of node 70 to point to 59

88
Delete a node with two children

• Note that after the removals, the remaining tree

is still correctly sorted

89
 Delete a node with two children
• In the two examples of removing a node with two
children, we promoted:
• A node with no children
• A node with right child

• Is it possible to promote a child with two children?

90
Delete a node with two children

• In the previous examples of erasing a node with two

children, we replace it with the smallest value in its
right subtree, then remove the smallest value in the
right subtree
• Alternatively, we can replace it with the largest value
in its left subtree, then remove the largest value in the
left subtree.

91
Delete
• Suppose that each node v contains the value nv
(the number of nodes in the subtree rooted at node
v).
• When we erase a node, there are two different
scenarios to update the nv values in the tree:
• The node has less than two children
• The node has two children
• These two scenarios are illustrated in the next two
slides.

92
Delete

Example 1: After removing node 8 from the BST, we

need to decrease the nv values of all the ancestors of
node 8: node 11, node 39 and node 42

93
Delete

Example 2: When removing node 47 from the following

BST, node 47 will be replaced by node 51.
• copy the nv value of node 47 as the nv of the new node 51
• decrease the nv values of all the ancestors of the original
node 51: node 70, the new node 51

94
Running Time of Operations on BST

95
Running Time: O(h)

• Most of the operations (search, insert, delete)

on a binary search tree are O(h) running time,
where h is the height of the tree
• The worst case: If the tree is close to a linked list, the
running times is O(n)
• For example, insert 1, 2, 3, 4, 5, 6, 7, ..., n into an empty
binary search tree, we obtain a linked list
• For efficient operations on a BST, the height of the
BST should be as small as possible

96
Running Time: O(h)

• To minimize the height of the binary search

tree, we can adjust the nodes in the binary
search tree
• For example, the AVL tree ensures that the height
remains O(log(n))
• We will learn AVL tree in the next lecture

97
Summary

• Binary search is very efficient for searching the

sorted data
• If the data are static, the sorted data can be
stored in array (or Python list), then perform
binary search
• If the data are inserted and deleted frequently,
the data can be stored in BST, then perform
search
• If the height of BST is optimally small, the search
on BST is as efficient as binary search

98
Programming Practice on BST

• At the following website, in the “category”, choose BST for

the exercises:
https://www.techiedelight.com/data-structures-and-algorithms-
problems/
• Exercises on BST:
https://www.w3resource.com/python-exercises/data-structures-and-
algorithms/python-binary-search-tree-index.php

99