CS 260 - Data Structures

Minimum Spanning
Trees
Yusuf Osmanlioglu
Department of Computer Science
Drexel University
Introduction to Minimum Spanning Tree
(MST)
● Drexel has buildings in various locations at Philadelphia.
● They want all the buildings to have direct fiber connections.
● They can lease fiber from Verizon.
● What is the lowest lease payment that connects all the buildings?
○ Nodes are buildings
○ Undirected weighted edges represent the distance between buildings
○ Given this graph, calculate a Minimum Spanning Tree that connects buildings with minimum
amount of cables!

2
Minimum Spanning Trees
● Fiber lease cost

$100
Caneris Hall Nesbitt

$25 $75 $75

$50
Rush Pearlstein
$75
$30 $15
$25
3675 Market LeBow Hall

$50

Disque

$35
$25
Korman

3
Minimum Spanning Trees
● Best Cost: $235 (pay for red edges) ● Tree:
○ A graph that contains no
cycles.
$100
Caneris Hall Nesbitt
● Spanning:
$25 $75 $75 ○ Reaching all nodes in a
$50
Rush Pearlstein set or subset.
$75
$30 $15 ● Minimum:
$25
3675 Market LeBow Hall ○ The smallest of all
available options.
$50
● MST:
Disque
○ A tree with minimum
$35
$25
total weight that spans
Korman
all nodes.
4
Minimum Spanning Tree: formal definition
● Let G=(V,E) be a graph on n vertices and m edges, and a weight function w on
edges in E.
● A sub-graph T of G through all vertices which avoids any cycle is a spanning tree.
● The weight of T is defined as sum of the weights of all edges in T:

8 7
4 b c d 9
2
11 4 14 e
a i
7 6
8 10
h g f
1 2 5
MST Calculation: A greedy approach
● Each step of a greedy algorithm must make one of a several choices
● Greedy strategy:
○ make the choice that is the best at the moment
● Does not always guarantee finding globally optimum solutions.
● However, our greedy heuristics for MST will yield optimal solutions!

6
Generic MST Algorithm
● The greedy algorithm tries to solve the MST problem by making locally optimal
choices:
○ sort the edges by weight
○ for each edge on sorted list, include that in MST if it does not form a cycle with the edges
already taken; otherwise discard it
● When does the algorithm halt?
○ as soon as |V|-1 edges have been kept
● Runtime:
○ Step 1 takes O(|E| log |E|)=O(|E| log |V|).
○ Step 2 can be done in O(|V| log |V|) time, later we will present a linear time implementation for
this step.

7
Set Operations
● We will need set operations for the MST algorithm!
● A Set is a collection of distinct elements (no duplicates allowed)
● In the proof of running time for our MST algorithm, we will use the
following set operations:
○ Make-Set(v): creates a set containing element v, {v}.
○ Find-Set(u): returns the set to which v belongs to.
○ Union(u,v): creates a set which is the union of the two sets, one containing v
and one containing u.
● As an example, we can use a pointer to implement a set system:
○ Make-Set(v) will create a single node containing element v.
○ Find-set(u) will return the name of the first element in the set that contains u,
○ Union(u,v) will concatenate the sets containing u and v.

8
Set Operations with pointers
● make-set(w) ● find-set(w) O(1)
O(1)
head head
S1 w \ S1 x y w \
tail tail

● union(S1,S2) O(?)

head head
S1 x y w \ S1 x y w
tail tail

head
S2 a b \ a b \
tail
9
Running time of Union
● How fast can we compute the union?
● Let us ask a different question.
○ Let N={1,…,n} be a set of n integers, and let P={(u,v)| u and v in N} be a subset of pairs from
nxn.
for u=1 to n
Make-Set(u);
for every pair (u,v) in P
if Find-Set(u) ≠ Find-Set(v)
Union(u,v);

● Question: How many times does the pointer for an element get redirected?

10
Union Operation
● Each merge of two sets might 1 2 … n-2 n-1 n

take linear number of pointer

changes. 1 2 … n-2 n-1 n

● If we update pointers one at a

time, in total we’ll have: 1 2 … n-2 n-1 n

⁞ n-1 steps

1 2 … n-2 n-1 n

O(n2) 0 1 n-2 n-1

11
Union Operation
● Can we make any better? 1 2 … n-2 n-1 n

○ Join in a binary fashion

● Let us keep a number 1 2 … n-2 n-1 n
associated with each set in its
root, Rank(u), which tell how
⁞ log n steps
many elements a set has.
● When merging two lists,
always change the pointers in
the list with smaller rank. 1 2 … n-2 n-1 n

O(nlogn) 0 1 logn-1 logn

12
Kruskal’s Algorithm: example
8 7 8 7 8 7
4 b c d 9 4 b c d 9 4 b c d 9
2 2 2
11 4 14 e 11 4 14 e 11 4 14 e
a i a i a i
7 6 7 6 7 6
8 10 8 10 8 10
h 1
g 2
f h 1
g 2
f h 1
g 2
f

8 7 8 7 8 7
4 b c d 9 4 b c d 9 4 b c d 9
2 2 2
11 4 14 e 11 4 14 e 11 4 14 e
a i a i a i
7 6 7 6 7 6
8 10 8 10 8 10
h 1
g 2
f h 1
g 2
f h 1
g 2
f

8 7 8 7 8 7
4 b c d 9 4 b c d 9 4 b c d 9
2 2 2
11 11 11 4 14 e
a i 4 14 e a i 4 14 e a i
7 7 7 6
6 6 8 10
8
h g f 10 8
h g f 10 h 1
g 2
f 13
1 2 1 2
Kruskal’s Algorithm: example
8 7 8 7 8 7
4 b c d 9 4 b c d 9 4 b c d 9
2 2 2
11 4 14 e 11 4 14 e 11 4
a i a i a i 14 e
7 6 7 6 7 6
8 10 8 10 8 10
h 1
g 2
f h 1
g 2
f h g f
1 2

● Alternatively, the following choice of edges would also be a valid MST

8 7 8 7
8 7 4 b c d 9 4 b c d 9
4 b c d 9 2 2
2 11 4 11
11 a i 14 e a i 4 14 e
4 14 e
a i 7 6 7 6
7 6 8 10 8 10
8 10 h 1
g 2
f h 1
g 2
f
h 1
g 2
f
14
Kruskal’s MST algorithm
● It is directly based on Generic MST.
● At each iteration, it finds a light edge, which is also safe, and adds it to an ever growing
set, A, which will eventually become the MST.
● Throughout iterations, the structure generated by algorithm is a forest.
function kruskal(Graph G = (E , V ))
Sort Edges
T = Empty Set
for each node n in V do
make-set(n)
end for
for each edge (x,y) in order do
if find-Set(x) ≠ find-Set(y) then
T = T ∪ {(x,y)}
union(Find-Set(x),Find-Set(y))
end if
end for
return T 15
end function
Kruskal’s Algorithm: Runtime
function kruskal(Graph G = (E , V ))
Sort Edges O(ElogE)
T = Empty Set O(1)
for each node n in V do
make-set(n) O(V)
end for
for each edge (x,y) in order do
if find-Set(x) ≠ find-Set(y) then
T = T ∪ {(x,y)} O(VlogV)
union(Find-Set(x),Find-Set(y))
end if
end for
return T
end function

● Sorting the edges is the hardest part of ● When to use Kruskal:

Kruskal’s algorithm. ○ When you can sort quickly.
● There are at most V2 edges, sorting which takes ○ Your edges are given sorted or partially sorted
O(V2 log V2 ) (use merge sort) ○ You have extra information to sort in linear time.
● Remember: log V2 = 2log V
● Total Run Time: O(V2 log V ) = O(E log V ) 16
Prim’s Algorithm: Example
8 7 8 7 8 7
4 4 ∞ ∞ 9 4 4 8 ∞ 9 4 4 8 7 9
2 2 2
11 4 14 ∞ 11 4 14 ∞ 11 4 14 ∞
0 ∞ 0 ∞ 0 2
7 6 7 6 7 6
8 10 8 10 8 10
8 1
∞ 2
∞ 8 1
∞ 2
∞ 8 1
∞ 2
4

8 7 8 7 8 7
4 4 8 7 9 4 4 8 7 9 4 4 8 7 9
2 2 2
11 4 14 10 11 4 14 10 11 4 14 ∞
0 2 0 2 0 2
7 6 7 6 7 6
8 10 8 10 8 10
1 1
2 2
4 7 1
2 2
4 7 1
6 2
4

8 7 8 7 8 7
4 4 8 7 9 4 4 8 7 9 4 4 8 7 9
2 2 2
11 4 14 10 11 4 14 9 11 4 14 9
0 2 0 i 0 2
7 6 7 6 7 6
8 10 8 10 8 10
1 1
2 2
4 h 1
g 2
f 1 1
2 2
4 17
Prim’s Algorithm
● At each step, the set A is one connected component.
● Start at an arbitrary vertex r (root of the tree).
● At each step, add a new vertex which is connected to A through a minimum
weight edge.
● The growth starts at r and continues until all vertices are covered, each vertex u
has a parent p(u), which represents its parent in the tree.
● Also, each vertex u has a key(u), which represents the cost of adding u to A at
each point of algorithm.

18
Prim’s Algorithm
function prim(Graph G = (V,E),w,r)
for each u in V do
u.key = ∞ O(V) ● To implement the
u.p = NIL priority queue Q, we can
end for
r.key = 0 O(1)
use a binary heap.
Q = V O(1) ● v.key = w(u,v) can be
while Q ≠ ∅ do implemented with
u = EXTRACT-MIN(Q) O(VlogV)
for each v in G.Adj[u] do decrease key which
if v in Q and w(u,v) < v.key then takes O(log V)-time.
v.p = u O(ElogV)
v.key = w(u,v) ● Since there are at most
end if |E| elements in all Adj[]
end for
return r list for all elements in Q,
the algorithm take
O(Vlog V+ E log V)
● Total runtime
○ O(E log V)
19
Prim vs Kruskal
● Kruskal’s Algorithm sorts the edges before starting
● Prim’s algorithm keeps a heap.
● Advantages:
○ Prim: You don’t need to know about all the edges before you start.
○ Kruskal: Better if your edges are already in a sorted or easy to sort format
● Runtimes are the same: O(ElogV)

20