Analysis of Algorithms

Minimum Spanning Trees (MST)

Dr. Manisha Chahal

 Minimum Spanning Trees
• Spanning Tree
– A tree (i.e., connected, acyclic graph) which contains
all the vertices of the graph
• Minimum Spanning Tree
– Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
g g f
• Spanning forest 1 2

– If a graph is not connected, then there is a spanning

tree for each connected component of the graph
2
 Applications of MST
– Find the least expensive way to connect a set of
cities, terminals, computers, etc.

3
 Example
Problem b
8
c
7
d
4 9
• A town has a set of houses 2
and a set of roads a 11 i 4 14 e
7 6
• A road connects 2 and only 8 10
h g f
2 houses 1 2

• A road connecting houses u and v has a repair

cost w(u, v)
Goal: Repair enough (and no more) roads such
that:
1. Everyone stays connected
i.e., can reach every house from all other houses
2. Total repair cost is minimum

4
 Minimum Spanning Trees
• A connected, undirected graph:
– Vertices = houses, Edges = roads
• A weight w(u, v) on each edge (u, v)  E

Find T  E such that: 8 7

b c d
4 9
1. T connects all vertices 2
a 11 i 4 14 e
2. w(T) = Σ(u,v)T w(u, v) is
7 6
8 10
minimized h g f
1 2

5
Properties of Minimum Spanning Trees

• Minimum spanning tree is not unique

• MST has no cycles – see why:

– We can take out an edge of a cycle, and still have
the vertices connected while reducing the cost

• # of edges in a MST:
– |V| - 1

6
 Growing a MST – Generic Approach

• Grow a set A of edges (initially empty)

• Incrementally add edges to A such that
they would belong
8 7
to a MST b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
– An edge (u, v) is safe for A if and only if A 
h g f
Idea: add
{(u, v)} only
is also “safe”
a subset edges
of some MST 1 2

7
 Generic MST algorithm

1. A ← 
2. while A is not a spanning tree
3. do find an edge (u, v) that is safe for A
4. A ← A  {(u, v)} b
8
c
7
d
4 9
2
5. return A a 11 i 4 14 e
7 6
8 10
h g 2
f
1
• How do we find safe edges?

8
 Prim’s Algorithm
• The edges in set A always form a single tree

• Starts from an arbitrary “root”: VA = {a}

• At each step: 8 7
b c d
4 9
– Find a light edge crossing (VA, V - VA) 2
a 11 i 4 14 e
– Add this edge to A
7 6
8 10
– Repeat until the tree spans all vertices
h g 2
f
1

9
 How to Find Light Edges Quickly?
Use a priority queue Q: 8 7
b c d
• Contains vertices not yet 4 9
2
11
included in the tree, i.e., (V – VA) a i 4 14 e
7 6
8 10
– VA = {a}, Q = {b, c, d, e, f, g, h, i}
h g 2
f
1
• We associate a key with each vertex v:
key[v] = minimum weight of any edge (u, v)
connecting v to VA

Key[a]=min(w1,w2) w1

w2
a

10
 How to Find Light Edges Quickly?
(cont.)
• After adding a new node to VA we update the weights of all
the nodes adjacent to it
e.g., after adding a to the tree, k[b]=4 and k[h]=8

• Key of v is  if v is not adjacent to any vertices in VA

8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g 2
f
1

11
 Example
  
8
b c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h, i}
a 11 i 14 e
4 VA = 
7 6
8 10
Extract-MIN(Q)  a
h g 2
f
1
  

4
  
8 7 key [b] = 4  [b] = a
b c d
4 9 key [h] = 8  [h] = a
 2 
a 11 i 4 14 e
4 8
7 6
8 10 Q = {b, c, d, e, f, g, h, i} VA = {a}
h g f
1 2 Extract-MIN(Q)  b
8  

12
 Example
4 8
  key [c] = 8  [c] = b
8 7
b c d key [h] = 8  [h] = a - unchanged
4 9
 2  8 8
a 11 i 14 e
4 Q = {c, d, e, f, g, h, i} VA = {a, b}
7 6
8 10 Extract-MIN(Q)  c
h g 2
f
1
8  
key [d] = 7  [d] = c
4 8 7

8 7
key [f] = 4  [f] = c
b c d
4 9 key [i] = 2  [i] = c
2 2
 
a 11 i 4 14 e
7 6 7 4 8 2
8 10
Q = {d, e, f, g, h, i} VA = {a, b, c}
h g 2
f
1
8   Extract-MIN(Q)  i
4
13
 Example
4 8 7 key [h] = 7  [h] = i
8 7
b c d key [g] = 6  [g] = i
4 9
2 2  7 46 8
a 11 i 4 14 e
6
Q = {d, e, f, g, h} VA = {a, b, c, i}
7
8 10
h g f Extract-MIN(Q)  f
1 2
87 
6 4
4 8 7 key [g] = 2  [g] = f
b
8
c
7
d key [d] = 7  [d] = c unchanged
4 9 key [e] = 10  [e] = f
2 2 10
a 11 i 4 14 e 7 10 2 8
7 6
8 10 Q = {d, e, g, h} VA = {a, b, c, i, f}
h g f
1 2 Extract-MIN(Q)  g
7 6
2 4
14
 Example
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c, i, f, g}
a 11 i 4 14 e
7 6 Extract-MIN(Q)  h
8 10
h g 2
f
1
7
1 2 4 7 10
4 8 7
8 7
Q = {d, e} VA = {a, b, c, i, f, g, h}
b c d
4 9 Extract-MIN(Q)  d
2 2 10
a 11 i 4 14 e
7 6
8 10
h g 2
f
1
1 2 4
15
 Example

4 8 7
8 7
b c d key [e] = 9  [e] = f
4 9 9
2 2 10
9
a 11 i 4 14 e
7 6 Q = {e} VA = {a, b, c, i, f, g, h, d}
8 10
h
1
g 2
f Extract-MIN(Q)  e
1 2 4 Q =  VA = {a, b, c, i, f, g, h, d, e}

16
 PRIM(V, E, w, r)
1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV)
2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)
Min-heap
7. while Q   Executed |V| times
operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)

9. for each v  Adj[u] Executed O(E) times total

10. do if v  Q and w(u, v) < key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)

12. DECREASE-KEY(Q, v, w(u, v))

17
 Using Fibonacci Heaps

• Depending on the heap implementation, running time

could be improved!

18
 Prim’s Algorithm
• Prim’s algorithm is a “greedy” algorithm
– Greedy algorithms find solutions based on a sequence
of choices which are “locally” optimal at each step.

• Nevertheless, Prim’s greedy strategy produces a

globally optimum solution!
– See proof for generic approach (i.e., slides 12-15)

19
 Kruskal’s Algorithm
• How is it different from Prim’s algorithm?
– Prim’s algorithm grows one
tree all the time
– Kruskal’s algorithm grows tree1

multiple trees (i.e., a forest)

at the same time.
u
– Trees are merged together
using safe edges
v
– Since an MST has exactly |V| - 1 tree2
edges, after |V| - 1 merges,
we would have only one component
20
 Kruskal’s Algorithm
8 7
• Start with each vertex being its b c d
4 9
own component 2
a 11 i 4 14 e
• Repeatedly merge two 7 6
8 10
components into one by h g f
1 2
choosing the light edge that
We would add
connects them edge (c, f)
• Which components to consider
at each iteration?
– Scan the set of edges in
monotonically increasing order by
weight

21
 Example
1. Add (h, g) {g, h}, {a}, {b}, {c}, {d}, {e}, {f}, {i}
8 7
b c d 2. Add (c, i) {g, h}, {c, i}, {a}, {b}, {d}, {e}, {f}
4 9
2 3. Add (g, f) {g, h, f}, {c, i}, {a}, {b}, {d}, {e}
a 11 i 4 14 e 4. Add (a, b) {g, h, f}, {c, i}, {a, b}, {d}, {e}
7 6
8 10 5. Add (c, f) {g, h, f, c, i}, {a, b}, {d}, {e}
h g 2
f 6. Ignore (i, g) {g, h, f, c, i}, {a, b}, {d}, {e}
1
7. Add (c, d) {g, h, f, c, i, d}, {a, b}, {e}
1: (h, g) 8: (a, h), (b, c) 8. Ignore (i, h) {g, h, f, c, i, d}, {a, b}, {e}
2: (c, i), (g, f) 9: (d, e) 9. Add (a, h) {g, h, f, c, i, d, a, b}, {e}
4: (a, b), (c, f) 10: (e, f) 10. Ignore (b, c) {g, h, f, c, i, d, a, b}, {e}
6: (i, g) 11: (b, h) 11. Add (d, e) {g, h, f, c, i, d, a, b, e}
7: (c, d), (i, h) 14: (d, f) 12. Ignore (e, f) {g, h, f, c, i, d, a, b, e}
{a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i} 13. Ignore (b, h) {g, h, f, c, i, d, a, b, e}
14. Ignore (d, f) {g, h, f, c, i, d, a, b, e}
22
 Implementation of Kruskal’s Algorithm

8 7
• Uses a disjoint-set data 4
b c d
9
2
structure (see Chapter a 11 i 4 14 e
7 6
21) to determine whether 8 10
h g 2
f
an edge connects 1
We would add
vertices in different edge (c, f)

components

23
 Operations on Disjoint Data Sets
• MAKE-SET(u) – creates a new set whose only
member is u
• FIND-SET(u) – returns a representative element
from the set that contains u
– Any of the elements of the set that has a particular
property
– E.g.: Su = {r, s, t, u}, the property is that the element be
the first one alphabetically
FIND-SET(u) = r FIND-SET(s) = r
– FIND-SET has to return the same value for a given set
24
 Operations on Disjoint Data Sets
• UNION(u, v) – unites the dynamic sets that
contain u and v, say Su and Sv

– E.g.: Su = {r, s, t, u}, Sv = {v, x, y}

UNION (u, v) = {r, s, t, u, v, x, y}

• Running time for FIND-SET and UNION depends

on implementation.
• Can be shown to be α(n)=O(lgn) where α() is a
very slowly growing function (see Chapter 21)
25
 KRUSKAL(V, E, w)
1. A← 
2. for each vertex v  V
O(V)
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
Running time: O(V+ElgE+ElgV)=O(ElgE) – dependent on
the implementation of the disjoint-set data structure
26
 KRUSKAL(V, E, w) (cont.)
1. A← 
2. for each vertex v  V
O(V)
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
- Running time: O(V+ElgE+ElgV)=O(ElgE) O(ElgV)
- Since E=O(V2), we have lgE=O(2lgV)=O(lgV)
27
 Kruskal’s Algorithm
• Kruskal’s algorithm is a “greedy” algorithm
• Kruskal’s greedy strategy produces a globally
optimum solution
S
• Proof for generic approach
x

applies to Kruskal’s
u y
algorithm too
v
V-S

28
Ques: Let G be an undirected connected graph
with distinct edge weight. Let emax be the edge
with maximum weight and emin the edge with
minimum weight. Which of the following
statements is false?
(A) Every minimum spanning tree of G must
contain emin.
(B) If emax is in a minimum spanning tree, then its
removal must disconnect G
(C) No minimum spanning tree contains emax
(D) G has a unique minimum spanning tree

29
Ques: Consider a complete undirected graph with
vertex set {0, 1, 2, 3, 4}. Entry Wij in the matrix W
below is the weight of the edge {i, j}. What is the
minimum possible weight of a spanning tree T in
this graph such that vertex 0 is a leaf node in the
tree T?

30
Ques: The number of distinct minimum spanning
trees for the weighted graph below is ____

31