Introduction to the Design and Analysis of Algorithms

Unit 5.
Greedy Technique

5.1 Introduction
– Computer scientists consider it a general design technique despite the fact that it is applicable
to optimization problems only.
– The greedy approach suggests constructing a solution through a sequence of steps, each
expanding a partially constructed solution obtained so far, until a complete solution to the
problem is reached.
– On each step—and this is the central point of this technique—the choice made must be:
 feasible, i.e., it has to satisfy the problem’s constraints
 locally optimal, i.e., it has to be the best local choice among all feasible choices available
on that step
 irrevocable, i.e., once made, it cannot be changed on subsequent steps of the algorithm

– As a rule, greedy algorithms are both intuitively appealing and simple.

– Given an optimization problem, it is usually easy to figure out how to proceed in a greedy
manner, possibly after considering a few small instances of the problem.
– What is usually more difficult is to prove that a greedy algorithm yields an optimal solution
(when it does).
– One of the common ways to do this is illustrated by the proof using mathematical induction, we
show that a partially constructed solution obtained by the greedy algorithm on each iteration
can be extended to an optimal solution to the problem.
– The second way to prove optimality of a greedy algorithm is to show that on each step it does
at least as well as any other algorithm could in advancing toward the problem’s goal
– The third way is simply to show that the final result obtained by a greedy algorithm is optimal
based on the algorithm’s output rather than the way it operates.

5.2 Prim’s Algorithm

– DEFINITION Aspanning tree of an undirected connected graph is its connected acyclic
subgraph (i.e., a tree) that contains all the vertices of the graph. If such a graph has weights
assigned to its edges, a minimum spanning tree is its spanning tree of the smallest weight,
where the weight of a tree is defined as the sum of the weights on all its edges. The minimum
spanning tree problem is the problem of finding a minimum spanning tree for a given
weighted connected graph.

– Below figure presents a simple example illustrating these notions.

Dept of Bachelor of Computer Science 1

 Introduction to the Design and Analysis of Algorithms

– Prim’s algorithm constructs a minimum spanning tree through a sequence of expanding subtrees.
– The initial subtree in such a sequence consists of a single vertex selected arbitrarily from the set
V of the graph’s vertices.
– On each iteration, the algorithm expands the current tree in the greedy manner by simply
attaching to it the nearest vertex not in that tree.
– The algorithm stops after all the graph’s vertices have been included in the tree being
constructed.
– Since the algorithm expands a tree by exactly one vertex on each of its iterations, the total
number of such iterations is n − 1, where n is the number of vertices in the graph.
– The tree generated by the algorithm is obtained as the set of edges used for the tree expansions.

– Here is pseudocode of this algorithm.

ALGORITHM Prim(G)
//Prim’s algorithm for constructing a minimum spanning tree
//Input: A weighted connected graph G = _V, E_
//Output: ET , the set of edges composing a minimum spanning tree of G

VT ←{v0} //the set of tree vertices can be initialized with any vertex
ET ←∅

find a minimum-weight edge e ∗ = (v∗, u∗) among all the edges (v, u)
for i ←1 to |V| − 1 do

VT←VT ∪ {u∗}
such that v is in VT and u is in V − VT

ET←ET ∪ {e∗}
return ET

– If a graph is represented by its adjacency lists and the priority queue is implemented as a min-
heap, the running time of the algorithm is in O(|E| log |V |).
– This is because the algorithm performs |V| − 1 deletions of the smallest element and makes |E|
verifications and, possibly, changes of an element’s priority in a min-heap of size not
exceeding
|V |.
– The running time of this implementation of Prim’s algorithm is in
(|V| − 1+ |E|)O(log |V |) = O(|E| log |V |)
because, in a connected graph, |V| − 1≤ |E|.

Dept of Bachelor of Computer Science 2

 Introduction to the Design and Analysis of Algorithms

– Below figure demonstrates the application of Prim’s algorithm to a specific graph.

Dept of Bachelor of Computer Science 3

 Introduction to the Design and Analysis of Algorithms

5.3 Kruskal’s Algorithm

– There is another greedy algorithm for the minimum spanning tree problem that also always
yields an optimal solution.
– It is named Kruskal’s algorithm after Joseph Kruskal, who discovered this algorithm.
– Kruskal’s algorithm looks at a minimum spanning tree of a weighted connected graph G = _V,
E_ as an acyclic subgraph with |V| − 1 edges for which the sum of the edge weights is the
smallest.
– Consequently, the algorithm constructs a minimum spanning tree as an expanding sequence of
subgraphs that are always acyclic but are not necessarily connected on the intermediate stages
of the algorithm.

– The algorithm begins by sorting the graph’s edges in nondecreasing order of their weights.
– Then, starting with the empty subgraph, it scans this sorted list, adding the next edge on the list
to the current subgraph if such an inclusion does not create a cycle and simply skipping the
edge otherwise.

– Algorithm is as follows:
ALGORITHM Kruskal(G)
//Kruskal’s algorithm for constructing a minimum spanning tree
//Input: A weighted connected graph G = _V, E_
//Output: ET , the set of edges composing a minimum spanning tree of G sort E in
nondecreasing order of the edge weights w(ei1) ≤ . . . ≤ w(ei|E|)

ET←∅; ecounter ←0 //initialize the set of tree edges and its size
k←0 //initialize the number of processed edges
while ecounter < |V| − 1 do

if ET ∪ { eik } is acyclic
k←k + 1

ET←ET ∪ {eik};
ecounter ←ecounter +
1
return ET

– With an efficient sorting algorithm, the time efficiency of Kruskal’s algorithm will be in
O(|E| log |E|).

Dept of Bachelor of Computer Science 4

 Introduction to the Design and Analysis of Algorithms

– Below Figure demonstrates the application of Kruskal’s algorithm

5.4Dijkstra’s Algorithm
– Consider the single-source shortest-paths problem: for a given vertex called the source in a
weighted connected graph, find shortest paths to all its other vertices.
– There are several well-known algorithms for finding shortest paths, the best-known algorithm
for the single-source shortest-paths problem, called Dijkstra’s algorithm.
– This algorithm is applicable to undirected and directed graphs with nonnegative weights only.

– Dijkstra’s algorithm finds the shortest paths to a graph’s vertices in order of their distance from
a given source.

Dept of Bachelor of Computer Science 5

 Introduction to the Design and Analysis of Algorithms

– First, it finds the shortest path from the source to a vertex nearest to it, then to a second nearest,
and so on.
– In general, before its ith iteration commences, the algorithm has already identified the shortest
paths to i − 1 other vertices nearest to the source.
– Since all the edge weights are nonnegative, the next vertex nearest to the source can be found
among the vertices adjacent to the vertices of Ti .

– The set of vertices adjacent to the vertices in Ti can be referred to as “fringe vertices”; they are
the candidates from which Dijkstra’s algorithm selects the next vertex nearest to the source.
– To identify the ith nearest vertex, the algorithm computes, for every fringe vertex u, the sum of
the distance to the nearest tree vertex v (given by the weight of the edge (v, u)) and the length
dv of the shortest path from the source to v and then selects the vertex with the smallest such
sum.

– Algorithm is as follows:

ALGORITHM Dijkstra(G, s)
//Dijkstra’s algorithm for single-source shortest paths
//Input: A weighted connected graph G = _V, E_ with nonnegative weights and its vertex s
//Output: The length dv of a shortest path from s to v and its penultimate vertex pv for every
vertex v in V

Initialize(Q) //initialize priority queue to empty

for every vertex v in V
dv ←∞;
pv ←null
Insert(Q, v, dv) //initialize vertex priority in the priority queue
Ds ←0;
Decrease(Q, s, ds) //update priority of s with ds

VT←∅
for i ←0 to |V| − 1 do

VT←VT ∪ {u∗}
u∗←DeleteMin(Q) //delete the minimum priority element

for every vertex u in V − VT that is adjacent to u∗ do

if du∗ + w(u∗, u) < du
du←du∗ + w(u∗,
u); pu←u∗
Decrease(Q, u, du)

Dept of Bachelor of Computer Science 6

 Introduction to the Design and Analysis of Algorithms

– Below figure shows the example for Dijkstra’s Algorithm

– The shortest paths (identified by following nonnumeric labels backward from a destination
vertex in the left column to the source) and their lengths (given by numeric labels of the tree
vertices) are as follows:
from a to b : a − b of length 3
from a to d : a − b − d of length 5
from a to c : a − b − c of length 7
from a to e : a − b − d − e of length 9

Dept of Bachelor of Computer Science 7