Dr.

Nour El-Houda BENALIA

2nd year -Preparatory Classes -
Graph Theory ENP

Basic Algorithms

1. Shortest path algorithms

3. Maximum Flows
2. Minimum Spanning Trees
 2 Graph Theory Basic Algorithms
• Modeling, problem representation
➢ Traffic in a city
➢ Evolution of the states of a system
➢ Installation of shortest path telephone circuits
➢ Electrical distribution
➢ Minimization of installation costs.....

• Problems solving
➢ Shortest path
➢ Minimum weight tree
➢ Maximum flow
➢ …
 3 Problems to deal with

Minimum weight
Shortest path Flows
tree

Apparaît dans
beaucoup de Their interpretation
situations pratiques. corresponds to the
Réseaux de circulation of physical Minimize the cost of
Télécommunications, flows on a network: installing telephone
circuits téléphoniques electrical distribution, lines in a locality.
sont routés au plus water supply network or
court chemin. any other liquid, routing of
La circulation dans packets on a computer
une ville. network, etc.
4

Part 2:
Maximum Flows
5
Network Flow
6 Outline

• Introduction
• Network flow definitions
• Flow examples
• Augmenting Paths
• Residual Graph
• Cuts/ Min Cut
• Ford Fulkerson Algorithm
• Maxflow-MinCut Theorem
 7 What is Network Flow ?

• Flow network is a directed graph G=(V,E) such that each edge has a
non-negative capacity c(u,v)≥0.

• Two distinguished vertices exist in G namely :

Source (denoted by s) : In-degree of this vertex is 0.

Sink (denoted by t) : Out-degree of this vertex is 0.

• Flow in a network is an integer-valued function f defined On the

edges of G satisfying 0≤f(u,v)≤c(u,v), for every Edge (u,v) in E.
 What is Network Flow ?
8
• Each edge (u,v) has a non-negative capacity c(u,v).
• We have source s and sink t.
• Assume that every vertex v in V is on some path from s to t.
• Remark: If (u,v) is not in E assume c(u,v)=0.
Following is an illustration of a network flow:

c(A,B)=5
c(B,A)=0
c(D,A)=0 …
 9 Conditions for Network Flow
For each edge (u,v) in E, the flow f(u,v) is a real valued function
that must satisfy following 3 conditions :

• Capacity Constraint : u,v V, f(u,v)  c(u,v)

• Skew Symmetry :  u,v V, f(u,v)= -f(v,u)
• Flow Conservation:  u V – {s,t}  f(s,v)=0
vV

Skew symmetry condition implies that f(u,u)=0.

 11 Example of a flow Capacity

Flow

1 8,8
10 , 9
1,1
s t

6, 6 10, 7
2

Table illustrating Flows and Capacity across different edges of graph above:
fs,1 = 9 , cs,1 = 10 (Valid flow since 10 > 9)
fs,2 = 6 , cs,2 = 6 (Valid flow since 6 ≥ 6)
f1,2 = 1 , c1,2 = 1 (Valid flow since 1 ≥ 1)
f1,t = 8 , c1,t = 8 (Valid flow since 8 ≥ 8)
f2,t = 7 , c2,t = 10 (Valid flow since 10 > 7)

The flow across nodes 1 and 2 are also conserved as flow into them = flow out
 10 Value of a Flow

The value of a flow is given by :

| f |=  f ( s , v ) =  f (v , t )
vV vV

The flow into the node is same as flow going out from the node and
thus the flow is conserved. Also the total amount of flow from source
s = total amount of flow into the sink t.
 13 Cuts of Flow Networks
A Cut in a network is a partition of V into S and T (T=V-S) such that s (source) is in S
and t (target) is in T.

2 9 5

Cut
10 15 15 10
4

s 5 3 8 6 10 t

4 6 15 10
15

4 30 7
 14 Capacity of Cut (S,T)

c(S ,T ) =  c (u , v )
u S
,vT
2 9 5

Cut
10 15 15 10
4

s 5 3 8 6 10 t

4 6 15 10
15
Capacity = 30
4 30 7
 15 Min Cut

Min s-t cut (Also called as a Min Cut) is a cut of minimum capacity

2 9 5

Min s-t Cut

10 15 15 10
4

s 5 3 8 6 10 t

4 6 15 10
15

Capacity = 28
4 30 7
16
Cut and the min cut
16
Cut and the min cut
16
Cut and the min cut
16
Cut and the min cut
 16 Flow of Min Cut (Weak Duality)
Let f be the flow and let (S,T) be a cut. Then | f | ≤ CAP(S,T).

In maximum flow, minimum cut problems forward edges are full

or saturated and the backward edges are empty because of the
maximum flow. Thus maximum flow is equal to capacity of cut.
This is referred to as weak duality.
Proof :

0 S T
8
t

s
7
17 Methods

Max-Flow Min-Cut Theorem

• The Ford-Fulkerson Method

• The Preflow-Push Method
18 The Ford-Fulkerson Method
• Try to improve the flow, until we reach the maximum value of the flow

• The residual capacity of the network with a flow f is given by:

The residual capacity (rc) of an edge (i,j) equals c(i,j) – f(i,j) when (i,j) is a
forward edge, and equals f(i,j) when (i,j) is a backward edge.
Moreover the residual capacity of an edge is always non-negative.

c f (u , v ) = c (u , v ) − f (u , v )

1 8,8 1 1 0
10 , 9
1,1 9 1 0 8
s t s t
6 7
6, 6 10, 7 0
2 2 3
Original Network Residual Network
 The Ford-Fulkerson Method

Begin
x := 0; // x is the flow.
create the residual network G(x);
while there is some directed path from s to t in G(x) do
begin
let P be a path from s to t in G(x);
Δ:= δ(P);
send Δunits of flow along P;
update the r's;
end
end {the flow x is now maximum}.

Click To See Ford Fulkerson’s Algorithm In Action (Animation)

20 Augmenting Paths ( A Useful Concept )
Definition:
An augmenting path p is a simple path from s to t on a residual network that is an
alternating sequence of vertices and edges of the form s,e1,v1,e2,v2,...,ek,t in which no
vertex is repeated and no forward edge is saturated and no backward edge is free.

Characteristics of augmenting paths:

• We can put more flow from s to t through p.

• The edges of residual network are the edges on which residual
capacity is positive.
• We call the maximum capacity by which we can increase the flow on p
the residual capacity of p.

cf ( p) = min{c f ( u , v ) : ( u , v ) is o n p }
21 The Ford-Fulkerson’s Algorithm
FORDFULKERSON(G,E,s,t) AUGMENT(f,P)
FOREACH e  E b  bottleneck(P)
f(e)  0 FOREACH e  P
Gf  residual graph IF (e  E)
WHILE (there exists augmenting path P) // backwards arc
f  augment(f, P) f(e)  f(e) + b
update Gf ELSE
ENDWHILE // forward arc
RETURN f
f(eR)  f(e) - b
RETURN f

Click To See Ford Fulkerson’s Algorithm In Action (Animation)

12
Maximum Flow Problem
representation

Given a Graph G (V,E) such that:

In simple terms maximize
xi,j = flow on edge (i,j) the s to t flow, while ensuring
ui,j= capacity of edge (i,j) that the flow is feasible.
s = source node
t = sink node

Maximize v

Subject To Σjxij - Σjxji = 0 for each i ≠s,t

Σjxsj = v

0 ≤ xij ≤ uij for all (i,j)  E.

The Ford-Fulkerson Algorithm for
the Maximum Flow Problem
 25 Search for a maximum flow: Ford-
Fulkerson algorithm
Step 1: Initialization
f0 (0, 0, ….. , 0), mark the vertex s.
Step 2: Calculation of the complete flow from the residual
network
Consider all the paths from s to p and improve the flow f the way to have a complete
flow.
Step 3: Calculation of the max flow and search for the min
cut
Phase I: Marking procedure
While exists an unmarked vertex y Do
mark y by + (direct marking) if f (u) < c(u)
mark y by - (indirect marking) if f (u) > 0
EndWhile
 Search for a maximum flow: Ford-
26
Fulkerson algorithm
Step 3: Calculation of the max flow and search for the min cut
Phase II: Improving the flow
If vertex p is marked Then Improve (f, C)
else f is maximum
Improve (f, C): if p is marked then an improving chain C is detected so we change the flow
as follows:

Where
 27 Ford-Fulkerson algorithm: Example
Consider the opposite network:

Find a maximum flow in R.

To determine a maximum flow in this

network, we can start by sending flow on
paths from s to p.
 28 Ford-Fulkerson algorithm: Example

We can start by sending a

stream of 2 along the path
(s, v1, v2, p).

The path (s; v1; p): 3

 29 Ford-Fulkerson algorithm: Example
The path (s, v2, p): 5

But this flow is not maximum,

we will use the marking
procedure

fcomplet = 10
 30 Ford-Fulkerson algorithm: Example
Consider the following improving chain: C = (s, v2, v1, p)
and thus = 1.

So by increasing the
flow by 1 on the arcs
(s, v2) and (v1, p) and
bydecreasing by 1 on
the arc (v1, v2) we
will have the following
flow:

We cannot mark the well so this flow is

maximum, fmax = 11
31 Ford-Fulkerson algorithm: Example
Let's look for the minimum cut and check if v(f) =c(K)
22 Proof of correctness of the algorithm
Lemma: At each iteration all residual capacities are integers.
Proof: It’s true at the beginning. Assume it’s true after the first k-1
augmentations, and consider augmentation k along path P.
The residual capacity Δ of P is the smallest residual capacity
on P, which is integral.
After updating, we modify the residual capacities by 0 or Δ, and thus
residual capacities stay integers.

Theorem: Ford-Fulkerson’s algorithm is finite

Proof: The capacity of each augmenting path is atleast 1. The augmentation
reduces the residual capacity of some edge (s,j) and doesn’t increase the
residual capacity for some edge (s,i) for any i.
So the sum of residual capacities of edges out of s keeps decr- easing, and is
bounded below 0.
Number of augmentations is O(nC) where C is the largest of the capacity in
the network.
23 When is the flow optimal ?

A flow f is maximum flow in G if :

(1)The residual network Gf contains no more augmented paths.

(2)| f | = c(S,T) for some cut (S,T) (a min-cut)
Proof:
(1)Suppose there is an augmenting path in Gf then it implies that
the flow f is not maximum, because there is a path through which
more data can flow. Thus if flow f is maximum then residual n/w
Gf will have no more augmented paths.

(2) Let v=Fx(S,T) be the flow from s to t. By assumption v=CAP(S,T)

By Weak duality, the maximum flow is at most CAP(S,T). Thus the
flow is maximum.
 27 Ford-Fulkerson algorithm: Example 2
Example:
We want to transport a product from warehouses to customers. 3 warehouses (1,
2 and 3), quantity in stock: 45, 25 and 25 resp., 4 customers (a, b, c and d),
demand: 30, 10, 20 and 30 resp.
The limitations for transporting a product from aWarehouse to a customer are
defined by:

Questions :
Can all demands be met?
If so, how?If not, how can they be met in the best possible way?
27 Ford-Fulkerson algorithm: Example 2
 38 Step 1 : Calculation of the complete flow
(Augmenting paths)
Building a complete flow in the first step of the algorithm → we
start from a complete flow instead of 0,
 39 Etape 2 : Calcul du flot complet

Start from a feasible flow (zero flow if b(u) = 0, ∀u ∈ U),

Examine all paths from E to S systematically,
For each path μ from E to S, pass a flow equal to the minimum residual capacity
of the arcs of μ.
40 Step 2: Calculation of the complete flow
(Augmenting paths)
 Step 2: Calculation of the complete flow
41
(Augmenting paths)
 Step 2: Calculation of the complete flow
42
(Augmenting paths)
 Step 2: Calculation of the complete flow
43
(Augmenting paths)
 Step 2: Calculation of the complete flow
44 (Augmenting paths)
 Step 2: Calculation of the complete flow
45 (Augmenting paths)
 Step 2: Calculation of the complete flow
46
(Augmenting paths)
47 Step 2: Calculation of the complete flow
(Augmenting paths)
 Step 3: Calculation of the maximum
48 flow and search for the minimum cut
Phase 1: Marking procedure
 Step 3: Calculation of the maximum
49 flow and search for the minimum cut
Phase 1: Marking procedure
 Step 3: Calculation of the maximum
50 flow and search for the minimum cut
Phase 1: Marking procedure
 Step 3: Calculation of the maximum
51 flow and search for the minimum cut
Phase 1: Marking procedure
 Step 3: Calculation of the maximum
52 flow and search for the minimum cut
Phase 1: Marking procedure
 Step 3: Calculation of the maximum
53 flow and search for the minimum cut
Phase 1: Marking procedure
 54 Flow change procedure

This procedure only applies if S is marked at the end of the

marking procedure. Let μ be a chain from E to S.
 55 Flow change procedure
Let μ+ be the set of arcs of the chain of type (xi, xi+1) “front arcs”.

Let μ− be the set of arcs of the chain of type (xj , xj+1) “back arcs”.

The arcs of μ+ are such that it is possible to increase the value of the
associated flux by a quantity + ≥ 0.

The arcs of μ− are such that it is possible to decrease the value of the
associated flux by a quantity − ≥ 0.
56 Improving path
57 New flow
58 Marking
59 Marking
60 Marking

We can no longer mark a vertex,

S is not marked
-> The flow is maximum.
61 Marquage

We can no longer mark a vertex,

S is not marked
-> The flow is maximum
The minimum value cut is generated by the subset of marked vertices
32

Part 4:
Minimum Spanning Trees
33 Outline

• Problem: Laying Telephone Wire

• Wiring: Naïve Approach
• Minimum Spanning Tree (MST)
• Applications of MST
• How Can We Generate a MST?
1. Prim’s Algorithm
2. Kruskal’s algorithm
34
Problem: Laying Telephone Wire

Central office
35 Wiring: Naïve Approach

Central office

Expensive!
36 Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

37 Minimum Spanning Tree (MST)

A minimum spanning tree is a subgraph of an undirected

weighted graph G, such that

• it is a tree (i.e., it is acyclic)

• it covers all the vertices V
– contains |V| - 1 edges
• the total cost associated with tree edges is the
minimum among all possible spanning trees
• not necessarily unique
39 How Can We Generate a MST?

9 b 9 b
a 2 6 a 2 6
d d
4 5 4 5
5 4 5 4

5 e 5 e
c c
40

Prim’s Algorithm
41 Prim’s Algorithm
Initialization
a. Pick a vertex r to be the root
b. Set D(r) = 0, parent(r) = null
c. For all vertices v  V, v  r, set D(v) = 
d. Insert all vertices into priority queue P,
using distances as the keys

9 Vertex Parent
b e -
a 2 6 e a b c d
d
5 4
4 5
0    
5 e
c
42 Prim’s Algorithm
While P is not empty:

1. Select the next vertex u to add to the tree

u = P.deleteMin()

2. Update the weight of each vertex w adjacent to

u which is not in the tree (i.e., w  P)
If weight(u,w) < D(w),
a. parent(w) = u
b. D(w) = weight(u,w)
c. Update the priority queue to reflect
new distance for w
43 Prim’s algorithm
Vertex Parent
e -
e d b c a b -
0     c -
9 b d -
a 2 6
d Vertex Parent
4 5
5 4 e -
5 e d b c a b e
c 4 5 5  c e
d e

The MST initially consists of the vertex e, and we update

the distances and parent for its adjacent vertices
44 Prim’s algorithm
Vertex Parent
e -
d b c a b e

9
4 5 5  c e
b d e
a 2 6
d
4 5
5 4
Vertex Parent
5 e e -
c a c b b e
2 4 5 c d
d e
a d
45 Prim’s algorithm
Vertex Parent
e -
a c b b e
2 4 5 c d
9 d e
b
a d
a 2 6
d
4 5
5 4
5 e
c Vertex Parent
e -
c b b e
4 5 c d
d e
a d
46 Prim’s algorithm
Vertex Parent
e -
c b b e
9 4 5 c d
b d e
a 2 6
a d
d
4 5
5 4

5 e Vertex Parent
c e -
b b e
5 c d
d e
a d
47 Prim’s algorithm
Vertex Parent
e -
b b e
5 c d
9 b d e
a 2 6 a d
d
4 5
5 4

5 e
c Vertex Parent
e -
b e
The final minimum spanning tree c d
d e
a d
48
Running time of Prim’s algorithm
(without heaps)
Initialization of priority queue (array): O(|V|)

Update loop: |V| calls

• Choosing vertex with minimum cost edge: O(|V|)
• Updating distance values of unconnected vertices:
each edge is considered only once during entire
execution, for a total of O(|E|) updates

Overall cost without heaps: O(|E| + |V| 2)

When heaps are used, apply same analysis as for Dijkstra’s

algorithm (p.469) (good exercise)
49 Prim’s Algorithm Invariant
• At each step, we add the edge (u,v) s.t. the weight of (u,v) is
minimum among all edges where u is in the tree and v is
not in the tree

• Each step maintains a minimum spanning tree of the

vertices that have been included thus far

• When all vertices have been included, we have a MST for

the graph!
 50 Correctness of Prim’s
• This algorithm adds n-1 edges without creating a cycle, so
clearly it creates a spanning tree of any connected graph (you
should be able to prove this).

But is this a minimum spanning tree?

Suppose it wasn't.

• There must be point at which it fails, and in particular there

must a single edge whose insertion first prevented the spanning
tree from being a minimum spanning tree.
 51 Correctness of Prim’s

• Let G be a connected,
undirected graph
• Let S be the set of edges
chosen by Prim’s x
algorithm before choosing y
an errorful edge (x,y)

• Let V' be the vertices incident with edges in S

• Let T be a MST of G containing all edges in S, but not (x,y).
 52 Correctness of Prim’s

v
• Edge (x,y) is not in T, so
there must be a path in T x
from x to y since T is
y
connected.
• Inserting edge (x,y) into T
will create a cycle
• There is exactly one edge on this cycle with exactly one
vertex in V’, call this edge (v,w)
53 Correctness of Prim’s

• Since Prim’s chose (x,y) over (v,w), w(v,w) >= w(x,y).

• We could form a new spanning tree T’ by swapping (x,y)
• for (v,w) in T (prove this is a spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also (x,y)

...Contradiction
54 Another Approach
• Create a forest of trees from the vertices
• Repeatedly merge trees by adding “safe edges” until only one
tree remains
• A “safe edge” is an edge of minimum weight which does not
create a cycle
9 b
a 2 6
d forest: {a}, {b}, {c}, {d}, {e}
4 5
5 4

5 e
c
55 Kruskal’s algorithm
Initialization
a. Create a set for each vertex v  V
b. Initialize the set of “safe edges” A
comprising the MST to the empty set
c. Sort edges by increasing weight

9 b
a 2 6 F = {a}, {b}, {c}, {d}, {e}
d A= 
4 5
5 4 E = {(a,d), (c,d), (d,e), (a,c),
5 e (b,e), (c,e), (b,d), (a,b)}
c
56 Kruskal’s algorithm
For each edge (u,v)  E in increasing order while
more than one set remains:
If u and v, belong to different sets U and V
a. add edge (u,v) to the safe edge set
A= A  {(u,v)}
b. merge the sets U and V
F = F - U - V + (U  V)

Return A

• Running time bounded by sorting (or findMin)

O(|E|log|E|), or equivalently, O(|E|log|V|) (why???)
57 Kruskal’s algorithm
9 b
a 2 6
E = {(a,d), (c,d), (d,e), (a,c),
d
5 4
4 5 (b,e), (c,e), (b,d), (a,b)}
5 e
c
Forest A
{a}, {b}, {c}, {d}, {e} 
{a,d}, {b}, {c}, {e} {(a,d)}
{a,d,c}, {b}, {e} {(a,d), (c,d)}
{a,d,c,e}, {b} {(a,d), (c,d), (d,e)}
{a,d,c,e,b} {(a,d), (c,d), (d,e), (b,e)}
58 Kruskal’s Algorithm Invariant

• After each iteration, every tree in the forest is a MST of the

vertices it connects

• Algorithm terminates when all vertices are connected into

one tree
59
Correctness of Kruskal’s
• This algorithm adds n-1 edges without creating a cycle, so
clearly it creates a spanning tree of any connected graph (you
should be able to prove this).

But is this a minimum spanning tree?

Suppose it wasn't.

• There must be point at which it fails, and in particular there

must a single edge whose insertion first prevented the
spanning tree from being a minimum spanning tree.
60 Correctness of Kruskal’s

K T
S
e

• Let e be this first errorful edge.

• Let K be the Kruskal spanning tree
• Let S be the set of edges chosen by Kruskal’s algorithm
• before choosing e
• Let T be a MST containing all edges in S, but not e.
 61 Correctness of Kruskal’s
Lemma: w(e’) >= w(e) for all edges e’in T - S

Proof (by contradiction):

Assume there exists some edge K T
e’in T - S, w(e’) < w(e) S
Kruskal’s must have e
considered e’ before e

• However, since e’ is not in K (why??), it must have been

discarded because it caused a cycle with some of the other
edges in S.
• But e’+ S is a subgraph of T, which means it cannot form a
cycle ...Contradiction
62
Correctness of Kruskal’s

• Inserting edge e into T will create a cycle

• There must be an edge on this cycle which is not in K
• (why??). Call this edge e’
• e’ must be in T - S, so (by our lemma) w(e’) >= w(e)
• We could form a new spanning tree T’ by swapping e for e’
• in T (prove this is a spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also e
• ...Contradiction
63 Greedy Approach

• Like Dijkstra’s algorithm, both Prim’s and Kruskal’s

algorithms are greedy algorithms

• The greedy approach works for the MST problem; however,

• it does not work for many other problems!
64

That’s All!