CS2040 – Data Structures

and Algorithms
Lecture 15 – Finding Shortest Way
from Here to There, Part I

[email protected]
 2

Outline
Single-Source Shortest Paths (SSSP) Problem
• Motivating example
• Some more definitions
• Discussion of negative weight edges and cycles

Algorithms to Solve SSSP Problem (CP4 Section 4.4)

• BFS algorithm (cannot be used for the general SSSP problem)
• Bellman Ford’s algorithm
– Precursor
– Pseudo code, example animation, and later: Java implementation
– Theorem, proof, and corollary about Bellman Ford’s algorithm
 3

Motivating Example
SoC

Sheares Hall
 4

Review: Definitions that you know

• Vertex set V (e.g. street intersections, houses, etc)
• Edge set E (e.g. streets, roads, avenues, etc)
– Directed (e.g. one way road, etc)
• Note that we can use bi-directed edges for two way roads, etc.
– Weighted (e.g. distance, time, toll, etc)
• Weight function w(a, b): E→R, sets the weight of edge from a to b
• Directed/Bi-directed Weighted Graph: G(V, E), w(a, b):
E→R
 5

More Definitions (1)

• Path p =  v0 , v1, v2 ,, vk 
– Where ( vi , vi +1 )  E , 0i ( k −1)
– In SSSP, the path is usually a simple path (no repeated vertex), unless
there is a negative cycle
p
• Shortcut notation: v0 vk
– Means that p is a path from v0 to vk
k −1
• Path weight: PW ( p ) =  w(vi , vi +1 )
i =0
 6

More Definitions (2)

• Shortest Path weight from vertex a to b: (a, b)
–  is pronounced as ‘delta’
If there exists such path
min( PW ( p)) p
 ( a, b) =  a b
 
If b is unreachable from a
• Single-Source Shortest Paths (SSSP) Problem:
– Given G(V, E), w(a, b): E->R, and a source vertex s
– Find (s, b) from vertex s to each vertex bV (together
with the corresponding shortest path)
• i.e. From one source to the rest
 7

More Definitions (3)

• Additional Data Structures to solve the SSSP Problem:
– An array/Vector D of size V (D stands for ‘distance’)
• Initially, D[v] = 0 if v = s; otherwise D[v] =  (a large number)
• D[v] decreases as we find better paths
• D[v] ≥ (s, v) throughout the execution of SSSP algorithm
• D[v] = (s, v) at the end of SSSP algorithm
– An array/Vector p of size V
• p[v] = the predecessor on best path from source s to v
• p[s] = -1 (not defined)
• Recall: The usage of this array/Vector p is already discussed
in BFS/DFS Spanning Tree
 8

Example

s=0 s=0
Initially: At the end of algorithm:
D[s] = D[0] = 0 D[s] = D[0] = 0 (unchanged)
D[v]=  for the rest D[v] = (s, v) for the rest
Denoted as values in red font/vertex e.g. D[2] = 6, D[4] = 7
p[s] = -1 (to say ‘no predecessor’) p[s] = -1 (source has no predecessor)
p[v] = -1 for the rest p[v] = the origin of orange edges for the rest
Denoted as orange edges (none initially) e.g. p[2] = 0, p[4] = 2
 9

Negative Weight Edges and Cycles

They exist in some applications
• Fictional application: Suppose you can travel back in time by
passing through time tunnel (edges with negative weight)

Take this as a cycle

• Shortest paths from 0 to {1, 2, 3} are undefined

– 1→2→1 is a negative cycle as it has negative total path (cycle) weight
– One can take 0→1→2→1→2→1→… indefinitely to get -
• Shortest path from 0 to 4 is ok, with (0, 4) = -99
 10

SSSP Algorithms
This SSSP problem is a(nother) well-known CS problem

We will discuss three algorithms in this lecture:

1. O(V+E) BFS which fails on general case of SSSP problem but
useful for a special case
– Introducing the “initSSSP” and “Relax” operations
2. General SSSP algorithm (pre-cursor to Bellman Ford)
3. O(VE) Bellman Ford’s SSSP algorithm
– General idea of SSSP algorithm
– Trick to ensure termination of the algorithm
– Bonus: Detecting negative weight cycle
 11

Initialization Step
We will use this initialization step
for all our SSSP algorithms

initSSSP(s)
for each v  V // initialization phase
D[v]  1000000000 // use 1B to represent INF
p[v]  -1 // use -1 to represent NULL
D[s]  0 // this is what we know so far
 12

“Relaxation” Operation
relax(u, v, w(u,v))
if D[v] > D[u]+w(u,v) // if SP can be shortened
D[v]  D[u]+w(u,v) // relax this edge
p[v]  u // remember/update the predecessor
// if necessary, update some data structure

4
u 4
4
p[u] = s
v
s 9 9
0 p[v] = s
8
p[v] = u
 13

BFS for SSSP

When the graph is unweighted/edges have same
weight*, the SSSP can be viewed as a problem of finding
the least number of edges traversed from source s to
other vertices
* We can view every edge as having weight 1

The O(V+E) Breadth First Search (BFS) traversal algorithm

precisely measures this (BFS Spanning Tree = Shortest Paths
Spanning Tree)
 14

Modified BFS
Do these three simple modifications:
1. Replace visited with D ☺
2. At the start of BFS, set D[v] = INF (say, 1 Billion) for all v in G,
except D[s] = 0 ☺
3. Change this part (in the BFS loop) from:
if visited[v] = 0 // if v is not visited before
visited[v] = 1; // set v as reachable from u
into:
if D[v] = INF // if v is not visited before
D[v] = D[u]+1; // v is 1 step away from u ☺
 15

Modified BFS Pseudo Code (1)

for all v in V
D[v]  INF
p[v]  -1 Initialization phase
Q  {s} // start from s
D[s]  0

while Q is not empty

u  Q.dequeue()
for all v adjacent to u // order of neighbor
if D[v] = INF // influences BFS Main
loop
D[v]  D[u]+1 // visitation sequence
p[v]  u
Q.enqueue(v)

// we can then use information stored in D/p

 17

SSSP: BFS on Unweighted Graph

Ask VisuAlgo to perform BFS from various sources
on the sample Graph (CP4 4.2)
In the screen shot below, we show the start of BFS from source
vertex 5 (the same example as in Lecture 13)
 18

But BFS will not work on general cases

The shortest path from 0 to 2 is not path 0→2 with weight 9,
but a “detour” path 0→1→3→4→2 with weight 2+3+2+1= 8
• BFS cannot detect this and will only report path 0→2 (wrong answer)
• You can draw this graph @ VisuAlgo and try it for yourself
Rule of Thumb:
If you know for sure that your graph is unweighted (all
edges have weight 1 or all edges have the same constant
weight), then solve the SSSP problem on it using the more
efficient O(V+E) BFS algorithm
 19

Reference: CP4 Section 4.4

visualgo.net/sssp

BELLMAN FORD’S SSSP ALGORITHM

 20

Precursor to Bellman Ford

How do we determine when an algorithm has solved the
SSSP?
• when for all edges (u,v), D[v] <= D[u] + w(u,v)
(i.e no edge can be relaxed further)

Validate this condition

on the example in slide 8
 21

Very simple algorithm to solve SSSP

initSSSP(s) // as defined in previous two slides

repeat // main loop

select edge(u, v)  E in arbitrary manner
relax(u, v, w(u, v)) // as defined in previous slide
until all edges have D[v] <= D[u] + w(u, v)
 22

Let’s Play a Simple Game

(Demo on Whiteboard – cannot be done on VisuAlgo)

s=0 s=0
Initially: At the end of algorithm:
D[s] = D[0] = 0 D[s] = D[0] = 0 (unchanged)
D[v]=  for the rest D[v] = (s, v) for the rest
Denoted as values in red font/vertex e.g. D[2] = 6, D[4] = 7
p[s] = -1 (to say ‘no predecessor’) p[s] = -1 (source has no predecessor)
p[v] = -1 for the rest p[v] = the origin of orange edges for the rest
Denoted as orange edges (none initially) e.g. p[2] = 0, p[4] = 2
 23

Algorithm Analysis
If given a graph without negative weight cycle,
when will this simple SSSP algorithm terminate?
A: Depends on your luck…
A: Can be very slow…

The main problem is in this line:

select edge(u, v)  E in arbitrary manner

Next, we will study Bellman Ford’s algorithm

that do these relaxations in a better order!
 24

Bellman Ford’s Algorithm

initSSSP(s)

// Simple Bellman Ford's algorithm runs in O(VE)

for i = 1 to |V|-1 // O(V) here
for each edge(u, v)  E // O(E) here
relax(u, v, w(u,v)) // O(1) here

// At the end of Bellman Ford's algorithm,

// D[v] = (s, v) if no negative weight cycle exist

// Q: Why "relaxing all edges V–1 times" works?

 25

SSSP: Bellman Ford’s

Ask VisuAlgo to perform Bellman Ford’s algorithm
from various sources on the sample Graph (CP4 4.14)

The screen shot below is the first pass of all E edges

of BellmanFord(0)
 26

Theorem 1 : If G = (V, E) contains no

negative weight cycle, then the shortest
path p from s to v is a simple path
4

Let’s do a Proof by Contradiction! 1 2 3

1. Suppose the shortest path p is not a simple path

2. Then p contains one (or more) cycle(s)
3. Suppose there is a cycle c in p with positive weight
4. If we remove c from p,
then we have a shorter ‘shortest path’ than p
5. This contradicts the fact that p is a shortest path
 27

Theorem 1 : If G = (V, E) contains no

negative weight cycle, then the shortest
path p from s to v is a simple path
0

6. Even if c is a cycle with zero total 1 0 3

weight (it is possible!), we can still remove c from p
without increasing the shortest path weight of p
7. So, p is a simple path (from point 5) or can always
be made into a simple path (from point 6)

In other words, path p has at most |V|-1 edges from

the source s to the “furthest possible” vertex v in G
(in terms of number of edges in the shortest path)
 28

Theorem 2 : If G = (V, E) contains no

negative weight cycle, then after Bellman
Ford’s terminates D[v] = (s, v), v  V
Let’s do a Proof by Induction!
1. Define vi to be any vertex that has shortest path p
requiring i number of edges from s
2. Initially D[v0] = (s, v0) = 0, as v0 is just s
3. After 1 pass through E, we have D[v1] = (s, v1)
4. After 2 passes through E, we have D[v2] = (s, v2), ...
5. After k passes through E, we have D[vk] = (s, vk)
 30

Theorem 2 : If G = (V, E) contains no

negative weight cycle, then after Bellman
Ford’s terminates D[v] = (s, v), v  V
6. When there is no negative weight cycle, the shortest
path p will be simple (see the previous proof)
7. Thus, after |V|-1 iterations, the “furthest” vertex v|V|-1
from s has D[v|V|-1] = (s, v|V|-1)
– Even if edges in E are processed in the worst
possible order
3

1 2 3 2
1
 31

“Side Effect” of Bellman Ford’s

Corollary: If a value D[v] fails to converge after |V|-1
passes, then there exists a negative-weight cycle
reachable from s

Additional check after running Bellman Ford’s:

for each edge(u, v)  E
if (D[u] != INF && D[v] > D[u]+w(u, v))
report negative weight cycle exists in G
 32

Java Implementation
See BellmanFordDemo.java
• Implemented using AdjacencyList ☺
– AdjacencyList or EdgeList can be used to have an O(VE) Bellman Ford’s

Show performance on:

• Small graph without negative weight cycle → OK, in O(VE)
• Small graph with negative weight cycle → terminate in O(VE)
– Plus we can report that negative weight cycle exists
• Small graph; some negative edges; no negative cycle → OK
 33

Summary
Introducing the SSSP problem

Revisiting BFS algorithm for unweighted SSSP problem

• But it fails on general case

Introducing Bellman Ford’s algorithm

• This one solves SSSP for general weighted graph in O(VE)
• Can also be used to detect the presence of -ve weight cycle