Introduction to Algorithms

6.046J/18.401J
LECTURE 15
Shortest Paths II
• Bellman-Ford algorithm
• DAG shortest paths
• Linear programming and
difference constraints
• VLSI layout compaction

Prof. Charles E. Leiserson

 Negative-weight cycles
Recall: If a graph G = (V, E) contains a negative-
weight cycle, then some shortest paths may not exist.
Example: …

<0

uu vv

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.2

 Negative-weight cycles
Recall: If a graph G = (V, E) contains a negative-
weight cycle, then some shortest paths may not exist.
Example: …

<0

uu vv

Bellman-Ford algorithm: Finds all shortest-path

lengths from a source s ∈ V to all v ∈ V or
determines that a negative-weight cycle exists.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.3
 Bellman-Ford algorithm
d[s] ← 0
for each v ∈ V – {s} initialization
do d[v] ← ∞
for i ← 1 to | V | – 1
do for each edge (u, v) ∈ E
do if d[v] > d[u] + w(u, v) relaxation
then d[v] ← d[u] + w(u, v) step
for each edge (u, v) ∈ E
do if d[v] > d[u] + w(u, v)
then report that a negative-weight cycle exists
At the end, d[v] = δ(s, v), if no negative-weight cycles.
Time = O(VE).
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.4
 Example of Bellman-Ford

–1 BB 2

AA 3 2 EE
1
4 –3
CC 5
D
D

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.5

 Example of Bellman-Ford
∞
–1 BB 2
0 ∞
AA 3 2 EE
1
4 –3
CC 5
D
D
∞ ∞
Initialization.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.6
 Example of Bellman-Ford
∞
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞ ∞
Order of edge relaxation.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.7
 Example of Bellman-Ford
∞
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞ ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.8

 Example of Bellman-Ford
∞
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞ ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.9

 Example of Bellman-Ford
∞
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞ ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.10

 Example of Bellman-Ford
−1
∞
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞ ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.11

 Example of Bellman-Ford
−1
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
∞4 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.12

 Example of Bellman-Ford
−1
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
4 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.13

 Example of Bellman-Ford
−1
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2
4 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.14

 Example of Bellman-Ford
−1
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.15

 Example of Bellman-Ford
−1
–1 BB 1
2
0
4
7 3 ∞
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 ∞
End of pass 1.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.16
 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 ∞1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.17

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 ∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.18

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 1
∞

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.19

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.20

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.21

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.22

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
D
D
2 1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.23

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
DD
2 −2
1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.24

 Example of Bellman-Ford
−1
–1 BB 1
2
0 7 3 1
4
AA 3 2 EE
5 1 8
2
4 6
–3
CC 5
DD
2 −2
End of pass 2 (and 3 and 4).
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.25
 Correctness
Theorem. If G = (V, E) contains no negative-
weight cycles, then after the Bellman-Ford
algorithm executes, d[v] = δ(s, v) for all v ∈ V.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.26

 Correctness
Theorem. If G = (V, E) contains no negative-
weight cycles, then after the Bellman-Ford
algorithm executes, d[v] = δ(s, v) for all v ∈ V.
Proof. Let v ∈ V be any vertex, and consider a shortest
path p from s to v with the minimum number of edges.
v
s
vv11 vv33 vvkk
p: vv0 vv22 …
0

Since p is a shortest path, we have

δ(s, vi) = δ(s, vi–1) + w(vi–1, vi) .
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.27
 Correctness (continued)
v
s
vv11 vv33 vvkk
p: vv0 vv22 …
0

Initially, d[v0] = 0 = δ(s, v0), and d[v0] is unchanged by

subsequent relaxations (because of the lemma from
Lecture 14 that d[v] ≥ δ(s, v)).
• After 1 pass through E, we have d[v1] = δ(s, v1).
• After 2 passes through E, we have d[v2] = δ(s, v2).
M
• After k passes through E, we have d[vk] = δ(s, vk).
Since G contains no negative-weight cycles, p is simple.
Longest simple path has ≤ |V| – 1 edges.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.28
 Detection of negative-weight
cycles
Corollary. If a value d[v] fails to converge after
|V| – 1 passes, there exists a negative-weight
cycle in G reachable from s.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.29

 Linear programming
Let A be an m×n matrix, b be an m-vector, and c
be an n-vector. Find an n-vector x that maximizes
cTx subject to Ax ≤ b, or determine that no such
solution exists.
n

m . ≤ maximizing .

A x ≤ b cT x
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.30
 Linear-programming
algorithms
Algorithms for the general problem
• Simplex methods — practical, but worst-case
exponential time.
• Interior-point methods — polynomial time and
competes with simplex.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.31

 Linear-programming
algorithms
Algorithms for the general problem
• Simplex methods — practical, but worst-case
exponential time.
• Interior-point methods — polynomial time and
competes with simplex.

Feasibility problem: No optimization criterion.

Just find x such that Ax ≤ b.
• In general, just as hard as ordinary LP.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.32

 Solving a system of difference
constraints
Linear programming where each row of A contains
exactly one 1, one –1, and the rest 0’s.
Example:
x1 – x2 ≤ 3
x2 – x3 ≤ –2 xj – xi ≤ wij
x1 – x3 ≤ 2

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.33

 Solving a system of difference
constraints
Linear programming where each row of A contains
exactly one 1, one –1, and the rest 0’s.
Example: Solution:
x1 – x2 ≤ 3 x1 = 3
x2 – x3 ≤ –2 xj – xi ≤ wij x2 = 0
x1 – x3 ≤ 2 x3 = 2

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.34

 Solving a system of difference
constraints
Linear programming where each row of A contains
exactly one 1, one –1, and the rest 0’s.
Example: Solution:
x1 – x2 ≤ 3 x1 = 3
x2 – x3 ≤ –2 xj – xi ≤ wij x2 = 0
x1 – x3 ≤ 2 x3 = 2

Constraint graph: (The “A”

wij matrix has
xj – xi ≤ wij vvii vvjj dimensions
|E | × |V |.)
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.35
 Unsatisfiable constraints
Theorem. If the constraint graph contains
a negative-weight cycle, then the system of
differences is unsatisfiable.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.36

 Unsatisfiable constraints
Theorem. If the constraint graph contains
a negative-weight cycle, then the system of
differences is unsatisfiable.
Proof. Suppose that the negative-weight cycle is
v1 → v2 → L → vk → v1. Then, we have
x2 – x1 ≤ w12
x3 – x2 ≤ w23
M
xk – xk–1 ≤ wk–1, k
x1 – xk ≤ wk1

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.37

 Unsatisfiable constraints
Theorem. If the constraint graph contains
a negative-weight cycle, then the system of
differences is unsatisfiable.
Proof. Suppose that the negative-weight cycle is
v1 → v2 → L → vk → v1. Then, we have
x2 – x1 ≤ w12
x3 – x2 ≤ w23 Therefore, no
M
xk – xk–1 ≤ wk–1, k values for the xi
x1 – xk ≤ wk1 can satisfy the
constraints.
0 ≤ weight of cycle
<0
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.38
 Satisfying the constraints
Theorem. Suppose no negative-weight cycle
exists in the constraint graph. Then, the
constraints are satisfiable.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.39

 Satisfying the constraints
Theorem. Suppose no negative-weight cycle
exists in the constraint graph. Then, the
constraints are satisfiable.
Proof. Add a new vertex s to V with a 0-weight edge
to each vertex vi ∈ V.

vv11
vv99
vv44
vv33
vv77
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.40
 Satisfying the constraints
Theorem. Suppose no negative-weight cycle
exists in the constraint graph. Then, the
constraints are satisfiable.
Proof. Add a new vertex s to V with a 0-weight edge
to each vertex vi ∈ V.

0 vv11
vv99 Note:
No negative-weight
s vv44 cycles introduced ⇒
vv33 shortest paths exist.
vv77
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.41
 Proof (continued)
Claim: The assignment xi = δ(s, vi) solves the constraints.
Consider any constraint xj – xi ≤ wij, and consider the
shortest paths from s to vj and vi:
δ(s, vi)
ss vvii

δ(s, vj) wij

vvjj
The triangle inequality gives us δ(s,vj) ≤ δ(s, vi) + wij.
Since xi = δ(s, vi) and xj = δ(s, vj), the constraint xj – xi
≤ wij is satisfied.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.42
 Bellman-Ford and linear
programming
Corollary. The Bellman-Ford algorithm can
solve a system of m difference constraints on n
variables in O(m n) time.
Single-source shortest paths is a simple LP
problem.
In fact, Bellman-Ford maximizes x1 + x2 + L + xn
subject to the constraints xj – xi ≤ wij and xi ≤ 0
(exercise).
Bellman-Ford also minimizes maxi{xi} – mini{xi}
(exercise).
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.43
 Application to VLSI layout
compaction
Integrated
-circuit
features:

minimum separation λ
Problem: Compact (in one dimension) the
space between the features of a VLSI layout
without bringing any features too close together.

© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.44

 VLSI layout compaction
d1

11
2

x1 x2
Constraint: x2 – x1 ≥ d 1 + λ
Bellman-Ford minimizes maxi{xi} – mini{xi},
which compacts the layout in the x-dimension.
© 2001–4 by Charles E. Leiserson Introduction to Algorithms November 3, 2004 L15.45