Matroids and The Greedy Algorithm
Matroids and The Greedy Algorithm
1
Matroids and the Greedy Algorithm
Independence Axioms
I1. ∅ ∈ I(M).
I2. X ⊂ Y ∈ I(M) =⇒ X ∈ I(M).
I3. X ∈ I(M), Y ∈ I(M), |Y | > |X | =⇒ ∃ e ∈ Y \ X such that X + e ∈
I(M).
49
The set I(M) is called the set of independent sets of M. The set E(M) is
called the ground set of M. Property I3 is called the exchange axiom.
What follows are some examples that we will revisit as we proceed.
Let G be a graph with vertex set V (G) and edge set E(G). We denote the
numbers of connected components of G (counting isolated vertices as compo-
nents) by κ(G). For F ⊂ E(G), let G.F (G restricted to F) denote the graph
with V (G.F) := V (G) and E(G.F) := F. A set of edges F of graph G is a
forest if it contains no cycle.
Proof. By induction of |F|. Clearly true for |F| = 0. For the inductive step, we
just observe that, for e ∈ F, κ(G.(F − e)) = κ(G.F) − 1.
Example (Graphic matroid). Let G be a graph. Let E(M) := E(G), and let
I(M) be the set of forests of G. In this case, we say that M is the graphic
matroid of G. It is easy to see that I1 and I2 hold. To see how I3 holds, suppose
that X and Y are forests such that X + e is not a forest for every e ∈ Y \ X .
Then every edge in Y \ X would have both ends in the same component of G.X .
Hence, κ(G.Y ) ≥ κ(G.X ). Therefore, by the Lemma (Forest components), we
have |Y | ≤ |X |. ♠
Example (Uniform matroid). Let E(M) be a finite set, and let r be an integer
satisfying 0 ≤ r ≤ |E(M)|. Let I(M) := {X ⊂ E(M) : |X | ≤ r }. In this case,
we say that M is a uniform matroid. ♠
Circuit Properties
C1. ∅ ∈ C(M).
C2. X ∈ C(M), Y ∈ C(M), X ⊂ Y =⇒ X = Y .
C3. X ∈ C(M), Y ∈ C(M), X = Y , e ∈ X ∩ Y =⇒ ∃ Z ⊂ (X ∪ Y ) − e
such that Z ∈ C(M).
Y
X
g
e
f
Hence,
1.3 Representations 53
It should be clear that C(M) completely determines I(M) for any indepen-
dence system. That is, given E(M) and C(M) satisfying C1 and C2, there is
precisely one choice of I(M) that has circuit set C(M) that will satisfy I1 and
I2. That choice is
1.3 Representations
The Fano matroid is the matroid represented over GF(2) by the matrix
1 2 3 4 5 6 7
⎛ ⎞
1 0 0 0 1 1 1
F = ⎝0 1 0 1 0 1 1 ⎠.
0 0 1 1 1 0 1
Exercise [Linear over GF(2) =⇒ graphic]. Prove that the Fano matroid
is not graphic.
Proof. If the Fano matroid can be represented over a field F, then it has a
minimal representation over F of the form
1 2 3 4 5 6 7
⎛ ⎞
a11 a12 a13 a14 a15 a16 a17
A = ⎝ a21 a22 a23 a24 a25 a26 a27 ⎠.
a31 a32 a33 a34 a35 a36 a37
The first three columns of A must be linearly independent, so, by using ele-
mentary row operations, we can bring A into the form
1 2 3 4 5 6 7
⎛ ⎞
1 0 0 a14 a15 a16 a17
A = ⎝ 0 1 0 a24 a25 a26 a27 ⎠.
0 0 1 a34 a35 a36 a37
1.3 Representations 55
1 2 3 4 5 6 7
⎛ ⎞
1 0 0 0 a b c
⎝0 1 0 d 0 e f ⎠,
0 0 1 g h 0 i
with the letters being nonzeros in the field F. We can bring the matrix into the
form
1 2 3 4 5 6 7
⎛ ⎞
1 0 0 0 1 1 1
⎝0 1 0 1 0 q 1 ⎠,
0 0 1 r s 0 1
with the letters being nonzeros, by nonzero row and column scaling (multiply
row 1 by c−1 , row 2 by f −1 , row 3 by i −1 , column 4 by d −1 f , column 5
by a −1 c, column 6 by b−1 c, column 1 by c, column 2 by f , and column 3
by i).
Now, columns 1, 4, and 7 should be dependent; calculating the determinant
and setting it to 0, we get r = 1. Similarly, the required dependence of columns
2, 5, and 7 implies s = 1, and the dependence of columns 3, 6, and 7 implies
q = 1. Therefore, over any field F, F is the only minimal representation of the
Fano matroid, up to projective equivalence.
Finally, columns 4, 5, and 6 should be dependent, so we get 1 + 1 = 0.
Therefore, the field must have characteristic 2.
The non-Fano matroid arises when the GF(2) representation of the Fano
matroid is used but the numbers are considered as rational. The representation
F, viewed over Q, is projectively equivalent to the rational matrix
1 2 3 4 5 6 7
⎛ ⎞
1 0 0 0 1/2 1/2 1/3
F− = ⎝ 0 1 0 1/2 0 1/2 1/3 ⎠.
1 1 1 1 1 1 1
Let F− be the matrix that we obtain by deleting the last row (of all 1’s) of F− .
The linear dependencies among the columns of F− are the same as the affine
dependencies among the columns of the matrix F− . We can plot the columns
of F− as points in the Euclidean plane and then visualize the independent sets
of the non-Fano matroid as the sets of points that are affinely independent (in
the plane, this means pairs of points that are not coincident and triples of points
that do not lie on a straight line):
6
4
7
3 5 1
rM (X ) := max{|Y | : Y ⊂ X, Y ∈ I(M)}.
We call r (E(M)) the rank of M. A set S ⊂ E(M) such that S ∈ I(M) and
M
|S| = rM (E(M)) is a base of M. We write B(M) for the set of bases of M. It
is a simple matter to find a base of the independence system M when M is
a matroid, provided that we can easily recognize when a set is in I(M). We
simply use a “greedy” algorithm:
Example [Vertex packing on a star, continued (see pp. 51, 53)]. If 1 is chosen
as the first element to put in S, then no other element can be added, but the only
base of M is {2, 3, . . . , n}. ♠
10
5 2
a
1 −1
c
−8
0
5
−10
e −9
d
Job j cj dj
1 20 3:00 p.m.
2 15 1:00 p.m.
3 10 2:00 p.m.
4 10 1:00 p.m.
5 6 2:00 p.m.
6 4 5:00 p.m.
7 3 5:00 p.m.
8 2 4:00 p.m.
9 2 2:00 p.m.
10 1 6:00 p.m.
Proof. We must prove that I(M) satisfies I3. The proof is by contradiction.
Choose Y and X so that I3 fails. We assign weights as follows:
⎧
⎨ 1 + , if e ∈ X
c(e) := 1, if e ∈ Y \ X ,
⎩
0, if e ∈ E(M) \ (X ∪ Y )
Swapping Algorithm
1. Choose any S ∈ I(M), such that |S| = k.
2. While (∃ S ∈ I(M) with |S | = k, |S S | = 2 and c(S ) > c(S)): Let
S := S .
Rank Properties
R1. 0 ≤ r (X ) ≤ |X |, and integer valued, ∀ X ⊂ E.
R2. X ⊂ Y =⇒ r (X ) ≤ r (Y ), ∀ X, Y ⊂ E.
R3. r (X ∪ Y ) + r (X ∩ Y ) ≤ r (X ) + r (Y ), ∀ X, Y ⊂ E.
Example [Vertex packing on a star, continued (see pp. 51, 53, 57)]. For X :=
{1, i} and Y := {1, j}, with i = j, we have rM (X ) = 1, rM (Y ) = 1, rM (X ∪
Y ) = 2, and rM (X ∩ Y ) = 1. ♠
Y
X
K \ JX
J
JX \ J
JY\ J
Our next goal is to show that R1–R3 characterize the rank functions of
matroids. That is, for every E and r satisfying R1–R3, there is a matroid M
with E(M) = E and rM = r . First, we establish a useful lemma.
Proof. We show that the choice of I(M) in the statement of the theorem satisfies
I1–I3, and then show that r is indeed the rank function of M.
R1 implies that r (∅) = 0; therefore, ∅ ∈ I(M), and I1 holds for I(M).
Now, suppose that X ⊂ Y ∈ I(M). Therefore, r (Y ) = |Y |. R3 implies that
r (X ∪ (Y \ X )) + r (X ∩ (Y \ X )) ≤ r (X ) + r (Y \ X ),
which reduces to
r (Y ) ≤ r (X ) + r (Y \ X ).
1.6 Duality 63
rM (Y ) = max{|X | : X ⊂ Y, |X | = r (X )}.
max{|X | : X ⊂ Y, |X | = r (X )} ≤ r (Y ).
1.6 Duality
Every matroid M has a natural dual M ∗ with E(M ∗ ) := E(M) and
X
B
rM ∗ (X ) = |X \ B X |
= |X | + |B X \ X | − |B X |
= |X | + rM (E(M) \ X ) − rM (E(M)).
It is clear from the specification of I(M ∗ ) that the bases of M ∗ are precisely
the complements of the bases of M. That is, B(M ∗ ) = {E(M) \ B : B ∈
B(M)}. Therefore, another algorithm for finding a maximum-weight base B of
M, with respect to the weight function c, is to use the Greedy Algorithm to
select a minimum-weight base B ∗ of M ∗ , and then let B := E(M) \ B ∗ . The
choice of algorithm may depend on the structure of the matroid. Indeed, for
graphic matroids, there are specialized algorithms that do not appear to extend
to arbitrary matroids.
1.6 Duality 65
1
2
6
3 5
We construct the planar dual as the graph G ∗ , shown in the following figure
with the hollow vertices and dashed edges:
1
2
5
6
PI(M) = x ∈ R+E(M) : xe ≤ rM (T ), ∀ T ⊂ E(M) .
e∈T
by a point of PI(M) . Without loss of generality, let E(M) = {1, 2, . . . , n}, and let
c(1) ≥ c(2) ≥ · · · ≥ c(n). Let k be the greatest index among the nonnegative
weights. Let Te := {1, 2, . . . , e} for 1 ≤ e ≤ n, and let T0 = ∅. The Greedy
Algorithm for finding a maximum-weight independent set S can be viewed as
determining its characteristic vector x(S) as
rM (Te ) − rM (Te−1 ), if 1 ≤ e ≤ k
xe (S) := .
0, if k < e ≤ n
subject to:
(P)
xe ≤ rM (T ), ∀ T ⊂ E(M);
e∈T
xe ≥ 0, ∀ e ∈ E(M).
xe (S) = rM (Te ) − rM (Te−1 )
e∈T e∈T :
1≤e≤k
≤ rM (Te ∩ T ) − rM (Te−1 ∩ T ) (by R3)
e∈T :
1≤e≤k
= rM (Tk ∩ T ) − rM (∅)
= rM (T ) (by R1).
min rM (T ) yT
T ⊂E(M)
subject to:
(D)
yT ≥ c(e), ∀ e ∈ E(M)
T : e∈T
yT ≥ 0, ∀ T ⊂ E(M).
E(M)
As for P, we can construct a potential solution y ∈ R2 of D, defined by
⎧
⎨ c(e) − c(e + 1), if T = Te with 1 ≤ e < k
yT := c(k), if T = Tk .
⎩
0, otherwise
We need check only that y is feasible to D and that the objective value of x(S)
in P and that of y in D are equal. Then, by the Weak Duality Theorem, x(S) is
optimal in P. Therefore, every linear function is maximized over
x ∈ R+E(M) : xe ≤ rM (T ), ∀ T ⊂ E(M)
e∈T
by a point of PI(M) .
= c(e),
which is certainly ≥ c(e). For e = k, we have
yT = yT = c(k),
k
T : k∈T
which is certainly ≥ c(e), because c(e) < 0 for e > k. Therefore, the solution
y is feasible to D. Finally, we have equality of the objective values because
k−1
yT rM (T ) = c(k)rM (Tk ) + (c(l) − c(l + 1))rM (Tl )
T ⊂E(M) l=1
k
k
= c(l)rM (Tl ) − c(l)rM (Tl−1 )
l=1 l=2
k
= c(l) rM (Tl ) − rM (Tl−1 )
l=1
n
= c(l)xl .
l=1
Example [Vertex packing on a star, continued (see pp. 51, 53, 57, 61)]. Let
c(1) = 2 and c(2) = c(3) = · · · = c(n) = 1. Following the definition of x(S)
in the previous proof, x1 (S) = 1, x2 (S) = 0, x3 (S) = x4 (S) = · · · = xn (S) = 1,
which picks out the dependent set S = {1, 3, 4, . . . , n} having weight n,
subject to:
xe ≤ r (T ), ∀ T ⊂ E;
e∈T
xe ≥ 0, ∀ e ∈ E.
Furthermore, if k = n and we drop the inequalities xe ≥ 0, ∀ e ∈ E, then we
can omit the hypothesis that r satisfies R2.
is the sum of
x j ≤ rM (U )
j∈U
and
x j ≤ rM (T \ U ),
j∈T \U
when rM (T ) = rM (U ) + rM (T \ U ).
For an independence system M, a set T ⊂ E(M) is closed if rM (T + e) =
rM (T ) for no e ∈ E(M) \ T . If M is a matroid, then for every T ⊂ E(M) there
is a unique maximal superset cl M (T ) of T , called the closure (or span) of T ,
such that rM (T ) = rM (cl M (T )).
Rank inequalities for sets that are not closed are redundant because
x j ≤ rM (T )
j∈T
is the sum of
x j ≤ rM (T )
j∈cl M (T )
and
−x j ≤ 0, ∀ j ∈ cl M (T ) \ T.
and let
X (T ) := {x(S) : S ∈ T } ⊂ F(T ).
Observe that S ∈ T if and only if
x(S) ∈ PI(M) ∩ x ∈ R E(M) : xe = rM (T ) .
e∈T
T1 T2
J J2
1
e'
There are many theoretical and practical studies of the application of greedy
and local-search algorithms to combinatorial-optimization problems. One start-
ing point is the book by Aarts and Lenstra (1997).
Chapter 13 of the book by Ahuja, Magnanti, and Orlin (1993) describes
the details of efficient implementations of algorithms for the minimum-weight
spanning tree problem.