Computational Complexity

(NP-completeness)
 Computational Complexity
• Almost all the algorithms we have studied thus far have been
polynomial-time algorithms
– on inputs of size n, their worst-case running time is O(nk) for some
constant k
• Can all problems can be solved in polynomial time?
– The answer is no
– There are problems, such as Turing’s famous “Halting Problem,” that
cannot be solved by any computer, no matter how much time we
allow
– There are problems for which polynomial time algorithms are not
known
 Computational Complexity
• In many search problems, we have to find the optimal
solution from an exponentially large number of possible
solutions
– Shortest path problem
– Minimum spanning tree
• We have polynomial time solutions to some of them
– These are exceptions!
• But there are many problems for which we don’t have any
efficient algorithm
– Efficient algorithms: O(nk) time algorithm (that’s the norm)
 Computational Complexity
• There are many problems for which we don’t have
any efficient algorithm
– How do you know there is no efficient solution (like the
lower bound on comparison based sorting)?

I cannot solve it efficiently because I am I cannot find an efficient

dumb! solution because there isn’t
any!

Image credit: “Computers and Intractability” by Garey and Johnson

 Computational Complexity
• It is really hard to prove that there is no efficient algorithm for
a particular problem.
– How do you know there is no efficient solution (like the lower bound
on comparison based sorting)?
• What to do?
– Show that the problem is as hard as many other problems that have
been worked on by a host of brilliant scientists over a very long time,
yet there is no efficient solutions
• A fundamental aspect of complexity theory is categorizing
problems into such equivalence classes
– Comparing relative difficulty of different problems
 P and NP
• P (polynomial): set of decision problems that can be solved in
polynomial time
– Relatively easier problems that can be solved quickly!
• NP (non-deterministic polynomial): Set of problems that can
be verified in polynomial time
– If we were somehow given a “certificate” of a solution, then we could
verify that the certificate is correct in polynomial time
• Eulerian cycle problem vs Hamiltonian (Rudrata) cycle
problem
 P and NP
• Example: Is the following Boolean formula satisfiable?
– (X v Y v Z) ^ (X’ v Y’ v Z)
– There is no efficient algorithm (to date) to solve this decision problem.
May not be in P (but nobody knows)
• What if we are given an assignment and just need to verify
whether it satisfies the given Boolean formula?
– I: (X v Y v Z) ^ (X’ v Y’ v Z); S: X = 1, Y = 0, Z = 1
– We can check in polynomial time whether S satisfies I.
– The satisfiability problem is in NP
– Do you see why there is no efficient algorithm to verify the “NO”
answer?
 P and NP
• Any problem in P is also in NP
– Since if a problem is in P then we can solve it in polynomial
time without even being supplied a certificate
• So P ⊆ NP
• The open question is whether or not P is a proper
subset of NP i.e. is P=NP?
Widely believed relationship
 NP-complete
• NP-complete problem (informally)
– The problem is in NP
– It is as “hard” as any problem in NP
• If any NP-complete problem can be solved in polynomial time,
then every problem in NP has a polynomial time algorithm
– A wide range of NP-complete problems have been studied to date—
without anyone having discovered a polynomial time solution to any
of them
– It would be truly astounding if all of them could be solved in
polynomial time
 Decision/search vs optimization
problems
• A decision problem is a problem whose output is a
single boolean value: YES or NO.
– Given a CNF Boolean formula, is there any satisfying
assignment?
– (X v Y v Z) ^ (X’ v Y’ v Z)
• In an optimization problem, we wish to find a
(feasible/valid) solution with the best value
– Shortest path problem
 Decision/search vs optimization
problems
• NP-completeness applies directly not to optimization
problems, but to decision problems
• But optimization problems can be converted to
decision problems
– Shortest path (Optimization problem): Find the shortest
path between two nodes s and t in a graph G
– Path (Decision problem): Is there a path P between s and t
in G such that the cost of the path is <= k?
 Decision/search vs optimization
problems
• If an optimization problem is easy, its related decision
problem is easy as well

• If we can provide evidence that a decision problem is

hard, we also provide evidence that its related
optimization problem is hard
 Polynomial time reductions
• A reduction from problem A to problem B is a
polynomial-time algorithm
– f that transforms any instance I of A into an instance f(I)
of B
– along with polynomial-time algorithm h that maps any
solution S of f(I) back into a solution h(S) of I
 Polynomial time reductions
Reduction
A ≤p B
Problem A is polynomial time reducible to problem B

• If A ≤p B and if we have a blackbox that solves B, then we can

solve A by reducing an instance of A into an instance of B

B is at least as hard as A

• If A ≤p B, then B ∈ P implies A ∈ P
 NP-complete problems
A problem L is NP-complete if
1. L ∈ NP, and
2. L’ ≤p L for every L’ ∈ NP

If a problem L satisfies property 2, but not

necessarily property 1, we say that L is NP-hard
NP-complete problems
Widely believed relationship
 A first NP-complete problem
• Satisfiability
– Boolean expression built from variables and
operators AND (conjunction, also denoted by ∧), OR
(disjunction, ∨), NOT (negation, ¬), and parentheses
– A formula is said to be satisfiable if it can be made
TRUE by assigning appropriate logical values (i.e.
TRUE, FALSE) to its variables
– The Boolean satisfiability problem (SAT) is, given a
formula, to check whether it is satisfiable
 A first NP-complete problem
• Cook’s theorem (1971)

Cook’s theorem

SAT is NP-complete
Proving NP-completeness
 Proving NP-completeness
• To prove that a problem L is NP-complete, we
need to
– Show L is in NP
– Find an NP-complete problem L’ which reduces to
L in polynomial time
• This shows that if someone finds a polynomial
algorithm for L, then P=NP
 Karp's 21 NP-complete problems
• Richard Karp showed 21 problems to be NP-
complete (1972)
 3-SAT
3-SAT satisfiability
• A literal in a boolean formula is an occurrence of a
variable or its negation
• A boolean formula is in conjunctive normal form, or
CNF, if it is expressed as an AND of clauses, each of
which is the OR of one or more literals.
• A boolean formula is in 3-SAT, if each clause has at
most three literals.
 Independent Set Problem
• In the INDEPENDENT SET problem we are given a
graph and an integer g, and the aim is to find g
vertices that are independent, that is, no two of
which have an edge between them
– In the optimization version, we seek to find the largest
independent set
 Independent Set is NP-complete
(proof)
1. Independent Set is in NP
2. Reduce an NP-complete problem to
Independent Set

• Convert any instance of 3SAT to an

instance of Independent Set
Independent Set is NP-complete
(proof)
 Independent Set is NP-complete
(proof)
• Given an instance I of 3SAT, we create an
instance (G, g) of INDEPENDENT SET as follows.
– Graph G has a triangle for each clause (or just an
edge, if the clause has two literals),with vertices
labeled by the clause’s literals, and has additional
edges between any two vertices that represent
opposite literals
– The goal g is set to the number of clauses.
 Independent Set is NP-complete
(proof)
1. Given an independent set S of g vertices in G, we need to
efficiently recover a satisfying truth assignment to I
– For any variable x, the set S cannot contain vertices
labeled both x and x’, because any such pair of vertices is
connected by an edge
– So assign x a value of true if S contains a vertex labeled x,
and a value of false if S contains a vertex labeled x’
– If S contains neither, then assign either value to x
– Since S has g vertices, it must have one vertex per clause;
this truth assignment satisfies those particular literals,
and thus satisfies all clauses
 Independent Set is NP-complete
(proof)
2. If graph G has no independent set of size g, then the Boolean
formula I is unsatisfiable
• It is usually cleaner to prove the contrapositive, that if I has a
satisfying assignment then G has an independent set of size g
• This is easy: for each clause, pick any literal whose value
under the satisfying assignment is true (there must be at
least one such literal). Add the corresponding vertex to S
 Hamiltonian (Rudrata) Cycle
Problem
• Hamiltonian (Rudrata) Cycle Problem
– Given a graph, find a cycle that visits each vertex
exactly once—or report that no such cycle exists.
HAM-CYCLE is NP-complete (proof)
1. HAM-CYCLE is in NP
2. Reduce an NP-complete problem to
Hamiltonian Cycle Problem

• Convert any instance of 3SAT to an

instance of Hamiltonian Cycle
HAM-CYCLE is NP-complete (proof)
• Create some graph structure (a “gadget”) that
represents the variables
• And some graph structure that represents the
clauses
• Hook them up in some way that encodes the
formula
• Show that this graph has a Ham. cycle iff the formula
is satisfiable
Gadget Representing the Variables

From slides by Carl Kingsford

Hooking in the Clauses

ck cj
(xi’ ∨ … ) … (xi’ ∨ … )
Connecting up the paths
Connecting up the paths

c1 c2 ck
(x1) (x1’∨ x2) … (x3’)
HAM-CYCLE is NP-complete (proof)
• A path encodes a truth assignment for the variables
(depending on which direction each chain is traversed)
• For there to be a Hamiltonian cycle, we have to visit
every clause node
• We can only visit a clause if we satisfy it (by setting one
of its terms to true)
• Hence, if there is a Hamiltonian cycle, there is a
satisfying assignment
• The converse is also true
 Traveling-salesperson problem
• Given a graph with n vertices, and cost c(i, j)
between each pair of vertices, does there exist a
tour visiting each city exactly once (finishing at the
starting vertex) of length ≤ k?
 TSP is NP-complete (proof)
1. TSP is in NP
2. Reduce an NP-complete problem to TSP

Hamiltonian Cycle ≤p TSP

• Convert any instance of Hamiltonian Cycle

to an instance of TSP
 TSP is NP-complete (proof)
• Let G = (V, E) be an instance of HAM-CYCLE
• We construct an instance of TSP as follows:
– We form the complete graph G’ = (V, E’) where E’
= {(i, j) | i, j ∈ V and i ≠ j}
– We define the cost function c by
 TSP is NP-complete (proof)
• Graph G has a hamiltonian cycle if and only if graph
G’ has a tour of cost at most 0
• Suppose that graph G has a hamiltonian cycle h
• Each edge in h belongs to E and thus has cost 0 in G’
• Thus, h is a tour in G’ with cost 0
 TSP is NP-complete (proof)
• Conversely, suppose that graph G’ has a tour h’ of
cost at most 0
• Since the costs of the edges in E’ are 0 and 1, the
cost of tour h’ is exactly 0 and each edge on the tour
must have cost 0
• Therefore, h’ contains only edges in E
• We conclude that h’ is a hamiltonian cycle in graph
G
 Clique Problem
• A clique in an undirected graph G = (V, E) is a subset
V’ ⊆ V of vertices, each pair of which is connected
by an edge in E
• In other words, a clique is a complete subgraph of G
• The clique problem is the optimization problem of
finding a clique of maximum size in a graph
• As a decision problem, we ask simply whether a
clique of a given size k exists in the graph
 Clique is NP-complete
• INDEPENDENT SET and CLIQUE are also easy to
reduce to one another
• Define the complement of a graph G = (V, E) to be G’
= (V, E’), where E’ contains precisely those
unordered pairs of vertices that are not in E
• Then a set of nodes S is an independent set of G if
and only if S is a clique of G’
• To paraphrase, these nodes have no edges between
them in G if and only if they have all possible edges
between them in G’
 Vertex Cover Problem
• A vertex cover of an undirected graph G= (V, E) is a
subset V’ ⊆ V such that if (u, v) ∈ E, then u ∈ V’ or v ∈
V’ (or both)
• That is, each vertex “covers” its incident edges, and a
vertex cover for G is a set of vertices that covers all the
edges in E
• The size of a vertex cover is the number of vertices in it
 Vertex Cover Problem
• The vertex-cover problem is to find a vertex cover of
minimum size in a given graph (optimization)
• Given a graph, determine whether a graph has a
vertex cover of a given size k (decision)
 Vertex Cover is NP-complete
• To reduce INDEPENDENT SET to VERTEX COVER we
just need to notice that a set of nodes S is a vertex
cover of graph G = (V, E) (that is, S touches every
edge in E) if and only if the remaining nodes, V − S,
are an independent set of G
 Vertex Cover is NP-complete
• Therefore, to solve an instance (G, g) of
INDEPENDENT SET, simply look for a vertex cover of
G with |V | − g nodes
• If such a vertex cover exists, then take all nodes not
in it
• If no such vertex cover exists, then G cannot possibly
have an independent set of size g