Intuitive Tour of Key Complexity Classes

Aimal Rextin

SEECS-NUST

Aimal Rextin (SEECS-NUST) Complexity Snapshot 1 / 15

Roadmap

1 P

2 NP

3 NP-Complete

4 EXP

5 Undecidable

Aimal Rextin (SEECS-NUST) Complexity Snapshot 2 / 15

Why decision problems?

Canonical yes/no form: every instance has a single-bit answer,

making time and
Optimisation ⇒ decision: most search or optimisation tasks are
polynomial-time equivalent to a decision version (e.g. is there k
weight path between s to other vertices versus shortest path).

Aimal Rextin (SEECS-NUST) Complexity Snapshot 3 / 15

Why decision problems?

Canonical yes/no form: every instance has a single-bit answer,

making time and
Optimisation ⇒ decision: most search or optimisation tasks are
polynomial-time equivalent to a decision version (e.g. is there k
weight path between s to other vertices versus shortest path).

Aimal Rextin (SEECS-NUST) Complexity Snapshot 3 / 15

P — “Fast” Algorithms

Intuition
Problems solvable by an algorithm whose running time grows at most like
a polynomial in the input size (e.g. n, n2 , n3 ).

Aimal Rextin (SEECS-NUST) Complexity Snapshot 4 / 15

P — “Fast” Algorithms

Intuition
Problems solvable by an algorithm whose running time grows at most like
a polynomial in the input size (e.g. n, n2 , n3 ).

Sorting n numbers (O(n log n)).

Finding a shortest path in a road network.
Checking if a year is a leap-year.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 4 / 15

P — “Fast” Algorithms

Intuition
Problems solvable by an algorithm whose running time grows at most like
a polynomial in the input size (e.g. n, n2 , n3 ).

Sorting n numbers (O(n log n)).

Finding a shortest path in a road network.
Checking if a year is a leap-year.
Take-away: These are the tasks we can usually handle even at
massive scale.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 4 / 15

NP — Solutions Check Quickly

Intuition
A problem is in NP if, **given a proposed solution**, a regular computer
can verify its correctness in polynomial time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 5 / 15

NP — Solutions Check Quickly

Intuition
A problem is in NP if, **given a proposed solution**, a regular computer
can verify its correctness in polynomial time.

Solving a Sudoku: easy to check a filled board, hard to fill it.

Finding a Hamiltonian path in a graph.
Satisfying a Boolean formula (SAT).

(¬x1 ∨ ¬x2 ) ∧ (x2 ∨ x3 ) ∧ (¬x3 ∨ x4 ∨ x1 )

Aimal Rextin (SEECS-NUST) Complexity Snapshot 5 / 15

NP-Complete — Toughest in NP

Intuition
If you design a fast (polynomial) algorithm for any NP-complete problem,
every NP problem inherits a fast algorithm.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 6 / 15

NP-Complete — Toughest in NP

Intuition
If you design a fast (polynomial) algorithm for any NP-complete problem,
every NP problem inherits a fast algorithm.

First discovered: SAT.

3-SAT, Clique, Travelling-Salesperson (decision version).

Aimal Rextin (SEECS-NUST) Complexity Snapshot 6 / 15

NP-Complete — Toughest in NP

Intuition
If you design a fast (polynomial) algorithm for any NP-complete problem,
every NP problem inherits a fast algorithm.

First discovered: SAT.

3-SAT, Clique, Travelling-Salesperson (decision version).
P vs NP: Do such fast algorithms exist? Still unknown.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 6 / 15

EXP — Exponential-Time Algorithms

Intuition
Problems whose best known **deterministic** algorithms may take time
like 2poly(n) in the worst case.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 7 / 15

EXP — Exponential-Time Algorithms

Intuition
Problems whose best known **deterministic** algorithms may take time
like 2poly(n) in the worst case.

Exhaustively exploring all game states in certain video games.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 7 / 15

EXP — Exponential-Time Algorithms

Intuition
Problems whose best known **deterministic** algorithms may take time
like 2poly(n) in the worst case.

Exhaustively exploring all game states in certain video games.

Provably P ⊊ EXP; so some tasks are guaranteed to lie beyond
polynomial time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 7 / 15

Undecidable Problems — No Algorithm Works for All
Inputs

Intuition
Some tasks cannot be solved by any algorithm that always halts with the
correct answer.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 8 / 15

Undecidable Problems — No Algorithm Works for All
Inputs

Intuition
Some tasks cannot be solved by any algorithm that always halts with the
correct answer.

“Will this program ever stop?” — the famous Halting Problem.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 8 / 15

Undecidable Problems — No Algorithm Works for All
Inputs

Intuition
Some tasks cannot be solved by any algorithm that always halts with the
correct answer.

“Will this program ever stop?” — the famous Halting Problem.

Consequence: No matter how clever we are, a universal solution is
impossible.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 8 / 15

Time/Space Hierarchy (Not to Scale)

EXP

NP
P

Aimal Rextin (SEECS-NUST) Complexity Snapshot 9 / 15

NP–Complete Optimisation Problem

Heuristic / AI approach

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Heuristic / AI approach
Often works well in practice on
typical instances.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Heuristic / AI approach
Often works well in practice on
typical instances.
But has no worst–case
guarantee on solution quality
or running time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Approximation algorithm

Heuristic / AI approach
Often works well in practice on
typical instances.
But has no worst–case
guarantee on solution quality
or running time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Approximation algorithm
Comes with a provable
Heuristic / AI approach performance ratio α.
Often works well in practice on
typical instances.
But has no worst–case
guarantee on solution quality
or running time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Approximation algorithm
Comes with a provable
Heuristic / AI approach performance ratio α.
Often works well in practice on Guarantees a solution within
typical instances. 1/α (or α times) of optimum.
But has no worst–case
guarantee on solution quality
or running time.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Approximation algorithm
Comes with a provable
Heuristic / AI approach performance ratio α.
Often works well in practice on Guarantees a solution within
typical instances. 1/α (or α times) of optimum.
But has no worst–case Optimal Our answer
guarantee on solution quality Value 100 80
or running time. Ratio 0.80 (80% of optimum)

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

NP–Complete Optimisation Problem

Approximation algorithm
Comes with a provable
Heuristic / AI approach performance ratio α.
Often works well in practice on Guarantees a solution within
typical instances. 1/α (or α times) of optimum.
But has no worst–case Optimal Our answer
guarantee on solution quality Value 100 80
or running time. Ratio 0.80 (80% of optimum)
The exact bound varies from algorithm
to algorithm.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 10 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

ρ = 2 ⇒ never worse than twice the best.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

ρ = 2 ⇒ never worse than twice the best.

ρ = 1 + ε (e.g. 1.1) is very tight.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

ρ = 2 ⇒ never worse than twice the best.

ρ = 1 + ε (e.g. 1.1) is very tight.
ρ may be a constant or a slow-growing function like log n.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

ρ = 2 ⇒ never worse than twice the best.

ρ = 1 + ε (e.g. 1.1) is very tight.
ρ may be a constant or a slow-growing function like log n.

OPT(I ) A(I )
⇒ ratio = 1.8 (a 1.8-approx)
100 180

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Approximation Ratio (quick view)

Goal. Get a solution that is close to optimum when exact optimisation is

too slow.
Definition (minimisation)
Algorithm A is a ρ-approximation if, for every instance I ,

cost(A(I )) ≤ ρ · OPT(I ).

ρ = 2 ⇒ never worse than twice the best.

ρ = 1 + ε (e.g. 1.1) is very tight.
ρ may be a constant or a slow-growing function like log n.

OPT(I ) A(I )
⇒ ratio = 1.8 (a 1.8-approx)
100 180

Aimal Rextin (SEECS-NUST) Complexity Snapshot 11 / 15

Vertex Cover (minimisation)

Problem statement
Input: an undirected graph G = (V , E ) of size n.
Goal: find the smallest subset C ⊆ V that touches every edge, i.e. for
every {u, v } ∈ E we have u ∈ C or v ∈ C .

Aimal Rextin (SEECS-NUST) Complexity Snapshot 12 / 15

Vertex Cover (minimisation)

Problem statement
Input: an undirected graph G = (V , E ) of size n.
Goal: find the smallest subset C ⊆ V that touches every edge, i.e. for
every {u, v } ∈ E we have u ∈ C or v ∈ C .

NP-complete =⇒ exact solutions need exponential time in the worst

case.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 12 / 15

Vertex Cover (minimisation)

Problem statement
Input: an undirected graph G = (V , E ) of size n.
Goal: find the smallest subset C ⊆ V that touches every edge, i.e. for
every {u, v } ∈ E we have u ∈ C or v ∈ C .

NP-complete =⇒ exact solutions need exponential time in the worst

case.
Best enumerate all 2n subsets and keep the smallest cover.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 12 / 15

Vertex Cover (minimisation)

Problem statement
Input: an undirected graph G = (V , E ) of size n.
Goal: find the smallest subset C ⊆ V that touches every edge, i.e. for
every {u, v } ∈ E we have u ∈ C or v ∈ C .

NP-complete =⇒ exact solutions need exponential time in the worst

case.
Best enumerate all 2n subsets and keep the smallest cover.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 12 / 15

Minimal Vertex Cover Example

minimal cover
|C | = 1

Larger graphs generally need many vertices in a cover, and brute-force checking
all subsets explodes to 2n .

Aimal Rextin (SEECS-NUST) Complexity Snapshot 13 / 15

2-Approximation for Vertex Cover

ApproxCover(G = (V, E)):

S = NULL # current cover
while E != empty set:
pick an arbitrary edge e = (u, v)
S = S U {u, v}
remove from E every edge incident to u or v
return S


Runs in O |E | time and guarantees |S| ≤ 2|O|.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 14 / 15

Why the Algorithm is a 2-Approximation

A — set of edges chosen by the algorithm (one per iteration)

O — an optimal vertex cover
S — vertices returned by the algorithm

|S| = 2 |A| (both endpoints of every edge in A) (1)

|O| ≥ |A| (each edge in A must be covered by O) (2)

From (1) and (2),

|S| ≤ 2 |O| .
Hence the algorithm is a 2-approximation.

Aimal Rextin (SEECS-NUST) Complexity Snapshot 15 / 15