Lecturer: Andrew Parkes
http://www.cs.nott.ac.uk/~pszajp/
COMP2009-ACE
The “Master Theorem”
Master Theorem (MT)
Consider recurrence relations of the form
T(n) = a T(n/b) + f(n)
• Designed for “divide and conquer” in which
problems are divided into ‘a’ instances of a
problem of size n/b.
• Aim is to be able to quickly express the “big-Oh
family” behavior of T(n) for various cases of the
values of a and b, and the scaling behavior of
f(n).
It does not cover all cases, but does cover many
useful cases.
2
Master Theorem
• No proof needed in this module.
• Just learn it, and how to apply it!
• Suggest to generate and try many examples
3
Motivations
• Consider the special case that f(n) = 0
• T(n) = a T(n/b) with T(1) = 1
• T(b) = a
• T(b2) = a2
• T(b3) = a3
• So T(bk) = ak
• Exercise (offline): prove by induction
4
Motivations
• Consider the special case that f(n) = 0
• T(n) = a T(n/b) with T(1) = 1
• Gives T(bk) = ak
• But now suppose f(n) = nc for some c
• We can ask which term dominates; the recurrence or
the f(n)?
• Put n = bk then compare
• (for clarity write “logb” as “log_b” )
• “Recurrence term”: ak = ( blog_b(a) )k.
• Note: used the identity that a=b log_b(a)
• “f term” n c = ( bk )c = ( bc )k
• So need to compare c and logb(a)
5
Master Theorem (MT): Cases
T(n) = a T(n/b) + f(n)
Results are split into 3 cases, according to
comparing the growth rate of f(n) to 𝑛^(log 𝑏 (𝑎))
• Case 1: f(n) “grows slower”. Recurrence term
dominates. “Solution ignores f”
• Case 2: f(n) grows same – up to log factors –
“mix of recurrence with a,b, and also the f(n)
term”
• Case 3: f(n) grows faster. “Solution ignores
recurrence terms and a,b”
6
MT: Case 1
T(n) = a T(n/b) + f(n)
f(n) is O( nc ) with c < logb a
Note: it is “<“ not “<=“ and it is a “big-Oh”
Then T(n) is Θ 𝑛^(log 𝑏 𝑎)
That is, T(bk) grows as bk log_b a = ak,
as expected from earlier
7
MT: Case 1: Example
T(n) = 2 T(n/2) + d
a = 2, b=2 so logb(a) = log2(2) = 1
f(n) is O( 1 ) which is O( nc ) with c = 0
and so we have that c < logb(a)
Hence MT gives that T(n) is Θ(n)
8
MT: Case 2
T(n) = a T(n/b) + f(n)
if
f(n) is Θ(𝑛𝑐 (log 𝑛)^𝑘)
with c = logb a and 𝑘 ≥ 0
(Note: it is “c =“ and Big-Theta)
Then T(n) is Θ 𝑛𝑐 (log 𝑛)^(𝑘 + 1)
Note the growth depends on both the recurrence,
a,b, and also depends on f via k.
9
MT: Case 2: Example
T(n) = 2 T(n/2) + 3 n + 5
f(n) is Θ( n (log n)k )
with
c = log2 2 = 1,
and
k=0
Then T(n) is Θ 𝑛 log 𝑛
(Same as merge sort of previous lecture)
10
MT: Case 3
T(n) = a T(n/b) + f(n)
f(n) is Ω(𝑛𝑐 ) with c > logb a
Notice: it is “c > ..” and big-Omega!
And f(n) satisfies the “regularity condition”
a f(n/b) <= k f(n) for some k < 1
Then T(n) is Θ 𝑓(𝑛) .
Growth is dominated by f(n) and so a,b of the
recurrence are not used.
11
MT: Case 3 : Example
T(n) = 2 T(n/2) + n2
f(n) is Ω(𝑛𝑐 ) with c = 2 > logb a = log2 2 =1
Also, f(n) satisfies the “regularity condition”
2 f(n/2) = 2 (n/2)2 <= k f(n) with k=1/2
Then T(n) is Θ 𝑛2
12
Regularity Condition (case 3)
• a f(n/b) <= k f(n) for some k < 1
• Suppose f(n) = d n^c then we need
• a d (n/b)^c <= k d n^c for some k < 1
• a / b^c <= k for some k < 1
• a / b^c < 1
• a < b^c
• Now take log_b of both sides
• Need log_b(a) < c
• Which is already satisfied for case 3 to apply.
• So is not a new condition in this case (needed for more
complex cases)
• (Note: this is included for completeness; usually not relevant)
13
MT Example
• T(n) = 4 T(n/2) + d n with T(1)=1
• a=4, b=2
• so logb a = log2 4 = log2 22 = 2
• Note that f(n) is O(nc) with c=1
• Hence c < 2, and is big-Oh
• Hence is case 1.
• Hence is Θ( 𝑛2 )
• Matches the exact solution in previous lecture.
14
Expectations
• Know and understand the Master Theorem (MT)
• Be able to apply it to examples
• (Do not need to be able to prove the MT itself!)
• May well be asked to solve a recurrence relation
exactly, and then also to solve it using the Master
Theorem
15
