Recap
Functions and Recurrences
Methods to solve Recurrences
In the previous lecture we looked at the problem of sorting and an
algorithm which can be used for sorting.
We also analysed the worst case running time (clock cycles) needed
to solve the problem. Every operation is free except the basic
operation required to compare to elements which is assumed to take
1 clock cycle irrespective of the size/nature of the elements.
The insertion sort takes O(n2 ) to correctly sort an input consisting
of n elements.
Recap
Functions and Recurrences
Methods to solve Recurrences
We arrived at the conclusion that the recurrence relation
T (n) = T (n − 1) + (n − 1) solves to O(n2 )
using some guess work and induction, but guessing cannot be used
at all times.
Even though guessing can be quite useful in various circumstances,
one needs a lot of experience dealing with such recurrence relations,
in order to guess the outcome.
In this lecture we shall discuss few techniques which are used to
solve recurrences.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Recursion Tree method
All recurrences require a base case to fall back on. These base cases
are usually assumed to be of the form T (k) = k 0 where k and k 0
are constants. By constants we refer to finite numbers. In most
cases if we are dealing with a constant size input, the problem is
considered to be solvable in constant time.
So most of the times, the base cases are usually not given explicitly.
In other words, for most problems that we will be dealing with it is
assumed that
T (O(1)) = O(1)
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
This method does not require us to guess the answer, but it may
require a little algebra.
The idea is to expand (iterate) the recurrence and express it as a
summation of terms dependent only on n and the initial conditions.
Techniques for evaluating summations can then be used to provide
bounds on the solution.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Let’s take the following recurrence
T (n) = 2 ∗ (T (n/2)) + 7n + 34
This could be interpreted as
Solving the problem for an instance of size n, involves solving two
instances of the same problem each of size n/2 and an additional
work of 7n + 34 is required just to put together the two smaller
solutions to get the solution for the original problem.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
n
T (n) = 2T + [7n + 34] (1)
2
n n
= 2[2T 2 + 7 + 34] + [7n + 34]
2 2
n
= 4T ( 2 ) + [7n ∗ (1 + 1) + 34(1 + 2)] (2)
2
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
!
n n
T (n) = 4 2T ( 3 ) + 7( ) + 34 + [7n(1 + 1) + 34(1 + 2)]
2 4
!
n
= 8T + [7n(1 + 1 + 1) + 34(1 + 2 + 4)]
23
!
n
= 23 T + [7n(3) + 34(1 + 2 + 22 )]
23
!
n
= 23 T + [7n(3) + 34(23 − 1)] (3)
23
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
In general, for an integer i, we can write the recursion as
n
T (n) = 2i T ( ) + 7n(i) + 34(2i − 1)
2i
Since the base case was not explicitly mentioned we assume that
T (1) = c for some constant c.
We also know that 2lg n = n, where lg denotes log to the base two.
By setting i = lg n, we can say that
T (n) = nT (1) + 7n lg n + 34(2lg n − 1)
= cn + 7n(lg n) + 34(n − 1)
= 7n(lg n) + (34 + c)n − 34
= O(n lg n) (4)
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Substitution method
Sometimes we may have to resort to guessing the recurrence, simply
because we don’t know how to use substitution method.
In such a scenario, we may guess the solution and try to confirm its
correctness by using induction.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
The Substitution Method:-
I guessing the form of the solution
I use mathematical induction to find the appropriate constants
and show that the solution works.
Quite powerful method, but it can be applied only in cases when it
is easy to guess the form of the answer.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Lets look at the following recursion
T (n) = T (n/5) + T (7n/10) + 8n
It is not clear how to use the previously described “Recursion Tree”
method to solve this recursion.
When everything fails, there is no harm in guessing the solution.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Lets look at the following recursion
T (n) = T (n/5) + T (7n/10) + 8n
It is not clear how to use the previously described “Recursion Tree”
method to solve this recursion.
When everything fails, there is no harm in guessing the solution.
Lets guess that the solution is O(n2 ).
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Lets look at the following recursion
T (n) = T (n/5) + T (7n/10) + 8n
It is not clear how to use the previously described “Recursion Tree”
method to solve this recursion.
When everything fails, there is no harm in guessing the solution.
Lets guess that the solution is O(n2 ).
We seek to prove that T (n) ≤ cn2 for some constant c, and for all
values of n > n0 where n0 is some threshold which we shall try to
find.
We are allowed to assume that T (1) ≤ d for some constant d.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Lets assume that for inputs of size less than n − 1, this argument is
true.
T (n) = T (n/5) + T (7n/10) + 8n
≤ c(n/5)2 + c(7n/10)2 + 8n
= cn2 (1/25 + 49/100) + 8n
= cn2 (53/100) + 8n
≤ cn2 (5)
800
As long as we n > n0 = 47c , we can claim that T (n) ≤ cn2
Thus, by induction we have figured out that T (n) = O(n2 )
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
One of the drawbacks of this method is that if we underestimate the
solution, we won’t be able to solve the recurrence correctly.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
One of the drawbacks of this method is that if we underestimate the
solution, we won’t be able to solve the recurrence correctly.
We may also overestimate the solution.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
One of the drawbacks of this method is that if we underestimate the
solution, we won’t be able to solve the recurrence correctly.
We may also overestimate the solution.
Let us check another improved guess for the same recursion.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
One of the drawbacks of this method is that if we underestimate the
solution, we won’t be able to solve the recurrence correctly.
We may also overestimate the solution.
Let us check another improved guess for the same recursion.
Suppose we wish to prove that T (n) ≤ cn for value of c.
We assume that for all inputs of size x < n, T (x) ≤ cx.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
T (n) = T (n/5) + T (7n/10) + 8n
≤ c(n/5) + c(7n/10) + 8n
= n(9c/10 + 8)
We can say that T (n) ≤ cn, if c ≥ 9c/10 + 8.
Thus, if we choose c to be at least 80, our claim would be true.
Thus, there exists a constant c, such that T (n) ≤ cn, for all input
sizes beyond a threshold.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
We do not concern ourselves with the threshold value. We know
that T (1) = d and since we can choose all values of c > 80, we can
choose d to be any value greater than or equal to 80.
Usually, we do not concern ourselves with the value of d, because it
can be chosen to be of finite but arbitrarily large value.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
The master method is useful if the recurrence is of the following
form.
Let a ≥ 1 and b > 1 be constants, let f (n) be an asymptotically
positive function, and let T (n) be defined on the nonnegative
integers by the recurrence
T (n) = aT (n/b) + f (n)
where we interpret n/b to mean either dn/be or bn/bc.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Then T (n) can be bounded asymptotically as follows.
I Case 1:
If f (n) = O(nlogb a− ) for some > 0, then T (n) = Θ(nlogb a )
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Then T (n) can be bounded asymptotically as follows.
I Case 1:
If f (n) = O(nlogb a− ) for some > 0, then T (n) = Θ(nlogb a )
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Then T (n) can be bounded asymptotically as follows.
I Case 1:
If f (n) = O(nlogb a− ) for some > 0, then T (n) = Θ(nlogb a )
I Case 2:
If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a lg n)
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Then T (n) can be bounded asymptotically as follows.
I Case 1:
If f (n) = O(nlogb a− ) for some > 0, then T (n) = Θ(nlogb a )
I Case 2:
If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a lg n)
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Then T (n) can be bounded asymptotically as follows.
I Case 1:
If f (n) = O(nlogb a− ) for some > 0, then T (n) = Θ(nlogb a )
I Case 2:
If f (n) = Θ(nlogb a ) then T (n) = Θ(nlogb a lg n)
I Case 3:
If f (n) = Ω(nlogb a+ ), for some > 0 and if a ∗ f ( bn ) ≤ c ∗ f (n)
for some constant c < 1 and all sufficiently large n, then
T (n) = Θ(f (n)).
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Suppose we are given the following recurrence
T (n) = 8T (n/2) + 1000n2
First we verify that the given recurrence is in the desired format i.e
T (n) = aT (n/b) + f (n)
From this we learn that a = 8, b = 2 and f (n) = 1000n2
lgb a = lg2 8 = 3
Here, f (n) = O(nlgb a− ) = O(n3− ), ∀ ∈ [0, 1]
Thus, using Case 1 of master method we solve the recurrence as
T (n) = Θ(nlgb a ) = Θ(n3 )
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Suppose we are given the following recurrence
T (n) = 2T (n/2) + 1000n
First we verify that the given recurrence is in the desired format i.e
T (n) = aT (n/b) + f (n)
From this we learn that a = 2, b = 2 and f (n) = 1000n
lgb a = lg2 2 = 1
Here, f (n) = Θ(nlgb a ) = Θ(n)
Thus, using Case 2 of master method we solve the recurrence as
T (n) = Θ(nlgb a lg n) = Θ(n lg n)
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Suppose we are given the following recurrence
T (n) = 2T (n/2) + n2
We learn that a = 2, b = 2 and f (n) = n2
lgb a = lg2 2 = 1
• Here, f (n) = Ω(nlgb a+ ) = Ω(n1+ ), for say 0.5.
• Moreover, a ∗ f (n/b) = 2 ∗ (n/2)2 = n2 /2 ≤ 1 ∗ (f (n)) (where c
is set to be 1)
Thus, using Case 3 of master method we solve the recurrence as
T (n) = Θ(f (n)) = Θ(n2 )
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
One advantage of the master method is that it gives a Θ relation
while the other two methods give a O relation.
So the master method gives asymptotically tight bounds of the
recurrence, whereas the other two methods give crude upper bounds
to the recurrence.
Curious readers may look at the proof in Chapter 4 of CLRS. The
proof is not as important as knowing when and how to apply the
master theorem.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Even the master method has few limitations. Sometimes the
functions may not fall in either of the three categories.
For example,
T (n) = 2n T (n/2) + nn
This recurrence cannot be solved using master theorem, because the
value of a (i.e 2n ) happens to be dependent on n and hence its not
a constant.
Few Groundrules
Recap
Recursion Tree/Iteration Method
Functions and Recurrences
Substitution method
Methods to solve Recurrences
Master Method
Additional optional materials to look at if you have spare time
I Akra Bazzi Method (Advanced techniques for solving
recurrences) only for educational purposes. Not to be used in
assignments/quizzes/exams.
I Check CLRS “Chapter 4” for more solved examples.
