Divide
Divide and
and Conquer
Conquer
(Merge
(Merge Sort)
Sort)
David A. Plaisted
Divide and Conquer
Recursive in structure
Divide the problem into sub-problems that are
similar to the original but smaller in size
Conquer the sub-problems by solving them
recursively. If they are small enough, just solve
them in a straightforward manner.
Combine the solutions to create a solution to
the original problem
An Example: Merge Sort
Sorting Problem: Sort a sequence of n elements into
non-decreasing order.
Divide: Divide the n-element sequence to be
sorted into two subsequences of n/2 elements each
Conquer: Sort the two subsequences recursively
using merge sort.
Combine: Merge the two sorted subsequences to
produce the sorted answer.
Merge Sort Example
18 26 32 6 43 15 9
1 22 26 19 55 37 43 99 2
18 26 32 6 43 15 9
22 26 19 55 37 43 99 2
18 26 32 6
43 15 9
22 26 19 55
18 26 32 6
43 15
22 26 19 55 37 43 99 2
18 26 32
43 15
22 26 19 55 37 43 99
37 43 99 2
Merge Sort Example
Original Sequence
Sorted Sequence
18 26 32 6 43 15 9
18 26 32 6
43 15 9
6 18 26 32
18 26 32 6
43 15
18 26
15 43
18 26 32
43 15
18 26 32
43 15
18 26 32
43 15
15 18 26 32 43
6 32
6
15 43
43
Merge-Sort (A, p, r)
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers
MergeSort (A, p, r) // sort A[p..r] by divide & conquer
1 if p < r
2
then q (p+r)/2
3
MergeSort (A, p, q)
4
MergeSort (A, q+1, r)
5
Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]
Initial Call: MergeSort(A, 1, n)
Procedure Merge
Merge(A, p, q, r)
1 n1 q p + 1
2 n2 r q
3
for i 1 to n1
4
do L[i] A[p + i 1]
5
for j 1 to n2
6
do R[j] A[q + j]
7
L[n1+1]
8
R[n2+1]
9
i1
10 j 1
11 for k p to r
12
do if L[i] R[j]
13
then A[k] L[i]
14
ii+1
15
else A[k] R[j]
16
jj+1
Input: Array containing
sorted subarrays A[p..q]
and A[q+1..r].
Output: Merged sorted
subarray in A[p..r].
Merge Example
A
6
i
61
86 26
1 32
9 42 43
8 32
9 26
k k
8 26 32
9 42 43
Analysis of Merge Sort
Running time T(n) of Merge Sort:
Divide: computing the middle takes (1)
Conquer: solving 2 subproblems takes 2T(n/2)
Combine: merging n elements takes (n)
Total:
T(n) = (1)
if n = 1
T(n) = 2T(n/2) + (n) if n > 1
T(n) = (n lg n)
Recurrences
Recurrences II
Recurrence Relations
Equation or an inequality that characterizes a
function by its values on smaller inputs.
Solution Methods (Chapter 4)
Substitution Method.
Recursion-tree Method.
Master Method.
Recurrence relations arise when we analyze the
running time of recursive algorithms.
Ex: Divide and Conquer.
T(n) = (1)
T(n) = a T(n/b) + D(n) + C(n)
if n c
otherwise
Recursion-tree Method
Recursion Trees
Show successive expansions of recurrences using
trees.
Keep track of the time spent on the subproblems of a
divide and conquer algorithm.
Help organize the algebraic bookkeeping necessary
to solve a recurrence.
Recursion Tree Example
Running time of Merge Sort:
T(n) = (1)
T(n) = 2T(n/2) + (n)
if n = 1
if n > 1
Rewrite the recurrence as
T(n) = c
if n = 1
T(n) = 2T(n/2) + cn
if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort
For the original problem,
we have a cost of cn,
plus two subproblems
each of size (n/2) and
running time T(n/2).
cn
Each of the size n/2 problems
has a cost of cn/2 plus two
subproblems, each costing
T(n/4).
cn
Cost of divide and
merge.
cn/2
T(n/2)
cn/2
T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
cn/2
cn/2
cn
cn
lg n
cn/4
cn/4 cn/4
cn/4
cn
c c c
cn
: cnlgn+cn
Total
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
Each level has total cost cn.
Each time we go down one level,
the number of subproblems doubles,
but the cost per subproblem halves
cn/2
cn/2
cost per level remains the same.
There are lg n + 1 levels, height is
lg n. (Assuming n is a power of 2.)
cn/4
cn/4 cn/4
cn/4
Total cost = sum of costs at each
level = (lg n + 1)cn = cnlgn + cn =
(n lgn).
c
c c c
The Master Method
Based on the Master theorem.
Cookbook approach for solving recurrences
of the form
T(n) = aT(n/b) + f(n)
a 1, b > 1 are constants.
f(n) is asymptotically positive.
Requires memorization of three cases.
The Master Theorem
Theorem
Theorem4.1
4.1
Let
Letaa 11 and
and bb >> 11be
beconstants,
constants,let
letf(n)
f(n)be
beaafunction,
function,and
and
Let
LetT(n)
T(n)be
bedefined
definedon
onnonnegative
nonnegativeintegers
integersby
bythe
therecurrence
recurrence
T(n)
T(n)==aT(n/b)
aT(n/b)++f(n),
f(n),where
wherewe
wecan
canreplace
replacen/b
n/bby
by n/b
n/b or
or n/b
n/b ..
T(n)
T(n)can
canbe
bebounded
boundedasymptotically
asymptoticallyin
inthree
threecases:
cases:
log
loga
a) for some constant > 0, then T(n) = (nlog
logaa).
1.1. IfIf f(n)
=
O(n
f(n) = O(n
) for some constant > 0, then T(n) = (n ).
log a
2.2. IfIf f(n)
f(n)==
(n
(nlog a),),then
thenT(n)
T(n)==(
(nnloglogaalglgn).
n).
log a+
3.3. IfIf f(n)
f(n)==(n
(nlog a+)) for
forsome
someconstant
constant>>0,
0,
and
andif,
if,for
forsome
someconstant
constantcc<<11and
andall
allsufficiently
sufficientlylarge
largen,n,
we
wehave
haveaf(n/b)
af(n/b)ccf(n),
f(n),then
thenT(n)
T(n)==
(f(n)).
(f(n)).
b
Well return to recurrences as we need them
