Lecture 02

Divide and Conquer (BinarySearch &

Mergesort)
CSE373: Design and Analysis of Algorithms
 A motivating Example of D&C Algorithm
Binary Search (recursive)
// Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1
Algorithm BinarySearch(A, first, last, x)
    if last ≥ first then
        mid = first + (last - first)/2
 
// If the element is present at the middle itself
        if A[mid] = x then
             return mid
 
// If element is smaller than mid, then it can only be present in left sub-array
else if A[mid] > x then
        return BinarySearch(A, first, mid-1, x)
 
// Otherwise the element can only be present in the right sub-array
else
Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

• Step 1
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first mid last

=(0+14)/2

• Step 1
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first mid last

=(0+14)/2
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

• Step 2
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first mid last

=(8+14)/2

• Step 2
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first mid last

=(8+14)/2

• Step 2
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

• Step 3
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

mid
=(8+10)/2
• Step 3
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

mid
=(8+10)/2
• Step 3
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

• Step 4
Retrieving an Item from Sorted List
• Find 84

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

first last

mid
=(10+10)/2
• Step 4
• 84 found at the midpoint
 Binary Search (recursive) Algorithm
// Returns location of x in the sorted array A[first..last] if x is in A, otherwise returns -1
Algorithm BinarySearch(A, p, q, x)
    if last ≥ first then
        mid  (p+q)/2 Time: Θ(lg n), why?
 
// If the element is present at the middle itself
        if A[mid] = x then
             return mid
 
// If element is smaller than mid, then it can only be present in left sub-array
if A[mid] > x then
        return BinarySearch(A, p, mid-1, x)
 
// Otherwise the element can only be present in the right sub-array
else
return BinarySearch(A, mid+1, q, x)
    
return -1// We reach here when element is not present in A

Initial call: BinarySearch(A,1,n,key) where key is an user input which is to be sought in A

 Divide and Conquer (D&C)
• In general, has 3 steps:
– Divide the problem into independent sub-
problems that are similar to the original but
smaller in size
– Conquer the sub-problems by solving them
recursively. If they are small enough, just solve
them in a straightforward manner.
– Combine the solutions to create a solution to the
original problem (this step may be empty)
D&C Algorithm Example: Binary Search
Searching Problem: Search for item in a sorted sequence A of n elements

Divide: Divide the n-element input array into two subarray of ≈ n/2 elements
each:
m  (p+q)/2

Conquer: Search either of the subarrays recursively by calling BinarySearch on

the appropriate subarray:
if A[m] > x then
        return BinarySearch(A, p, m-1, x)
else
return BinarySearch(A, m+1, q, x)

Combine: Nothing to be done

D&C Example: Merge Sort (Section 2.3)
Sorting Problem: Sort a sequence A of n elements into non-decreasing order:
MergeSort (A[p..r]) //sort A[p..r]

Divide: Divide the n-element input array into two subarray of ≈ n/2 elements
each [easy]:
q  (p+r)/2

Conquer: Sort the two subsequences recursively by calling merge sort on

each subsequence [easy]:
MergeSort (A[p .. q]) // A[p .. q] becomes sorted after this call
MergeSort (A[q+1 .. r]) //A[q+1..r] becomes sorted after this call

Combine: Merge the two sorted subsequences to produce the sorted

sequence [how?]
Merging two sorted subsequeces

6 18 56 62 1 9 15 43
Merging two sorted subsequeces
Unsorted

6 18 56 62 1 9 15 43

Sorted Sorted
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43 56 62
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
1 6 9 15 18 43 56 62
Merging

1 6 9 15 18 43 56 62
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
1 6 9 15 18 43 56 62
Merging two sorted subsequeces
Merge(A, p, q, r)
1 n1  q – p + 1
2 n2  r – q
Input: Array containing
3 for i  1 to n1
sorted subarrays A[p..q] and
4 do L[i]  A[p + i – 1]
A[q+1..r].
5 for j  1 to n2
6 do R[j]  A[q + j] Output: Merged sorted
7 L[n1+1]   subarray in A[p..r].
8 R[n2+1]  
9 i1
10 j1
11 for k p to r Sentinels, to avoid having to
12 do if L[i]  R[j] check if either subarray is
13 then A[k]  L[i] fully copied at each step.
14 ii+1
15 else A[k]  R[j]
16 jj+1
Time complexity of Merge
Merge(A, p, q, r) //Let r-p+1 = n
1 n1  q – p + 1//Θ(1)
2 n2  r – q //Θ(1)
Input: Array containing
3 for i  1 to n1 //Θ(q-p+1)
sorted subarrays A[p..q] and
4 do L[i]  A[p + i – 1]
A[q+1..r].
5 for j  1 to n2 //Θ(r-q)
6 do R[j]  A[q + j] Output: Merged sorted
7 L[n1+1]   subarray in A[p..r].
8 R[n2+1]  
9 i1
10 j1
11 for k p to r //Θ(r-p+1) = Θ(n)
12 do if L[i]  R[j]
13 then A[k]  L[i]
14 ii+1
15 else A[k]  R[j]
16 jj+1
//Total time: Θ(n)
Merge Sort (recursive/D&C version)
MergeSort (A, p, r) // sort A[p..r] via merge sort
1 if p < r
2 then q  (p+r)/2 //divide
3 MergeSort (A, p, q) //conquer
4 MergeSort (A, q+1, r) //conquer
5 Merge (A, p, q, r) //combine: merge A[p..q] with A[q+1..r]

Initial Call: MergeSort(A, 1, n)

98 23 45 14 6 67 33 42
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

23 98 14 45

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6 67

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

Merge
98 23 45 14 6 67 33 42

6 14 23 33 42 45 67 98
Analysis of Merge Sort
Statement Cost
MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n) [let]
1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r)
Analysis of Merge Sort
Statement Cost (time)
MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n), to sort n elements
1 if p < r (1)
2 then q  (p+r)/2 //q ≈ n/2 (1)
3 MergeSort (A, p, q) T(n/2), to sort n/2 elements
4 MergeSort (A, q+1, r) T(n/2), to sort n/2 elements
5 Merge (A, p, q, r) (n)

So T(n) = (1) ; when n = 1, and 2T(n/2) + (n) + 2(1) ;

when n > 1
It’s a recurrence relation. Equivalent recurrence relation:
T(n) = (1) ; when n = 1, and 2T(n/2) + (n) ;
when n > 1
Equivalent recurrence relation:
T(n) = c if n = 1
= 2T(n/2) + cn if n > 1
Recurrence Relations (RR)
Equation or an inequality that characterizes a function by
its values on smaller inputs.
Recurrence relations arise when we analyze the running
time of iterative or recursive algorithms.
Ex: Divide and Conquer algorithms typically have r.r. of the form:
T(n) = (1) if n  c
T(n) = a T(n/b) + D(n) otherwise
Mthods to solve recurrence relations
•Substitution Method.
•Recursion-tree Method.
Substitution Method
Illustration of guessing solution of a r.r. (representing time
complexity of MergeSort) via substitution method:
T(n) = 2T(n/2) + cn
= 2(2T(n/4)+cn/2) + cn = 22T(n/22) + 2cn
= 22(2T(n/8)+cn/4) + 2cn = 23T(n/23) + 3cn
…
= 2kT(n/2k) + kcn [guess the pattern from previous equations]
Let 2k = n (so that we get T(n/2k) = T(1) which is known to us)
⸫ T(n) = n T(n/n) + (lg n) cn
= n T(1) + (lg n) cn
= n T(1) + cn lg n
= cn + (lg n) cn which is (n lg n)
Recursion-tree Method
• Recursion trees can also be used to solve r.r.
Recursion Trees
•Show successive expansions of recurrences using trees.
•Keep track of the time spent on the subproblems of a divide and
conquer algorithm.
•Help organize the algebraic bookkeeping necessary to solve a
recurrence.
Recursion Tree – Example
Running time of Merge Sort:
T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
Rewrite the recurrence as
T(n) = c if n = 1
T(n) = 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort
For the original problem, Each of the size n/2 problems has
we have a cost of cn, plus a cost of cn/2 plus two
two subproblems each of subproblems, each costing T(n/4).
size (n/2) and running time T(n/2) = 2T(n/4) + cn/2
T(n/2). Þ cn/2: parent node with
T(n) = 2T(n/2) + cn 2 children: each T(n/4)
Þ cn: parent node with cn
2 children: each T(n/2)
cn
Cost of divide
and merge.
cn/2 cn/2

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn cn

cn/2 cn/2 cn

lg n

cn/4 cn/4 cn/4 cn/4 cn

c c c c c c cn
Total : cnlgn+cn
Counting Inversions Problem
• Given two ranked list of items, how can you compare
these two lists?
• Application: Recommendation systems try to match your
preferences (for books, movies, restaurants, etc.) with
those of other people in the internet
• Idea: represent one ranked list by <1,2, …, n> and another
by a permutation of the first list. Then count the number of
inversions (i.e. out-of-order pairs in the second list.
Merging & Counting Inversions
MergeAndCount(A, p, q, r)
1 n1  q – p + 1
2 n2  r – q
3 for i  1 to n1
Input: Array containing
4 do L[i]  A[p + i – 1] sorted subarrays A[p..q] and
5 for j  1 to n2 A[q+1..r].
do R[j]  A[q + j]
6
Output: Merged sorted
7 L[n1+1]  
subarray in A[p..r].
8 R[n2+1]  
9 i1
10 j1
11 cnt  0
12 for k p to r
13 do if L[i]  R[j]
14 then A[k]  L[i]
15 ii+1
16 else A[k]  R[j]
17 jj+1
18 cnt  cnt + n1-i+1
19 return cnt
Counting Inversions
Statement Cost
CountInversions(A, p, r)
1 if p < r
2 then q  (p+r)/2
3 x  CountInversions (A, p, q)
4 y  CountInversions(A, q+1, r)
5 z  MergeAndCount(A, p, q, r)
6 return x+y+z

So T(n) = (1) when n = 1, and 2T(n/2) + (n)

when n > 1