Chapter 2: Divide & Conquer
General Method:
• Divide and Conquer is an algorithm design paradigm based on
multi-branched recursion.
• Divide and conquer algorithm works by recursively breaking down a
problem into two or more sub-problems of the same (or related)
type, until these become simple enough to be solved directly. The
solutions to the sub-problems are then combined to give a solution
to the original problem.
• The correctness of a divide and conquer algorithm is usually proved
by mathematical induction, and its computational cost is often
determined by solving recurrence relations.
Divide and Conquer Strategy:
a problem of size n
(instance)
subproblem 1 subproblem 2
of size n/2 of size n/2
a solution to a solution to
subproblem 1 subproblem 2
a solution to
the original problem
General Method(Algorithm)
• A control abstraction of Divide and Conquer is as given below:
Algorithm DC(P)
{
If P is too small then
return solution of P
else
{
Divide (P) and obtain P1,P2,P3,…Pn
where n>= 1
Apply DC to each subproblem
return combine (DC(P1),DC(P2),….DC(Pn));
}
}
Binary Search
• Binary search algorithm finds the position of a specified input value
(the search "key") within an array sorted by key value.
• For binary search, the array should be arranged in ascending or
descending order.
• In each step, the algorithm compares the search key value with the
middle element of the array. If the values match, then a matching
element has been found and its index, or position, is returned.
• Otherwise, if the search key is less than the middle element, then
the algorithm repeats its action on the sub-array to the left of the
middle element or, if the search key is greater, on the sub-array to
the right. If the remaining array to be searched is empty, then the
key cannot be found in the array and a special "not found"
indication is returned.
Binary Search Representation:
• Let A[m] be the mid element of array A. Then there are 3 conditions
that need to be tested while searching the array which if sorted in
ascending order:
A[0]…A[m-1] A[m] A[m+1…A[n-1]
Search here if Search here if Search here if
key < A[m] key = A[m] key > A[m]
Binary Search Algorithm:
Algorithm Binarysearch(A[0..n-1],key)
//Input: An array A from which the key element is to be searched.
//Output: returns the index of an array element if it is equal to key otherwise
It returns -1.
Low 0
High n-1
While (low < high) do
{
m (low +high)/2 //mid of the array is obtained
If (key = A[m]) then
return m;
Else if (key < A[m]) then
high m-1; // search left sub list
Else
low m+1; // search right sub list
}
return -1; // if element is not in list
Example: (Binary Search)
Analysis of Binary Search Algorithm:
Recurrence Equation for Binary Search:
• T(n)= T(n/2) + 1
Time Required to compare Time Required to compare
left/right sub list key with middle element
• Refer class notes for analysis using Recurrence
Relation.
Finding Minimum and Maximum
Problem Description:
To find minimum and maximum element from a set of n elements
using divide and conquer strategy.
• Algorithm for finding min and max using general method:
MaxMin(a,n,max,min)
{
max := min := a[1];
for i := 2 to n do
{
if(a[i] > max) then max := a[i];
if(a[i] > min) then min := a[i];
}
}
• This algorithm will require 2(n-1) comparisons.
Finding Minimum and Maximum
• Algorithm for finding min and max using divide and conquer :
MaxMin(i, j, max, min)
// a[1:n] is a global array. Parameters i (low) and j(high) are integers
{
if (i=j) then max := min := a[i]; //Small(P)
else if (i=j-1) then // Another case of Small(P)
{
if (a[i] < a[j]) then max := a[j]; min := a[i];
else max := a[i]; min := a[j];
}
else
{
// if P is not small, divide P into sub-problems.
// Find where to split the set.
mid := ( i + j )/2;
// Solve the sub-problems.
MaxMin( i, mid, max, min );
MaxMin( mid+1, j, max1, min1 );
// Combine the solutions.
if (max < max1) then max := max1;
if (min > min1) then min := min1;
}
}
Example:
• Show algorithmic execution of following
example: (Refer class notes)
50 40 -5 -9 45 90 65 25 75
Analysis:
• Recurrence relation for finding min and max algorithm:
T (n) = 0 when n=1
T (n) = 1 when n=2
T (n) = T (n/2) + T (n/2) + 2 when n> 2
Time required to execute recursive calls for
each half divided sub list. Time required to
compare min
and max
• Refer class notes for analysis using Recurrence relation
Merge Sort
• Merge sort is a divide and conquer algorithm that was invented by John von
Neumann.
• First divide the list into the smallest unit (1 element), then compare each
element with the adjacent list to sort and merge the two adjacent lists. Finally
all the elements are sorted and merged.
• 3 Steps to sort A[p .. r]:
1. Divide Step
If a given array A has zero or one element, simply return; it is already
sorted. Otherwise, split A[p .. r] into two subarrays A[p.. q] and A[q +
1 .. r], each containing about half of the elements of A[p .. r]. That
is, q is the halfway point of A[p .. r].
2. Conquer Step
Conquer by recursively sorting the two subarrays A[p .. q] and
A[q + 1 .. r].
3. Combine Step
Combine the elements back in A[p .. r] by merging the two sorted
subarrays A[p .. q] and A[q + 1 .. r] into a sorted sequence. To
accomplish this step, we will define a procedure MERGE (A, p, q, r).
Merge Sort Algorithm
• To sort the entire sequence A[1 .. n], make the initial call to the
procedure MERGE-SORT (A, 1, n).
• MERGE-SORT (A, p, r)
1. IF p < r // Check for base case
2. q = [(p + r)/2] // Divide step
3. MERGE-SORT(A, p, q) //conquer step
4. MERGE-SORT (A, q + 1, r) //conquer step
5. MERGE (A, p, q, r) //merge step
Merge Sort Algorithm
• The pseudo code of the MERGE procedure is as follow:
• MERGE (A, p, q, r )
1. n1 ← q − p + 1
2. n2 ← r − q
3. Create arrays L[1 . . n1 + 1] and R[1 . . n2 + 1]
4. FOR i ← 1 TO n1
5. DO L[i+ ← A*p + i − 1+
6. FOR j ← 1 TO n2
7. DO R[j+ ← A*q + j ]
8. L[n1 + 1+ ← ∞
9. R[n2 + 1+ ← ∞
10. i ← 1
11. j ← 1
12. FOR k ← p TO r
13. DO IF L[i + ≤ R* j]
14. THEN A[k+ ← L*i]
15. i←i+1
16. ELSE A*k+ ← R*j+
17. j←j+1
Example:
Analysis:
• Recurrence relation for merge sort algorithm:
T (n) = 2 T (n/2) + n + 1
Time required to conquer by Time required to divide the
recursively sorting 2 sub arrays array
Time required to merge 2
sorted sub arrays
• Refer class notes for analysis of merge sort
algorithm by recurrence relation.
Quick Sort
• Quicksort is a divide and conquer algorithm. Quicksort first divides a
large array into two smaller sub-arrays: the low elements and the
high elements. Quicksort can then recursively sort the sub-arrays.
• It is also sometimes called partition-exchange sort (since it uses a
pivot element to partition the array)
• The steps are:
1. Pick an element, called a pivot, from the array.
2. Reorder the array so that all elements with values less than the
pivot come before the pivot, while all elements with values
greater than the pivot come after it (equal values can go either
way). After this partitioning, the pivot is in its final position. This is
called the partition operation.
3. Recursively apply the above steps to the sub-array of elements
with smaller values and separately to the sub-array of elements
with greater values.
Quick-Sort Algorithm
Quicksort(A,p,r) //where p=low, r=high
If p< r
then q <- Partition(A,p,r)
Quicksort(A,p,q-1)
Quicksort(A,q+1,r)
Quick Sort Algorithm
Partition( A, p, r)
x A[r] // taking last element as the pivot
i p-1
For j p to r-1
do if A[j] <= x
then i i+1
exchange A[i] & A[j]
Exchange A[i+1] & A[r]
Return i+1
Example:
Analysis:
• Recurrence relation for quick sort algorithm:
T (n) = 2 T (n/2) + n
Time required to conquer by Time required to partition the
recursively sorting 2 sub arrays array
• Refer class notes for analysis of quick sort
algorithm by recurrence relation.
Strassen’s Matrix Multiplication
Overview:
Given two square matrices A and B of size n x n each, find their multiplication matrix.
Naive Method:
Following is a simple way to multiply two matrices.
Matrix_Mul(A,B)
{
for (int i = 0; i < N; i++)
for (int j = 0; j < N; j++)
C[i][j] = 0;
for (int k = 0; k < N; k++)
C[i][j] += A[i][k]*B[k][j];
Return C
}
Time Complexity of above method is O(N3).
Strassen’s Matrix Multiplication
• Therefore, Strassen’s proposed method is designed to reduce on the
number of matrix multiplications which uses divide & conquer strategy.
• Steps to solve:
1. Divide the matrices A & B into n/2 X n/2 submatrices
2. Create 10 matrices S1….S10 each of which is n/2 X n/2 and is the
sum/difference of 2 matrices created in step 1.
S1= B12-B22
S2= A11+A12
S3= A21+A22
S4= B21-B11
This should
S5= A11+A22 approximately
S6= B11+B22 take (n2 )time
S7= A12-A22
S8= B21+B22
S9= A11-A21
S10= B11+B12
Strassen’s Matrix Multiplication
Steps to solve:
3. Using the submatrices created in 1 and 2 compute 7 matrix
products P1….P7
P1= A11 X S1
P2= S2 X B22
P3= S3 X B11
P4= A22 X S4 This should
approximately
P5= S5 X S6 take
P6= S7 X S8 7T(n/2)time
P7= S9 X S10
4. Compute the desired C11,C12,C21 & C22 by adding/subtracting various
combinations as shown in the following figure.
Analysis
Recurrence Equation for Strassen’s Matrix Multiplication:
T(n)= 7 T(n/2) + n2
Solving this further using Masters Method we get (refer class notes for
derivation)
T(n) = θ(n^2.81) which is less than (n^3)
