MODULE6
RANDOMIZED ALGORITHMS
What are randomized algorithms?
Deterministic Algorithms :
The goal is to prove that the algorithm solves the problems correctly always
and quickly
The number of steps is preferably polynomial in terms of the size of the input
Input can be anything (random)
Randomized Algorithms: In addition to random inputs, algorithm
Takes a source of random numbers (e.g. Hiring problem)
Makes Random choices during execution (e.g. Randomized quick sort)
Behaviour can vary even on a fixed input
What are randomized algorithms?
Input X Output Y(X)
Deterministic Algorithm
Input X
Output Y(X,
Random bits R Randomized Algorithm R)
• Repeated runs of data with fixed input data will not in generally produce the same results
• Outcome depends not only in X but also R
Features of randomized algorithms
• Randomized algorithms are known for their simplicity. These algorithms are very simple to
understand and implement.
• Any deterministic algorithm can easily be converted to a randomized algorithm.
• Randomized algorithms are very efficient.
• They utilize little execution time and space compared to any deterministic algorithms.
• Easy to analyze and reason
• Exploits modern-day computing platforms
• Reliability is an important issue in many critical applications, as not all randomized algorithms
always give correct answers.
• In addition, many randomized algorithms may not terminate.
Types of Random Algorithms
Las Vegas randomized algorithms
For a given Input X, Output of algorithm is always correct
Running time is random
Analyse the expected running time
E.g. Randomized quick sort
Monte Carlo randomized algorithms
For given Input X, Output is random; correct with some probability
Running time is deterministic
Analyse Probability of correct output in addition to running time
Karger’s algorithm (Global minimum cut problem)
Deterministic Vs Las Vegas Deterministic Vs Monte Carlo
Algo Search Repeat (A) Algo Search (A, a)
{ {
for(i=0; i<n-1; i++) for(i=0; i<n-1; i++)
for(j=i+1; j<n; j++) if (A[i]== a)
if (A[i]== A[i]) return true
return true }
}
Algo-Las-Search Repeat (A) Algo-MC-Search (A, a, x)
{ {
while (true) do for(i=0; i<=x1; i++)
i=random() mod n+1 {
j=random() mod n+1 j=rand()mod n+1
If ((i≠j) and (A[i]=A[j] if(A[j]==a)
return true return true
} }
Deterministic Quicksort - Recap
Pivo
O(n) t
T(n1 )
T(n2 )
O(n)
T(n) = T(n1) + T(n2) +
O(n)
A is the array,
p is the starting index of the array and
r is the last index of the array
Quicksort -
Recap
Tn = T(n1) + T(n2) + O(n)
Best Case Worst Case
• Partition() function divides the array • Partition() function divides array into
into two equal parts two parts, one (n-1) and other zero
• Pivot element first or last • Input is in sorted order either
• T(n) = T(n/2) + T(n/2) + cn ascending order or descending
• O(n log n) order
• Pivot element first or last
• T(n) = T(n-1) + cn
• O(n2)
Why Randomize
Worst case occurs when we repeatedly choose the smallest number or the largest
number as pivot
A good pivot separates the array into roughly equal parts
If the pivot gives a [n/10, 9n/10] – split, we get a recurrence
T(n) = T(n/10) + T(9n/10) + cn
Even this gives us O(n log n) comparisons
A random pivot is likely work probability 0.8
Randomized Quicksort
Pivot
A is the array, p is starting index of the array and r is the last index of array
Picking a
random element
from the list
The Hiring Problem
You want hire a office assistant (secretary) from an employment agency
You are always committed to employ best candidate irrespective of cost of
replacement
There in total n candidates and you are going to eventually interview all of
them
You interview the candidate, if better than current candidate, you replace
the
candidate
The Hiring Problem
The cost is little different than running time complexity but similar
Cost of interviewing (Ci) and Cost of Hiring (Ch)
Cost of Hiring > Cost of interviewing
Say n people are interviewed and m people are hired
The total cost associated with algorithm is O( Ci n + Ch m)
Irrespective of how many people we hire, we always interview n candidates and
incur interviewing cost Ci n .
So we concentrate on Hiring cost Ch m
This quantity varies with each run of run of the algorithm
The Hiring Problem
Best Case: Best candidate comes first. We end up hiring only once. Hiring cost (Ch)
Worst case: Candidates come in increasing order of quality. we actually hire every
candidate we interview. We hire n times, hiring cost O(Ch n)
We don’t have control on the order of quality of candidates
Average Case is more useful; but to perform we need to perform probabilistic
analysis
For probabilistic Analysis, we need to know the distribution of inputs; Often
distribution of inputs is un-known
Nevertheless, we can use probability and analysis as tool for algorithm
design by
making the algorithm do some kind of randomization of inputs
In Hiring Problem, we ask agency to give the n number of candidates upfront and we
call candidates at random
Hiring Problem -
Randomized
Hiring problem is now Randomized problem; algorithm itself makes random choices
Expected Running time vs Average case running time
Hiring Problem – Expected Cost (Running Time)
Depends on expected number of times we hire a new office assistant
X – number of times we hire the new office assistant
E(X) – Expected value of a random variable X
This calculation is cumbersome
(1)(P(1 person selected) ) + (2)(P(2 persons selected) ) + ….
Instead of this we use Indicator Variable Xi
Hiring Problem – Expected Cost
1/(i)
Candidate i is hired exactly when the
candidate i is better than each of
Using Indicator Variable candidates from 1 through i-1 interviews
By linearity of expectation
Harmonic Series
Hiring cost O(Ch x m) O(Ch x ln n)
where m = number of candidates hired n = total number of candidates
m= ln n
Global Minimum Cut
A Cut of a graph is a set of edges, which when removed disconnects the graph
Given a connected undirected graph G = (V,E), Find the cut which has minimum
Cardinality
A B
Applications: Clustering, Network reliability
Various deterministic algorithms known (e.g. max C D E
flow minimum cut algorithm ; Best known O( |V|2
|E| - Push Relabel)
F
All are complex
Cut with minimum cardinality is 3
Edge Contraction
To contract an edge e = (E, F) of G, we merge the vertices E and F and create
a
single vertex EF.
We retain the multiple edges that may result
Do not retain the self loop
A B
A B
C D E
C D EF
F
Edge Contraction
To contract an edge e = (A, B) of G
A B AB
C D E
C D E
F
F
The collapsed edge is G’
Contraction can be done in Ɵ(V)
Karger’s Min-Cut Algorithm
1. Pick an edge e = {x, y} at random
2. Contract the edge e and get G’
3. If there are more than 2 vertices, repeat
4. Else, output the edges remaining as your cut
Picking random edge e is not same as picking two connected vertices x, y at random
Monte Carlo Randomized Algorithm – May not find the Min-Cut always
Time complexity of algorithm Ɵ(V2)
Karger Algorithm
A cut of G’ is a cut of G
The min-cut size of successive graphs never decrease
Algorithms returns cut of the graph
Cut need not be minimal
A cut C is returned as long as none of edges e ↋ C are randomly chosen.
Alternately, an edge e ↋ C is randomly selected for contraction, then
algorithm doesn’t return that cut C
If the probability of finding min-cut using Karger algorithm is significant, we can
iterate Karger algorithm multiple times to improve the probability of finding
min-cut
What is the probability of finding the min-cut?
Probability of finding min-cut
Let C be the a minimum cut; Probability of finding this min-cut C will be lower bound
on probabilities of finding any min-cut
Let Ei be event of NOT choosing an edge in min-cut C in ith contraction and removed
Let Fi be event of NOT choosing an edge in min-cut till the ith iteration
P(Fn-2) = probability that an edge chosen for contraction is NOT an edge min-cut till
the last step ( 2 nodes left)
2
P(Fn-2) >=
𝑛 𝑛−1
n – no of vertices
Boosting
𝟐
If C is a min-cut of G , then the algorithm outputs C with probability at least
𝒏 𝒏−𝟏
>= 2
Probability that C is NOT output by the algorithm is at Max <= 1 -
𝑛 𝑛−1
We can improve the probability by repeating the algorithm
2
Probability that min-cut C is not chosen in any of t trials <= 𝑛 𝑛−1 ) 𝑡
(1 -
Setting t = 𝑛(𝑛 − 1) Gives us a probability of failure 1
<= 𝑒
2
We can boost even further using more repeats
If C is a min-cut of G , then the probability that any of the n(n − 1)/2 repeated trials of the
algorithm does not output C is at most 1/e.
Counting Min-Cuts
𝟐
If C is a min-cut of G , then the algorithm outputs C with probability at least
𝒏 𝒏−𝟏
>=
If G has k min-cuts, C1, C2, . . . , Ck , then the above theorem can be applied for
each Ci
Each of these events are mutually exclusive, since the algorithm
𝟐𝒌 outputs only one cut
The probability of outputting any min-cut is at least
𝒏 𝒏−𝟏
>= 𝒏(𝒏 − 𝟏)
Since probability can NOT exceed 1, we can conclude that 𝒌 𝟐
≤
Probability of finding min-cut – Proof (in first step)
Cut C is returned as long as none of the edges e ∈ C are randomly chosen
If |C | = k, then all vertices have degree at least k
Else the single vertex can form a cut smaller than C
Total number of edges |E | ≥ kn/2
Pr(edge e ∈ C is chosen) ≤ (k) /(kn/2) = 2/n
The probability that an edge in C is contracted in the first step is at most
2/n
The probability that an edge in C is contracted in the first step is at most 2/n
Probability of finding min-cut – Proof (ith step)
If C remains a cut after i steps, then it is a min-cut of the graph
Since C is a min-cut, total number of edges |E | ≥ |C |(n −
i)/2
Pr(edge e ∈ C is chosen) ≤ (|C |) / (|C |(n−i)/2) = 2/(n − i)
Let C be a min-cut of G . If C remains a cut of the graph after i steps,
the probability that an edge in C is contracted in step i + 1 is at most
2/n −
i
Probability of finding min-cut – Proof (main theorem)
If C is a min-cut of G , then the algorithm outputs C with
probability at least 2/n(n − 1)
