Randomized Algorithms

Prabhakar Raghavan
IBM Almaden Research Center
San Jose, CA.

{ Typeset by FoilTEX {
 Deterministic Algorithms
INPUT
ALGORITHM
OUTPUT

Goal: To prove that the

algorithm solves the problem
correctly (always) and quickly
(typically, the number of steps
should be polynomial in the
size of the input).

{ Typeset by FoilTEX { 1
 Randomized Algorithms
INPUT
ALGORITHM
OUTPUT

RANDOM NUMBERS

In addition to input, algorithm

takes a source of random
numbers and makes random
choices during execution.

Behavior can vary even on a
xed input.
{ Typeset by FoilTEX { 2
 Randomized Algorithms
INPUT
ALGORITHM
OUTPUT

RANDOM NUMBERS

Design algorithm + analysis

to show that this behavior is
likely to be good, on every
input.
(The likelihood is over the
random numbers only.)
{ Typeset by FoilTEX { 3
Not to be confused with the
Probabilistic Analysis of
Algorithms
RANDOM
INPUT
ALGORITHM
OUTPUT
DISTRIBUTION

Here the input is assumed

to be from a probability
distribution.

Show that the algorithm works
for most inputs.
{ Typeset by FoilTEX { 4
 Monte Carlo and Las Vegas

A Monte Carlo algorithm runs

for a xed number of steps,
and produces an answer that is
correct with probability  1=3.
A Las Vegas algorithm always
produces the correct answer
its running time is a random
variable whose expectation is
bounded (say by a polynomial).
{ Typeset by FoilTEX { 5
 Monte Carlo and Las Vegas

These probabilities/expectations
are only over the random
choices made by the algorithm
{ independent of the input.
Thus independent repetitions
of Monte Carlo algorithms drive
down the failure probability
exponentially.

{ Typeset by FoilTEX { 6
 Advantages of randomized
algorithms

Simplicity

Performance
For many problems, a randomized
algorithm is the simplest, the
fastest, or both.

{ Typeset by FoilTEX { 7
 Scope

Number-theoretic algorithms:
Primality testing (Monte Carlo).

Data structures: Sorting,
order statistics, searching,
computational geometry.

Algebraic identities: Polynomial
and matrix identity verication.
Interactive proof systems.
{ Typeset by FoilTEX { 8

Mathematical programming:
Faster algorithms for linear
programming. Rounding linear
program solutions to integer
program solutions.

Graph algorithms: Minimum
spanning trees, shortest paths,
minimum cuts.

Counting and enumeration:
Matrix permanent. Counting
combinatorial structures.
{ Typeset by FoilTEX { 9

Parallel and distributed computing:
Deadlock avoidance, distributed
consensus.

Probabilistic existence proofs:
Show that a combinatorial
object arises with non-zero
probability among objects drawn
from a suitable probability
space.

{ Typeset by FoilTEX { 10

Derandomization: First devise
a randomized algorithm, then
argue that it can be \derandomized"
to yield a deterministic algorithm.

{ Typeset by FoilTEX { 11
 Game-tree evaluation
9

MAX

9 3

MIN MIN

7 9 3 17

MAX MAX MAX MAX

1 7 9 5 3 2 17 12

Tree with alternating MAX/MIN

nodes.
Each leaf has a real value.

Goal: To evaluate the root.
{ Typeset by FoilTEX { 12
 Game-tree evaluation
9

MAX

9 3

MIN MIN

7 9 3 17

MAX MAX MAX MAX

1 7 9 5 3 2 17 12

Cost: The number of leaves

inspected.

Algorithm: Computes leaf-
inspection sequence.
{ Typeset by FoilTEX { 13
 Simple special case

Uniform binary tree, height h,

with n=2h
leaves.

All values are boolean, so
MAX!OR and MIN!AND.
Exercise: Every deterministic
algorithm can be forced to read
n leaves.
{ Typeset by FoilTEX { 14
 Randomized algorithm
AND

1 0

OR OR

0 1 0 0

AND AND AND AND

1 0 1 1 1 0 1 0

Evaluate (recursively) a random

child of the current node.
If this does not determine
the value of the current node,
evaluate the other child.
{ Typeset by FoilTEX { 15
 Analysis of tree evaluation
AND-OR trees of depth 2k (k
levels of AND and k levels of
OR on any path) will prove by
induction on k.
Expected cost for an OR
subtree evaluating to 1:
1  3k;1 + 1  2  3k;1 = 3  3k;1:
2 2 2

{ Typeset by FoilTEX { 16
 Analysis of tree evaluation
For an OR subtree evaluating
to 0, we must evaluate both
of its subtrees, incurring cost
2(3k;1).
Now consider the root AND:
(a) If it is 1, both OR subtrees
must evaluate to 1 expected
cost paid is twice above, = 3k.
(b) If it is 0, at least one OR
subtree evaluates to 0.
{ Typeset by FoilTEX { 17
 Game tree analysis

Expected cost for root AND

evaluating to 0:
k 1 1 3
2  3 + 2 23  3 :
; k ;1 k

k
Number of leaves in tree = 4 .
Thus expected cost of randomized
algorithm  n0:793

{ Typeset by FoilTEX { 18
 Lower bounds and the
minimax principle

Basic Idea: For a lower

bound on the performance of
all randomized algorithms for a
problem, derive instead a lower
bound for any deterministic
algorithm for the problem when
the inputs are drawn from a
probability distribution (of your
choice).
{ Typeset by FoilTEX { 19
 Let I denote the set of
instances of the problem, and
A the set of deterministic
algorithms for the problem.
Any randomized algorithm can
be viewed as a probability
distribution on the algorithms in
A.

{ Typeset by FoilTEX { 20
 Minimax Principle

For all distributions p over I

and q over A,
min
A2A
E
C ( Ip  A )]  max
I 2I
E
C ( I A q )] :

{ Typeset by FoilTEX { 21
 Lower bound for game tree
evaluation

We will specify a probability

distribution on instances (0=1
values for the leaves) and lower
bound the expected running
time of any deterministic algorithm.

{ Typeset by FoilTEX { 22
 NOR trees instead

Exercise: Show that a balanced

AND-OR tree of even depth is
equivalent to a tree of the same
depth, all of whose nodes are
NOR nodes.
We will show the lower bound
for a NOR tree with n leaves.

{ Typeset by FoilTEX { 23
 The input distribution
p
Let p = (3 ; 5)=2.
Claim: The value of any
node is 1 with probability p,
independent of all other nodes
at the same level.
Claim: Any deterministic
algorithm may as well determine
the value of one sub-tree of a
node, before inspecting any leaf
of its sibling sub-tree.
{ Typeset by FoilTEX { 24
 The Analysis

Let W (h) be the expected

number of leaves it inspects in
determining the value of a node
at distance h from the leaves.
Clearly
W (h) = W (h;1)+(1;p)W (h;1):
Letting h = log2 n, this gives a
lower bound of n .
0:694

{ Typeset by FoilTEX { 25
 Exercise: Why is this lower
bound weak?

{ Typeset by FoilTEX { 26
 The 2-SAT Problem
Given a set of boolean clauses
in CNF each containing two
literals, nd a satisfying assignment
if one exists.
(x1 + x4)( x4 + x3)(x2 + x 3)   

Start with any tentative assignment

If there is an unsatised
clause, pick one of its two
literals at random, and ip it.
{ Typeset by FoilTEX { 27

If no solution found in 2n2
steps, declare \none exists".
Monte Carlo: If a solution
exists, will nd it with probability
> 1=2.
If not, will always declare \none
exists".

{ Typeset by FoilTEX { 28
 Random Walk Analysis
What is the probability of
missing a satisfying assignment
in 2n2 steps?

Fix any particular satisfying
assignment A.

Consider the number of literals
on which the algorithm's
tentative assignment agrees
with A.
{ Typeset by FoilTEX { 29

This is a random walk on the
integers that increases with
probability at least 1=2 at
each step.
Expected time to nd an
assignment < n2.
Markov's inequality ) probability
of missing an assignment in 2n2
steps is < 1=2.

{ Typeset by FoilTEX { 30
 Binary planar partitions
L2 L1
L1
s1
L2 L3
s
3
s1 s3 s3 s
s 2
2

L3

Given n non-intersecting line-

segments in the plane, build a
small linear decision tree that
has (pieces of) at most one
segment in each cell.

{ Typeset by FoilTEX { 31
 Autopartitions

Choose the input segments in

random order.
Extend the chosen segment (if
necessary) to cut any cells
where it is not \alone".
The expected size of the
resulting tree is  n + 2nHn.

{ Typeset by FoilTEX { 32
Analysis of autopartition size
v
1

u
v
2

Dene l(u) to be the line

containing segment u.
u a v denotes l(u) cuts v in
the constructed tree.
index(u v) is i if l(u) intersects
i ; 1 other segments before
hitting v.
{ Typeset by FoilTEX { 33
 Autopartitions

Pr
u a v]  1=(index(u v)+1):
Thus the expected size of the
X X
tree constructed is
n+ Pr
u a v] 
n+
X X 1
u v6=u

u v6=u index (u v ) + 1

{ Typeset by FoilTEX { 34
For any u i, index(u v) = i for
at most two v thus the above
sum is  n + 2nHn.
It follows that for any n
segments in the plane, there
always exists an autopartition of
size O(n log n).

{ Typeset by FoilTEX { 35
 Matrix product verication

Suppose A B and C are n 

n matrices with entries from a
nite eld F . We wish to verify
whether AB = C .
Multiplying out takes about
n3 steps { this can be cut to
O(n2:38) steps using sophisticated
algorithms.

{ Typeset by FoilTEX { 36
 Simple randomized
algorithm

Pick an n-vector x with

entries randomly drawn from
F.

Compute z = A(Bx).
If z = Cx output AB = C ,
else output AB 6= C .
Takes O(n2) steps.
{ Typeset by FoilTEX { 37
 Simple randomized
algorithm

If AB = C , will always output

AB = C .
If AB 6= C , will output
AB = C with probability at
most 1=jFj.

{ Typeset by FoilTEX { 38
 Sources
R.M. Karp. An introduction to
randomized algorithms. Discrete
Applied Mathematics, 34:165{
201, 1991.
R. Motwani and P. Raghavan.
Randomized Algorithms. Cambridge
University Press, 1995.
D. J. A. Welsh. Randomised
algorithms. Discrete Applied
Mathematics, 5:133{145, 1983.
{ Typeset by FoilTEX { 39