Mathematics for Computer Science

Jacob D. Bernoulli (1659 – 1705)

MIT 6.042J/18.062J

Deviation of
Repeated Trials

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.1 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.2

Jacob D. Bernoulli (1659 – 1705) Jacob D. Bernoulli (1659 – 1705)

Even the stupidest man---by some instinct of It certainly remains to be inquired whether
nature per se and by no previous instruction after the number of observations has been
(this is truly amazing) -- knows for sure that increased, the probability…of obtaining the
the more observations ...that are taken, the true ratio…finally exceeds any given
less the danger will be of straying from the degree of certainty; or whether the problem
mark. has, so to speak, its own asymptote---that
---Ars Conjectandi (The Art of Guessing), 1713* is, whether some degree of certainty is
Ars Conjectandi by Jacob Bernoulli,

quoted in Introduction to Probability by Charles Grinstead and J. Laurie Snell,

published by the American Mathematical Society, Providence RI, in 1997. The book is freely available here:
given which one can never exceed.
http://www.dartmouth.edu/%7Echance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf
Ars Conjectandi by Jacob Bernoulli, quoted in Introduction to Probability by Charles Grinstead and J. Laurie Snell,
published by the American Mathematical Society, Providence RI, in 1997. The book is freely available here:
http://www.dartmouth.edu/%7Echance/teaching_aids/books_articles/probability_book/amsbook.mac.pdf
Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.3 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.4

Deviation from the Mean Deviation from the Mean

Pr{observed value far from Observed value means

random variable, R.

expected value}
far from may mean:

is SMALL
• distance or
How small? • amount above (or below)
Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.5 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.6

1
 Markov Bound Chebychev Bound

μ σ2
Pr{R above x} ≤ Pr{|R–µ| > x} ≤
x x2
{

{
{
distance far

{
far small

{
small

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.7 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.8

Binomial Bound Weak Law of Large Numbers

An ::= Avg. of n independent trials

− x2 μ μ::= E[single trial]
Pr{|Bn,p–µ| > x} ≤ e 2 ?
lim ⎡⎣ P r{ An − μ > ε}⎤⎦ = 0

{
n→∞ far
{

{
distance small

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.9 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.11

Jacob D. Bernoulli (1659 – 1705) The Principle Behind:

Therefore, this is the problem which I
now set forth and make known after I • Estimation (polling)
have pondered over it for twenty years. • Algorithm analysis
Both its novelty and its very great • Design against failure
usefulness, coupled with its just as
great difficulty, can exceed in • Communication thru noise
weight and value all the remaining • Gambling
chapters of this thesis.
Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.12 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.13

2
 Not Usable as Stated Repeated Trials

Need to know the

X1,", Xn independent
rate of convergence to 0
with mean, μ, and variance σ2
for any application.
An ::= (X1 + " + Xn)/n
E[An] = nμ/n = μ

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.14 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.15

Repeated Trials Repeated Trials

Var[X1 + " + Xn] = nσ2 So by Chebychev

(by independence) 1

2 Pr{| An − μ | > ε} ≤ (σ ε) 2 ⋅
σ Nn
Var[An] = nσ2/n2 = as n → ∞
n →0

decreases with # trials

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.16 Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.17

Weak Law of Large Numbers

Therefore

⎣ r{ An − μ
> ε}⎤⎦ = 0
lim ⎡P
n→∞
QED

Copyright ©Albert R. Meyer, 2005. December 12, 2005 lec15M.18