Asymptotic behavior of stochastic algorithms with

statistical applications
Part 1

Bernard Bercu

University of Bordeaux, France

ETICS Annual Research School, Fréjus, 2019

Bernard Bercu Stochastic algorithms with statistical applications 1 / 47

 Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 2 / 47

 Introduction Sample mean and variance.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 3 / 47

 Introduction Sample mean and variance.

Sample mean and variance.

Let (Xn , X ) be a sequence of square integrable independent and
identically distributed random variables with E[X ] = m, Var(X ) = σ 2 .
The sample mean and the sample variance are defined by
n
1 ÿ
Xn = Xk ,
n k =1
n
and 1 ÿ
Sn2 = (Xk − X n )2 .
n k =1

Goal
−→ Recursively estime the unknown mean and variance
 
m
θ= .
σ2

Bernard Bercu Stochastic algorithms with statistical applications 4 / 47

 Introduction Recursive estimation of mean and variance.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 5 / 47

 Introduction Recursive estimation of mean and variance.

Two recursive equations.

We have
n
ÿ
(n + 1)X n+1 = Xk + Xn+1 .
k =1

Consequently,

(n + 1)X n+1 = nX n + Xn+1 = (n + 1)X n + Xn+1 − X n ,

which implies that

1  
X n+1 = X n + Xn+1 − X n .
n+1

Bernard Bercu Stochastic algorithms with statistical applications 6 / 47

 Introduction Recursive estimation of mean and variance.

We also have
n
1 ÿ 2 2
Sn2 = Xk − X n .
n
k =1
Consequently,

n+1
ÿ
2 2
(n + 1)Sn+1 = Xk2 − (n + 1)X n+1 ,
k =1
n+1
ÿ  1  2
= Xk2 − (n + 1) X n + Xn+1 − X n ,
n+1
k =1
n+1
ÿ 2
 
= Xk2 − (n + 1)X n − 2X n Xn+1 − X n − ξn+1
k =1
where
1  2
ξn+1 = Xn+1 − X n .
n+1
Bernard Bercu Stochastic algorithms with statistical applications 7 / 47
 Introduction Recursive estimation of mean and variance.

We also have
n
1 ÿ 2 2
Sn2 = Xk − X n .
n
k =1
Consequently,

n+1
ÿ
2 2
(n + 1)Sn+1 = Xk2 − (n + 1)X n+1 ,
k =1
n+1
ÿ  1  2
= Xk2 − (n + 1) X n + Xn+1 − X n ,
n+1
k =1
n+1
ÿ 2
 
= Xk2 − (n + 1)X n − 2X n Xn+1 − X n − ξn+1
k =1
where
1  2
ξn+1 = Xn+1 − X n .
n+1
Bernard Bercu Stochastic algorithms with statistical applications 7 / 47
 Introduction Recursive estimation of mean and variance.

We also have
n
1 ÿ 2 2
Sn2 = Xk − X n .
n
k =1
Consequently,

n+1
ÿ
2 2
(n + 1)Sn+1 = Xk2 − (n + 1)X n+1 ,
k =1
n+1
ÿ  1  2
= Xk2 − (n + 1) X n + Xn+1 − X n ,
n+1
k =1
n+1
ÿ 2
 
= Xk2 − (n + 1)X n − 2X n Xn+1 − X n − ξn+1
k =1
where
1  2
ξn+1 = Xn+1 − X n .
n+1
Bernard Bercu Stochastic algorithms with statistical applications 7 / 47
 Introduction Recursive estimation of mean and variance.

Two recursive equations.

Therefore,
n
ÿ
2 2 2
(n + 1)Sn+1 = Xk2 − nX n + Xn+1
2
− 2X n Xn+1 + X n − ξn+1 ,
k =1
 2
= nSn2 + Xn+1 − X n − ξn+1 .

Hence
 2
2
(n + 1)Sn+1 = (n + 1)Sn2 + Xn+1 − X n − Sn2 − ξn+1 ,

leading to

2 1  2  1  2
Sn+1 = Sn2 + Xn+1 − X n −Sn2 − Xn+1 − X n .
n+1 (n + 1)2

Bernard Bercu Stochastic algorithms with statistical applications 8 / 47

 Introduction Recursive estimation of mean and variance.

Two recursive equations.

Therefore,
n
ÿ
2 2 2
(n + 1)Sn+1 = Xk2 − nX n + Xn+1
2
− 2X n Xn+1 + X n − ξn+1 ,
k =1
 2
= nSn2 + Xn+1 − X n − ξn+1 .

Hence
 2
2
(n + 1)Sn+1 = (n + 1)Sn2 + Xn+1 − X n − Sn2 − ξn+1 ,

leading to

2 1  2  1  2
Sn+1 = Sn2 + Xn+1 − X n −Sn2 − Xn+1 − X n .
n+1 (n + 1)2

Bernard Bercu Stochastic algorithms with statistical applications 8 / 47

 Introduction Recursive estimation of mean and variance.

A recursive matrix equation.

Denote  
Xn
θpn = .
Sn2

It follows from the previous calculation that

1 1
θpn+1 = θpn + F θpn , Xn+1 +


rn+1
n+1 (n + 1)
where
!
Xn+1 − X n
F θpn , Xn+1 =



2
Xn+1 − X n − Sn2
and  
0
rn+1 =  1
2  .
− Xn+1 − X n
(n + 1)
Bernard Bercu Stochastic algorithms with statistical applications 9 / 47
 Introduction Recursive estimation of mean and variance.

A recursive matrix equation.

However,
E F θpn , Xn+1 |Fn = f θpn + sn



where f θpn = θ − θpn and

!
0
sn =
2 .
m − Xn
Consequently, we obtain the martingale decomposition

1  
θpn+1 = θpn + f θpn + εn+1 + Rn+1


n+1

where (εn ) is a martingale difference sequence, E[εn+1 |Fn ] = 0 and

the remainder Rn+1 = rn+1 + sn is negligeable.

Bernard Bercu Stochastic algorithms with statistical applications 10 / 47

 Introduction Recursive estimation of mean and variance.

A first warm-up result.

Theorem
Assume that (Xn , X ) is a sequence of iid random variables such that
E[X 4 ] is finite. Denote E[(X − m)3 ] = µ3 and E[(X − m)4 ] = τ 4 . Then,
we have the almost sure convergence

lim θpn = θ a.s.

n→∞

In addition, we also have the asymptotic normality

‘  
L
n θpn − θ −→ N (0, Γ)

where
σ2 µ3
 
Γ= .
µ3 τ 4 − σ4

Bernard Bercu Stochastic algorithms with statistical applications 11 / 47

 Introduction Quantile of a continuous distribution.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 12 / 47

 Introduction Quantile of a continuous distribution.

Quantile of a continuous distribution.

Let X be a continuous random variable with unknown distribution
function F . Assume that F is continuous and strictly increasing.
Definition
For any α in ]0, 1[, the quantile of order α of X is the unique solution θα
of the equation F (x) = α,

F (θα ) = α.

For the Exponential E(λ) distribution with λ > 0,

1
θα = − log(1 − α).
λ
Goal
−→ Recursively estime the unknown quantile θα .
Bernard Bercu Stochastic algorithms with statistical applications 13 / 47
 Introduction Recursive estimation of quantile.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 14 / 47

 Introduction Recursive estimation of quantile.

Let (Xn ) be a sequence of iid random variables sharing the same

distribution as X . We estimate θα by the recursive estimator

1  
θpn+1 = θpn − Yn+1 − α
n+1

where

Yn+1 = F θpn , Xn+1 = I{Xn+1 6θpn } .

We clearly have E Yn+1 |Fn = F θpn leading to the martingale

 

decomposition

1  
θpn+1 = θpn − F θpn + εn+1 − α


n+1

where (εn ) is a martingale difference sequence, E[εn+1 |Fn ] = 0.

Bernard Bercu Stochastic algorithms with statistical applications 15 / 47
 Introduction Recursive estimation of quantile.

A second warm-up result.

Denote by f the probability density function of X .

Theorem
We have the almost sure convergence

lim θpn = θα a.s.

n→∞

Moreover, as soon as f (θα ) > 1/2, we have the asymptotic normality

‘  
L  α(1 − α) 
n θpn − θα −→ N 0, .
2f (θα ) − 1

Bernard Bercu Stochastic algorithms with statistical applications 16 / 47

 Introduction Recursive estimation of quantile.

A second warm-up result.

Consider the slow down Robbins-Monro algorithm given by

 
θpn+1 = θpn − γn Yn+1 − α

where

1 1
γn = with < c < 1.
nc 2

At time n > 1, compute de Cesaro mean

n
1 ÿ
θn = θpk .
n k =1

Bernard Bercu Stochastic algorithms with statistical applications 17 / 47

 Introduction Recursive estimation of quantile.

A second warm-up result.

We already saw that

1  
θ n+1 = θ n + θpn+1 − θ n .
n+1

Theorem
We have the almost sure convergence

lim θ n = θα a.s.
n→∞

Moreover, we also have the asymptotic normality

‘  
L  α(1 − α) 
n θ n − θα −→ N 0, 2 .
f (θα )

Bernard Bercu Stochastic algorithms with statistical applications 18 / 47

 Convergence of martingales Definition and Examples.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 19 / 47

 Convergence of martingales Definition and Examples.

Let (Ω, A, P) be a probability space with a filtration F = (Fn ) where Fn

is the σ-algebra of events occurring up to time n.
Definition
Let (Mn ) be a sequence of integrable random variables defined on
(Ω, A, P) such that, for all n > 0, Mn is Fn -measurable.

1 (Mn ) is a martingale MG if for all n > 0,

E[Mn+1 | Fn ] = Mn a.s.

2 (Mn ) is a submartingale sMG if for all n > 0,

E[Mn+1 | Fn ] > Mn a.s.

3 (Mn ) is a supermartingale SMG if for all n > 0,

E[Mn+1 | Fn ] 6 Mn a.s.

Bernard Bercu Stochastic algorithms with statistical applications 20 / 47

 Convergence of martingales Definition and Examples.

Let (Ω, A, P) be a probability space with a filtration F = (Fn ) where Fn

is the σ-algebra of events occurring up to time n.
Definition
Let (Mn ) be a sequence of integrable random variables defined on
(Ω, A, P) such that, for all n > 0, Mn is Fn -measurable.

1 (Mn ) is a martingale MG if for all n > 0,

E[Mn+1 | Fn ] = Mn a.s.

2 (Mn ) is a submartingale sMG if for all n > 0,

E[Mn+1 | Fn ] > Mn a.s.

3 (Mn ) is a supermartingale SMG if for all n > 0,

E[Mn+1 | Fn ] 6 Mn a.s.

Bernard Bercu Stochastic algorithms with statistical applications 20 / 47

 Convergence of martingales Definition and Examples.

Let (Ω, A, P) be a probability space with a filtration F = (Fn ) where Fn

is the σ-algebra of events occurring up to time n.
Definition
Let (Mn ) be a sequence of integrable random variables defined on
(Ω, A, P) such that, for all n > 0, Mn is Fn -measurable.

1 (Mn ) is a martingale MG if for all n > 0,

E[Mn+1 | Fn ] = Mn a.s.

2 (Mn ) is a submartingale sMG if for all n > 0,

E[Mn+1 | Fn ] > Mn a.s.

3 (Mn ) is a supermartingale SMG if for all n > 0,

E[Mn+1 | Fn ] 6 Mn a.s.

Bernard Bercu Stochastic algorithms with statistical applications 20 / 47

 Convergence of martingales Definition and Examples.

Martingales with sums.

Example (Sums)
Let (Xn ) be a sequence of integrable and independent random
variables such that, for all n > 1, E[Xn ] = m. Denote
n
ÿ
Sn = Xk .
k =1

We clearly have

Sn+1 = Sn + Xn+1 .

Consequently, (Sn ) is a sequence of integrable random variables with

E[Sn+1 | Fn ] = Sn + E[Xn+1 | Fn ],
= Sn + E[Xn+1 ],
Bernard Bercu = S +m
Stochastic algorithms n
with statistical applications 21 / 47
 Convergence of martingales Definition and Examples.

Martingales with sums.

Example (Sums)

E[Sn+1 | Fn ] = Sn + m.

(Sn ) is a martingale if m = 0,
(Sn ) is a submartingale if m > 0,
(Sn ) is a supermartingale if m 6 0.

−→ It holds for Rademacher R(p) distribution with 0 < p < 1 where

m = 2p − 1.

Bernard Bercu Stochastic algorithms with statistical applications 22 / 47

 Convergence of martingales Definition and Examples.

Martingales with Rademacher sums.

Martingales with Rademacher sums
60
Martingale
Submartingale
Supermartingale
40

20

−20

−40

−60
0 20 40 60 80 100

Bernard Bercu Stochastic algorithms with statistical applications 23 / 47

 Convergence of martingales Definition and Examples.

Martingales with products.

Example (Products)
Let (Xn ) be a sequence of positive, integrable and independent
random variables such that, for all n > 1, E[Xn ] = m. Denote
n
ź
Pn = Xk .
k =1

We clearly have

Pn+1 = Pn Xn+1 .

Consequently, (Pn ) is a sequence of integrable random variables with

E[Pn+1 | Fn ] = Pn E[Xn+1 | Fn ],
= Pn E[Xn+1 ],
Bernard Bercu = mP
Stochastic algorithms with statistical
n applications 24 / 47
 Convergence of martingales Definition and Examples.

Martingales with products.

Example (Products)

E[Pn+1 | Fn ] = mPn .

(Pn ) is a martingale if m = 1,
(Pn ) is a submartingale if m > 1,
(Pn ) is a supermartingale if m 6 1.

−→ It holds for Exponential E(λ) distribution with λ > 0 where

1
m= .
λ

Bernard Bercu Stochastic algorithms with statistical applications 25 / 47

 Convergence of martingales On Doob’s convergence theorem.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 26 / 47

 Convergence of martingales On Doob’s convergence theorem.

Doob’s convergence theorem.

Theorem (Doob)
Let (Mn ) be a MG, sMG, or SMG bounded in L1 which means

sup E[|Mn |] < +∞.

n>0

Then, we have the almost sure convergence

lim Mn = M∞ a.s.
n→∞

where M∞ is an integrable random variable.

Bernard Bercu Stochastic algorithms with statistical applications 27 / 47

 Convergence of martingales On Doob’s convergence theorem.

Joseph Leo Doob

Bernard Bercu Stochastic algorithms with statistical applications 28 / 47

 Convergence of martingales On Doob’s convergence theorem.

Jacques Neveu

Bernard Bercu Stochastic algorithms with statistical applications 29 / 47

 Convergence of martingales On Doob’s convergence theorem.

Convergence of martingales.

Theorem
Let (Mn ) be a MG bounded in Lp with p > 1, which means that

sup E[|Mn |p ] < +∞.

n>0

1 If p > 1, (Mn ) converges almost surely to an integrable random

variable M∞ . The convergence is also true in Lp .
2 If p = 1, (Mn ) converges almost surely to an integrable random
variable M∞ . The convergence holds in L1 as soon as (Mn ) is
uniformly integrable that is
 
lim sup E |Mn |I{|Mn |>a} = 0.
a→∞ n>0

Bernard Bercu Stochastic algorithms with statistical applications 30 / 47

 Convergence of martingales On Doob’s convergence theorem.

Chow’s Theorem.
Theorem (Chow)
Let (Mn ) be a MG such that for 1 6 a 6 2 and for all n > 1,

E[|Mn |a ] < ∞.

Denote, for all n > 1, ∆Mn = Mn − Mn−1 and assume that

∞
ÿ
E[|∆Mn |a |Fn−1 ] < ∞ a.s.
n=1

Then, we have the almost sure convergence

lim Mn = M∞ a.s.
n→∞

where M∞ is an integrable random variable.

Bernard Bercu Stochastic algorithms with statistical applications 31 / 47
 Convergence of martingales On Doob’s convergence theorem.

Exponential Martingale.

Example (Exponential Martingale)

Let (Xn ) be a sequence of independent random variable sharing the

same N (0, 1) distribution. For all t ∈ R∗ , let Sn = X1 + · · · + Xn and
denote
 nt 2 
Mn (t) = exp tSn − .
2

It is clear that (Mn (t)) is a MG which converges a.s. to zero. However,

E[Mn (t)] = E[M1 (t)] = 1. It means that (Mn (t)) does not converge in L1 .

Bernard Bercu Stochastic algorithms with statistical applications 32 / 47

 Convergence of martingales On Doob’s convergence theorem.

Autoregressive Martingale.

Example (Autoregressive Martingale)

Let (Xn ) be the autoregressive process given for all n > 0 by

Xn+1 = θXn + (1 − θ)εn+1

where the initial state X0 = p, 0 < p < 1 and the parameter 0 < θ < 1.
Assume that L(εn+1 |Fn ) is the Bernoulli B(Xn ) distribution. We can
show that 0 < Xn < 1 and that (Xn ) is a MG satisfying

lim Xn = X∞ a.s.
n→∞

The convergence also holds in Lr for all r > 1. Finally, we can prove
that X∞ has the Bernoulli B(p) distribution.

Bernard Bercu Stochastic algorithms with statistical applications 33 / 47

 Convergence of martingales Square integrable martingales.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 34 / 47

 Convergence of martingales Square integrable martingales.

Increasing process.

Definition
Let (Mn ) be a square integrable MG that is for all n > 1,

E[Mn2 ] < ∞.

The increasing process associated with (Mn ) is given by < M >0 = 0

and, for all n > 1,
n
ÿ
< M >n = E[∆Mk2 |Fk −1 ]
k =1

where ∆Mk = Mk − Mk −1 .

Bernard Bercu Stochastic algorithms with statistical applications 35 / 47

 Convergence of martingales Square integrable martingales.

Example (Increasing Process)

Let (Xn ) be a sequence of square integrable and independent random

variables such that, for all n > 1, E[Xn ] = m and Var(Xn ) = σ 2 > 0.
Denote
n
ÿ
Mn = (Xk − m).
k =1

Then, (Mn ) is a martingale and its increasing process reduces to

< M >n = σ 2 n.

Bernard Bercu Stochastic algorithms with statistical applications 36 / 47

 Convergence of martingales Robbins-Siegmund theorem.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 37 / 47

 Convergence of martingales Robbins-Siegmund theorem.

Theorem (Robbins-Siegmund)
Let (Vn ), (An ) and (Bn ) be three positive sequences adapted to
F = (Fn ). Assume that V0 is integrable and, for all n > 0,

E[Vn+1 |Fn ] 6 Vn + An − Bn a.s.

Assume also that

∞
ÿ
An < +∞ a.s.
n=0

1 The sequence (Vn ) converges a.s. to a random variable V∞ .

2 We also have
∞
ÿ
Bn < +∞ a.s.
n=0

Bernard Bercu Stochastic algorithms with statistical applications 38 / 47

 Convergence of martingales Robbins-Siegmund theorem.

Corollary
Let (Vn ), (An ), (Bn ) and (an ) be four positive sequences adapted to
F = (Fn ). Assume that V0 is integrable and, for all n > 0,

E[Vn+1 |Fn ] 6 Vn (1 + an ) + An − Bn a.s.

Assume also that

∞
ÿ ∞
ÿ
an < +∞, An < +∞ a.s.
n=0 n=0

1 The sequence (Vn ) converges a.s. to a random variable V∞ .

2 We also have
n
ÿ
Bk < +∞ a.s.
k =0
Bernard Bercu Stochastic algorithms with statistical applications 39 / 47
 Convergence of martingales Strong law of large numbers for martingales.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 40 / 47

 Convergence of martingales Strong law of large numbers for martingales.

Strong law of large numbers for martingales.

Theorem (Strong Law of large numbers)
Let (Mn ) be a square integrable MG and denote

< M >∞ = lim < M >n .

n→∞

1 Assume that < M >∞ < ∞ a.s. Then, we have

lim Mn = M∞ a.s.
n→∞

2 Assume that < M >∞ = ∞ a.s. Then, we have

Mn
lim =0 a.s.
n→∞ < M >n

−→ If it exists a positive sequence (an ) increasing to infinity such that

< M >n = 0(an ) a.s., then we have Mn = o(an ) a.s.
Bernard Bercu Stochastic algorithms with statistical applications 41 / 47
 Convergence of martingales Strong law of large numbers for martingales.

Strong law of large numbers for martingales, continued

Theorem (Strong Law of large numbers)
Let (Mn ) be a square integrable MG such that

lim < M >n = ∞ a.s.

n→∞

1 For any positive γ, we have

Mn2  1+γ 
=o log < M >n a.s.
< M >n
2 If the increments of (Mn ) have conditional moments of order > 2,

Mn2  
= O log < M >n a.s.
< M >n

Bernard Bercu Stochastic algorithms with statistical applications 42 / 47

 Convergence of martingales Strong law of large numbers for martingales.

Example on sums.
Let (Xn ) be a sequence of square integrable and independent random
variables such that, for all n > 1, E[Xn ] = m and Var(Xn ) = σ 2 > 0. We
already saw that
ÿn
Mn = (Xk − m)
k =1

is square integrable MG with < M >n = σ 2 n. It follows from the SLLN

for martingales that Mn = o(n) a.s. which means that
n
1 ÿ
lim Xk = m a.s.
n→∞ n k =1

More precisely, for any positive γ,

 M 2 1 ÿ n 2  (log n)1+γ 
n
= Xk − m = o a.s.
n n k =1 n

Bernard Bercu Stochastic algorithms with statistical applications 43 / 47

 Convergence of martingales Central limit theorem for martingales.

Outline

1 Introduction
Sample mean and variance.
Recursive estimation of mean and variance.
Quantile of a continuous distribution.
Recursive estimation of quantile.

2 Convergence of martingales
Definition and Examples.
On Doob’s convergence theorem.
Square integrable martingales.
Robbins-Siegmund theorem.
Strong law of large numbers for martingales.
Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 44 / 47

 Convergence of martingales Central limit theorem for martingales.

Central limit theorem for martingales.

Theorem (Central Limit Theorem)

Let (Mn ) be a square integrable MG and let (an ) be a sequence of
positive real numbers increasing to infinity. Assume that
1 It exists a deterministic limit L > 0 such that
< M >n P
−→ L.
an

2 Lindeberg’s condition. For all positive ε,

n
1 ÿ P
E[|∆Mk |2 I{|∆Mk |>ε‘an } |Fk −1 ] −→ 0
an
k =1

where ∆Mk = Mk − Mk −1 .

Bernard Bercu Stochastic algorithms with statistical applications 45 / 47

 Convergence of martingales Central limit theorem for martingales.

Central limit theorem for martingales.

Theorem (Central Limit Theorem)

Let (Mn ) be a square integrable MG and let (an ) be a sequence of
positive real numbers increasing to infinity. Assume that
1 It exists a deterministic limit L > 0 such that
< M >n P
−→ L.
an

2 Lindeberg’s condition. For all positive ε,

n
1 ÿ P
E[|∆Mk |2 I{|∆Mk |>ε‘an } |Fk −1 ] −→ 0
an
k =1

where ∆Mk = Mk − Mk −1 .

Bernard Bercu Stochastic algorithms with statistical applications 45 / 47

 Convergence of martingales Central limit theorem for martingales.

Central limit theorem fro martingales, continued.

Theorem (Central Limit Theorem)
Then, we have the asymptotic normality

1 L
‘ Mn −→ N (0, L).
an

Moreover, if L > 0, we also have

‘  Mn 
L
an −→ N (0, L−1 ).
< M >n

−→ Lyapunov’s condition implies Lindeberg’s condition : For b > 2,

n
ÿ
E[|∆Mk |b |Fk −1 ] = O(an ) a.s.
k =1

Bernard Bercu Stochastic algorithms with statistical applications 46 / 47

 Convergence of martingales Central limit theorem for martingales.

Bernard Bercu Stochastic algorithms with statistical applications 47 / 47