Lesson 5: The Autocovariance Function of a
stochastic process
Umberto Triacca
Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica
Università dell’Aquila,
[email protected] Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Mean and Autocovariance Functions of a stochastic
process
A discrete stochastic process {xt ; t ∈ Z} is a family of random
variables indexed by a parameter t (usually the time).
Thus the moments of the random variables in a stochastic process
are function of the parameter t.
We will consider two moments functions:
1 The mean function;
2 The autocovariance function.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Mean Function of a stochastic process
First, we consider the mean function.
Definition. Let {xt ; t ∈ Z} be a stochastic process such that
Var(xt ) < ∞ ∀t ∈ Z. The function
µx : Z → R
defined by
µx (t) = E (xt )
is called Mean Function of the stochastic process {xt ; t ∈ Z}.
The mean function describes the expectation of the random
variables in the process.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Mean Function of a Random Walk with Drift
Let
{xt ; t = 0, 1, 2, ...}
be a stochastic processs where x0 = δ and xt = λ + xt−1 + ut for
t = 1,2,..., with ut ∼ WN(0, σu2 ).
This process is called random walk with drift. The constant λ is
called the drift.
The mean function of this process is
µx (t) = δ + λt
which is linear function with intercept δ and slope λ.
Why?
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a stochastic process
Definition. Let {xt ; t ∈ Z} be a stochastic process such that
Var(xt ) < ∞ ∀t ∈ Z. The function
γx : Z × Z → R
defined by
γx (t1 , t2 ) = cov(xt1 , xt2 )
is called Autocovariance Function of the stochastic process
{xt ; t ∈ Z}.
The autocovariance function describes the strength of the linear
relationship between the random variables xt1 and xt2 .
It is clear that autocovariance function evaluated in (t,t) gives the
variance, because
h i
γx (t, t) = E (xt − µt )2 = var(xt )
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a Random Walk
Let
{xt ; t = 0, 1, 2, ...}
be a random walk processs with initial condition x0 = 0, and where
xt = xt−1 + ut for t = 1,2,..., with ut ∼ WN(0, σu2 ).
We have
γx (t1 , t2 ) = min {t1 , t2 } σu2
Why?
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a stationary stochastic
process
Consider a weakly stationary stochastic process {xt ; t ∈ Z}.
We have that
γx (t + k, t) = cov(xt+k , xt ) = cov(xk , x0 ) = γx (k, 0) ∀t, k ∈ Z.
We observe that γx (t + k, t) does not depend on t. It depends
only on the time difference k, therefore is convenient to redefine
the autocovariance function of a weakly stationary process as the
function of one variable.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a weakly stationary
process
Definition. The function
γx : Z → R
defined by
γx (k) = Cov(xt , xt−k )
is called autocovariance function of the weakly stationary
stochastic process {xt ; t ∈ Z}.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a weakly stationary
process
Example. Consider a stochastic process {xt ; t ∈ Z} defined by
xt = ut + θut−1
with ut ∼ WN(0, σu2 ).
It is possible to show that this process is weakly stationary.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a weakly stationary
process
The autocovariance function of this process is given by
γx (k) = Cov(xt , xt−k )
1 + θ2 σu2
for k = 0
= θσu2 for k = 1
0 for k = 2, 3, ...
We note that autocovariance function of this process ‘cuts off’
after lag 1.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a weakly stationary
process
Some basic properties of the autocovariance function are:
1 γx (0) ≥ 0
2 |γx (k)| ≤ γx (0) ∀k
3 γx (k) = γx (−k) ∀k
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Functions of a weakly stationary
process
γx (0) ≥ 0 The first is simply the statement that Var(xt ) ≥ 0
|γx (k)| ≤ γx (0) ∀k The second is an immediate consequence
of the fact that the correlation between xt and xt−k is less
than or equal to 1 in absolute value
γx (k) = γx (−k) ∀k The third follows straight from the
definition
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a stationary process
Another important property of γx (.) is that it is non-negative
definite, that is
X n Xn
αi γx (i − j)αj ≥ 0
i=1 j=1
for all positive integers n and vectors α = (α1 , ..., αn )0 ∈ Rn .
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
Parametric Functions of a covariance-stationary process
In fact, we have
n X
n n
!
X X
αi γx (i − j)αj = Var αi xi
i=1 j=1 i=1
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a stationary process
In the class of stationary, zero mean, Gaussian processes
there is a one-to-one correspondence between the family of
the finite dimensional distributions and autocovariance
function
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function of a stationary process
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
The Autocovariance Function
Due to this one-to-one correspondence the statistical properties of
a stationary, zero mean, Gaussian process are completely
determined by its autocovariance function.
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
Autocovariance function
Stationary, zero mean, Gaussian process
'$
DGP γx (k)
&%
7
?
x1 , ..., xT
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process
