Copyright Protected (Do Not Copy)

What we will cover soon

Extract the deterministic components to have a

STATIONARY time series
• De-trending
• Differencing

Find a proper MODEL to describe the stochastic

behavior
• Model Selection and Identification
• Parameter Estimation

Test the non-modeled RESIDUALS to make sure they don’t

carry any information (Model Adequacy)
• Ideally, Residuals should be White Noise
• Several statistical tests

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

48

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Classical Decomposition

X t = mt + St + Yt

Original Stationary
Seasonal
Time series Trend Time series
component
(Nonstationary) (zero-mean)

Seasonal component St satisfies

St+d=St where d= period of seasonality

d

Also for mathematical convenience assume åS

j =1
j =0

Most observed time series are non-stationary but they can be

transformed to stationary processes.
1. 49
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

49

1
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Classical Decomposition X t = mt + St + Yt
Idea of transformation is to estimate mt and St by mt and St, then work
with the stationary process:
æ^ ^ ö
X = X t - ç mt + S t ÷
*
t
è ø
Assume there is no seasonal component (St=0)
X t = mt + Yt
Consider a parametric form for mt e.g.
^
m t = a0 + a1t + a2t 2
Using observed data X1, X2, … Xn, choose a0, a1, a2 to minimize
n 2
^

Farshid Maghami Asl

å(X
t =1
t - mt ) 1. 50
G63.2707 - Financial Econometrics and Statistical Arbitrage

50

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

12000

10000

8000
Dow Jones Index

6000

4000

2000

0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Time in days from 1/1/1975 to 07/30/2005

1. 51
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

51

2
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

10

9.5
Log Transform of Dow Jones Index
9

8.5

7.5

6.5

5.5

5
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Time in days from 1/1/1975 to 07/30/2005

1. 52
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

52

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

10

9.5
Log Transform of Dow Jones Index

8.5

mt = 6.1513 + 0.0004t
8

7.5

6.5

5.5

5
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Time in days from 1/1/1975 to 07/30/2005

1. 53
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

53

3
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Trend Elimination by Differencing

Definition: Differencing Operator Ñ
ÑX t = X t - X t -1
Definition: Backshift Operator B

BX t = X t -1
Therefore
ÑX t = X t - X t -1 = (1 - B ) X t
Also
Ñ 2 X t = (1 - B ) 2 X t = (1 - 2 B + B 2 ) X t
Ñ 2 X t = X t - 2 X t -1 + X t - 2

1. 54
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

54

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Definition: Integrated Process of order n. Time series yt I(n) is

integrated of order n if it is non-stationary, but it becomes stationary after
differencing a minimum of n times.
Example: A stationary process is I(0)
Example: Random Walk is I(1)

Note: Difference removes linear trends as well.

Suppose X t = b 0 + b1t + Yt
ÑX t = b 0 + b1t + Yt - b 0 - b1 (t - 1) - Yt -1
= b1 + Yt - Yt -1
Constant Stationary Process
with mean zero
Note: Difference twice removes quadratic trends.
Warning: Don’t difference too much. Error will be magnified in forecasting
1. 55
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

55

4
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Differencing when the seasonal component is present

Definition: Lagged Differencing Operator Ñ
d

Ñ d X t = X t - X t -d

= (1 - B d ) X t

Note:
Ñd X t ¹ Ñd X t
(1 - B d ) X t (1 - B) d X t

1. 56
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

56

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

Suppose
X t = mt + St + Yt
mt = b 0 + b1t
St = St + d Usually d is known

Ñ d X t = b 0 + b1t + St + Yt - b 0 - b1 (t - d ) - St - d - Yt - d

Ñ d X t = b1d + (Yt - Yt - d )
1. 57
Constant Stationary Process
with mean zero

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

57

5
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

12000

10000

8000
Dow Jones Index

6000

4000

2000

0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Time in days from 1/1/1975 to 07/30/2005

1. 58
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

58

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

0.1
Difference of the Log Transform of Dow Jones Index

0.05

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3
0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Time in days from 1/1/1975 to 07/30/2005

1. 59
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

59

6
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

0.15

0.1

Forecast
0.05
Forecast of the model

-0.05

-0.1

-0.15

-0.2

-0.25

-0.3
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Time in days from 1/1/1975 to 07/30/2005

1. 60
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

60

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

10
Convert back the difference in the Forecast of the model

Forecast
9.5

8.5

7.5

6.5

6
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Time in days from 1/1/1975 to 07/30/2005

1. 61
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

61

7
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

15000

convert back the Log of the Forecast of the model

Forecast

10000

5000

0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Time in days from 1/1/1975 to 07/30/2005

1. 62
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

62

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

4
x 10
2

1.8
Monte Carlo Simulation of the Forecast

1.6

1.4

1.2

0.8

0.6

0.4

0.2

0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Time in days from 1/1/1975 to 07/30/2005

1. 63
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

63

8
 Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

4
x 10
2

1.8
Monte Carlo Simulation of the Forecast

1.6

1.4

1.2

0.8

0.6

0.4

0.2

0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Time in days from 1/1/1975 to 07/30/2005

1. 64
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

64

Copyright Protected (Do Not Copy)

Transforming a Non-Stationary Process to a Stationary Process

4
x 10
1.3

1.2

1.1

0.9

0.8
0 10 20 30 40 50 60 70 80 90 100

60

40

20

0
0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6

4
x 10

1. 65
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

65

9
 Copyright Protected (Do Not Copy)

Message from 2012 to FMA: Need some work. Compare with Lec2 2012

Financial Econometrics and Statistical

Arbitrage

Master of Science Program in Mathematical Finance

New York University

Time Series Analysis:

Building Blocks 2

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

66

Copyright Protected (Do Not Copy)

Properties of Autocovariance Function

Autocovariance Function: (from last lecture)

For a stationary time series {Xt}
Does not depend on t
g X (h) = cov( X t , X t + h )
Properties:

1 g X (0) = var( X t ) ³ 0
2 2
ò f ( x) dx ò g ( x) dx ³ (ò f ( x)g ( x)dx)
2
Schwartz' s Inequality

E ( X t2 ) E ( X t2- h ) ³ [ E ( X t X t - h )]2
2 | g X (h) |£ g X (0) "h
g X (0)g X (0) ³ [g X (h)]2
g X (0) ³| g X (h) |
3 g X (h) = cov( X t , X t + h ) = cov( X t + h , X t ) = g X (-h)

Symmetric
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

67

10
 Copyright Protected (Do Not Copy)

Autocovariance Function

g X (0) g X (2)
Sample Autocovariance Function

0 2 4 6 8 10 12 14 16 18 20

Lag

g X (1)

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

68

Copyright Protected (Do Not Copy)

Autocorrelation Function

Autocorrelation Function is the normalized version of the autocovariance

function:

g X ( h)
r X ( h) = = corr ( X t , X t + h )
g X (0)
From property 2 | g X (h) |£ g X (0) :"h

r X ( h) £ 1
( r X (0) = 1)
Correlogram is the graph of autocorrelation function which is the scaled version of
the autocovariance graph.

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

69

11
 Copyright Protected (Do Not Copy)

Correlogram

r X (0) = 1 r X (2)
Sample Autocorrelation Function
1

0 2 4 6 8 10 12 14 16 18 20

Lag

r X (1)

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

70

Copyright Protected (Do Not Copy)

Sample Autocovariance Function

• Autocovariance function can be obtained from the time series models.

• In practical problems, we do not start with a model, but with the observed (or
realized) data {X1 , X2 , … , Xn }.

Autocovariance
Function

Can be obtained from

Time Series Model g X (h) Observed Time Series

gˆ X (h)
Data {X1 , X2 , … , Xn }

Sample Autocovariance
Function

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

71

12
 Copyright Protected (Do Not Copy)

Sample Autocovariance Function

Let {X1 , X2 , … , Xn } be observations of a time series.

Sample Mean of the observations is:
1 n
x= å xi
n i =1
Sample Autocovariance of the observations is:

 1 n -|h|
g ( h) = å ( xi +|h| - x )( xi - x ) - n < h < n.
n i =1
Sample Autocorrelation of the observations is:

 g ( h)
r ( h) =  - n < h < n.
g (0)
Note: If you observe n data points, you can only calculate g X (h) up to
h=n-1.
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

72

Copyright Protected (Do Not Copy)

Sample Autocovariance Function

Sample Autocorrelation Function (ACF)

4 1

White Noise
2
0.5
Sample Autocorrelation

0
-2

-4 -0.5
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Lag

Sample Autocorrelation Function (ACF)

60 1

40
Random Walk
0.5
Sample Autocorrelation

20

0
0

-20 -0.5
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Lag

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

73

13
 Copyright Protected (Do Not Copy)

Sample Autocovariance Function

Sample Autocorrelation Function (ACF)

6 1

4
MA(1) q = 0.5
0.5
2

Sample Autocorrelation
0
0

-2

-4 -0.5
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Lag
Sample Autocorrelation Function (ACF)

10 1

AR(1) f = 0.9
5
0.5

Sample Autocorrelation
0

0
-5

-10 -0.5
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Lag

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

74

Copyright Protected (Do Not Copy)

Sample Autocovariance Function

Sample Autocorrelation Function (ACF)

4 1

0.5
0
Sample Autocorrelation

-2
0

MA(1) q = -0.5
-4

-6 -0.5
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Sample Autocorrelation Function (ACF)

10 1

AR(1) f = -0.9
5 0.5
Sample Autocorrelation

0 0

-5 -0.5

-10 -1
0 100 200 300 400 500 600 700 0 2 4 6 8 10 12 14 16 18 20

Lag

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

75

14
 Copyright Protected (Do Not Copy)

Confidence Intervals for ACF

If we have n number of IID observations { Xt ; t = 1,2,…,n }, for large n and

h>1, sample ACF will be Normally distributed with
distribution N æç 0, 1 ö÷
è nø
Example: WN simulation with n=200 and confidence level 95%
1.0
Sample Autocorrelation Function (ACF)

0.8

0.6

0.4

-0.3 -0.2 -0.1 0 0.1 0.2 0.3

0.2
95% of the times
1.96
0
0±
n
-0.2
1.96
0± = ±0.1386
-0.4
0 1 2 3 4 5
200
Lag
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

76

Copyright Protected (Do Not Copy)

Sample Autocovariance Function

Sample Autocovariance Function can be used for Model Selection and finding a
good model structure.

Observed Time Series

Data {X1 , X2 , … , Xn }

Calculate Sample Auto-

Correlation function

r X (h)

Compare to Time Series Model’s

Autocorrelation functions r X (h)
and select the structure.

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

77

15
 Copyright Protected (Do Not Copy)

Time Series Models

Moving Average Process: MA(1)

{Zt } is WN(0, s ) 2
Yt = Z t + qZ t -1
Moving Average Process: MA(q)
Yt = Z t + q1Z t -1 + q 2 Z t - 2 +  + q q Z t - q
Using the Backshift operator B,
Yt = (1 + q1 B + q 2 B 2 +  + q q B q ) Z t

Yt = Z t - 0.6 Z t -1 + 0.3Z t - 2 - 0.5Z t -3 + 0.5Z t - 4

Sam ple Autocorrelation Function (ACF)
4 1

2
0.5
0
0
-2

-4 -0.5

-6 -1
0 100 200 300 400 500 600 0 2 4 6 8 10 12 14 16 18 20
Lag
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

78

Copyright Protected (Do Not Copy)

Time Series Models

Autoregressive Process: AR(1)

{Zt } is WN(0, s )
2
X t = fX t -1 + Z t
Autoregressive Process: AR(p)
X t = f1 X t -1 + f2 X t - 2 +  + f p X t - p + Z t
Using the Backshift operator B,
X t - f1 X t -1 - f2 X t - 2 -  - f p X t - p = Z t
(1 - f1 B - f2 B 2 -  - f p B p ) X t = Z t

4
X t = 0.5 X t -1 + 0.1X t - 2 + Z t Sam ple Autocorrelation Function (ACF)
1

2
0.5
0

0
-2

-4 -0.5
0 100 200 300 400 500 600 0 2 4 6 8 10 12 14 16 18 20
Lag
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

79

16
 Copyright Protected (Do Not Copy)

Time Series Models

Moving Average Process: MA(q)

Yt = Z t + q1Z t -1 + q 2 Z t - 2 +  + q q Z t - q

Autoregressive Process: AR(p)

X t = f1 X t -1 + f2 X t - 2 +  + f p X t - p + Yt

Autoregressive Moving Average Process: ARMA(p,q)

X t = f1 X t -1 + f2 X t -2 +  + f p X t - p + Z t + q1Z t -1 + q 2 Z t -2 +  + q q Z t -q

(1 - f1 B - f2 B 2 -  - f p B p ) X t = (1 + q1 B + q 2 B 2 +  + q q B q ) Z t

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

80

Copyright Protected (Do Not Copy)

Time Series Models

(1 - f1 B - f2 B 2 -  - f p B p ) X t = (1 + q1 B + q 2 B 2 +  + q q B q ) Z t
(1 + q1 B + q 2 B 2 +  + q q B q )
Xt = Z t = Y ( B) Z t
(1 - f1 B - f2 B 2 -  - f p B p )
Transfer Function
Autoregressive Moving Average Process: ARMA(2,2)
{Zt } is WN(0, s )
2

(1 - 0.75 B - 0.2 B 2 ) X t = (1 + 0.4 B + 0.3B 2 ) Z t

Sam ple Autocorrelation Function (ACF)

20 1

10
0.5

0
-10

-20 -0.5
0 100 200 300 400 500 600 0 2 4 6 8 10 12 14 16 18 20
Lag

Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

81

17
 Copyright Protected (Do Not Copy)

More on AR(1) Process

AR(1) X t = fX t -1 + Z t Z t » WN (0, s 2 )
Assume |f |<1 and
E ( X t ) = k1 and var( X t ) = k 2
where k1 and k2 are finite constants. By taking expectations and variances of the
AR(1) process:
k1 = fk1 and k2 = f 2 k2 + s 2
So if |f |<1 and for all t
s2
E( X t ) = 0 and var( X t ) =
1-f 2
Autocovariance at lag 1 is: AR(1) process is
fs 2 Stationary if |f |<1
g X (1) = E[(fX t -1 + Z t ) X t -1 ] = fE[ X t2-1 ] =
1-f 2
f 2s 2
g X ( 2) = E[(fX t -1 + Z t ) X t - 2 ] = fE[ X t -1 X t - 2 ] = fg X (1) =
1-f 2
f hs 2
g X ( h) = E[(fX t -1 + Z t ) X t - h ] = fE[ X t -1 X t - h ] = fg X ( h - 1) =
1-f 2
Farshid Maghami Asl G63.2707 - Financial Econometrics and Statistical Arbitrage

82

18