Outline
1. Objectives of time series analysis. Examples.
2. Overview of the course.
3. Time series models.
4. Time series modelling: Chasing stationarity.
49
A Time Series
400
350
300
250
200
150
100
50
0
0 1000 2000 3000 4000 5000 6000 7000
4
A Time Series
400
350
300
250
200
150
100
50
0
1960 1965 1970 1975 1980 1985 1990
year
5
A Time Series
400
350
300
250
200
$
150
100
50
0
1960 1965 1970 1975 1980 1985 1990
year
6
A Time Series
SP500: 1960−1990
400
350
300
250
200
$
150
100
50
0
1960 1965 1970 1975 1980 1985 1990
year
7
A Time Series
SP500: Jan−Jun 1987
340
320
300
280
$
260
240
220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year
8
A Time Series
SP500 Jan−Jun 1987. Histogram
30
25
20
15
10
0
240 250 260 270 280 290 300 310
$
9
A Time Series
SP500: Jan−Jun 1987. Permuted.
340
320
300
280
$
260
240
220
0 20 40 60 80 100 120
10
Objectives of Time Series Analysis
1. Compact description of data.
2. Interpretation.
3. Forecasting.
4. Control.
5. Hypothesis testing.
6. Simulation.
11
Classical decomposition: An example
Monthly sales for a souvenir shop at a beach resort town in Queensland.
(Makridakis, Wheelwright and Hyndman, 1998)
4
x 10
12
10
0
0 10 20 30 40 50 60 70 80 90
12
Transformed data
12
11.5
11
10.5
10
9.5
8.5
7.5
7
0 10 20 30 40 50 60 70 80 90
13
Trend
12
11.5
11
10.5
10
9.5
8.5
7.5
7
0 10 20 30 40 50 60 70 80 90
14
Residuals
1.5
0.5
−0.5
−1
0 10 20 30 40 50 60 70 80 90
15
Trend and seasonal variation
12
11.5
11
10.5
10
9.5
8.5
7.5
7
0 10 20 30 40 50 60 70 80 90
16
Objectives of Time Series Analysis
1. Compact description of data.
Example: Classical decomposition: Xt = Tt + St + Yt .
2. Interpretation. Example: Seasonal adjustment.
3. Forecasting. Example: Predict sales.
4. Control.
5. Hypothesis testing.
6. Simulation.
17
Unemployment data
Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994)
5
x 10
8
7.5
6.5
5.5
4.5
4
1983 1984 1985 1986 1987 1988 1989 1990
18
Trend
5
x 10
8
7.5
6.5
5.5
4.5
4
1983 1984 1985 1986 1987 1988 1989 1990
19
Trend plus seasonal variation
5
x 10
8
7.5
6.5
5.5
4.5
4
1983 1984 1985 1986 1987 1988 1989 1990
20
Residuals
4
x 10
8
−2
−4
−6
1983 1984 1985 1986 1987 1988 1989 1990
21
Predictions based on a (simulated) variable
5
x 10
8
7.5
6.5
5.5
4.5
4
1983 1984 1985 1986 1987 1988 1989 1990
22
Objectives of Time Series Analysis
1. Compact description of data:
Xt = Tt + St + f (Yt ) + Wt .
2. Interpretation. Example: Seasonal adjustment.
3. Forecasting. Example: Predict unemployment.
4. Control. Example: Impact of monetary policy on unemployment.
5. Hypothesis testing. Example: Global warming.
6. Simulation. Example: Estimate probability of catastrophic events.
23
Overview of the Course
1. Time series models
2. Time domain methods
3. Spectral analysis
4. State space models(?)
24
Overview of the Course
1. Time series models
(a) Stationarity.
(b) Autocorrelation function.
(c) Transforming to stationarity.
2. Time domain methods
3. Spectral analysis
4. State space models(?)
25
Overview of the Course
1. Time series models
2. Time domain methods
(a) AR/MA/ARMA models.
(b) ACF and partial autocorrelation function.
(c) Forecasting
(d) Parameter estimation
(e) ARIMA models/seasonal ARIMA models
3. Spectral analysis
4. State space models(?)
26
Overview of the Course
1. Time series models
2. Time domain methods
3. Spectral analysis
(a) Spectral density
(b) Periodogram
(c) Spectral estimation
4. State space models(?)
27
Overview of the Course
1. Time series models
2. Time domain methods
3. Spectral analysis
4. State space models(?)
(a) ARMAX models.
(b) Forecasting, Kalman filter.
(c) Parameter estimation.
28
Time Series Models
A time series model specifies the joint distribution of the se-
quence {Xt } of random variables.
For example:
P [X1 ≤ x1 , . . . , Xt ≤ xt ] for all t and x1 , . . . , xt .
Notation:
X1 , X2 , . . . is a stochastic process.
x1 , x2 , . . . is a single realization.
We’ll mostly restrict our attention to second-order properties only:
EXt , E(Xt1 , Xt2 ).
29
Time Series Models
Example: White noise: Xt ∼ W N (0, σ 2 ).
i.e., {Xt } uncorrelated, EXt = 0, VarXt = σ 2 .
Example: i.i.d. noise: {Xt } independent and identically distributed.
P [X1 ≤ x1 , . . . , Xt ≤ xt ] = P [X1 ≤ x1 ] · · · P [Xt ≤ xt ].
Not interesting for forecasting:
P [Xt ≤ xt |X1 , . . . , Xt−1 ] = P [Xt ≤ xt ].
30
Gaussian white noise
Z xt
1 −x2 /2
P [Xt ≤ xt ] = Φ(xt ) = √ e dx.
2π −∞
2.5
1.5
0.5
−0.5
−1
−1.5
−2
−2.5
0 5 10 15 20 25 30 35 40 45 50
31
Gaussian white noise
2.5
1.5
0.5
−0.5
−1
−1.5
−2
−2.5
0 5 10 15 20 25 30 35 40 45 50
32
Time Series Models
Example: Binary i.i.d. P [Xt = 1] = P [Xt = −1] = 1/2.
1
0.8
0.6
0.4
0.2
−0.2
−0.4
−0.6
−0.8
−1
0 5 10 15 20 25 30 35 40 45 50
33
Random walk
Pt
St = i=1 Xi . Differences: ∇St = St − St−1 = Xt .
8
−2
−4
0 5 10 15 20 25 30 35 40 45 50
34
Random walk
ESt ? VarSt ?
10
−5
−10
−15
0 5 10 15 20 25 30 35 40 45 50
35
Random Walk
Recall S&P500 data. (Notice that it’s smooth)
SP500: Jan−Jun 1987
340
320
300
280
$
260
240
220
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year
36
Random Walk
Differences: ∇St = St − St−1 = Xt .
SP500, Jan−Jun 1987. first differences
10
0
$
−2
−4
−6
−8
−10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year
37
Trend and Seasonal Models
P
Xt = Tt + St + Et = β0 + β1 t + i (βi cos(λi t) + γi sin(λi t)) + Et
6
5.5
4.5
3.5
2.5
0 50 100 150 200 250
38
Trend and Seasonal Models
Xt = Tt + Et = β0 + β1 t + Et
6
5.5
4.5
3.5
2.5
0 50 100 150 200 250
39
Trend and Seasonal Models
P
Xt = Tt + St + Et = β0 + β1 t + i (βi cos(λi t) + γi sin(λi t)) + Et
6
5.5
4.5
3.5
2.5
0 50 100 150 200 250
40
Trend and Seasonal Models: Residuals
0.5
0.4
0.3
0.2
0.1
−0.1
−0.2
−0.3
−0.4
−0.5
0 50 100 150 200 250
41
Time Series Modelling
1. Plot the time series.
Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.
42
Nonlinear transformations
Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998)
4
x 10 12
12
11.5
10 11
10.5
8
10
9.5
6
4
8.5
8
2
7.5
0 7
0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90
43
Time Series Modelling
1. Plot the time series.
Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.
44
Differencing
Recall: S&P 500 data.
SP500: Jan−Jun 1987 SP500, Jan−Jun 1987. first differences
340 10
320
6
4
300
280 0
$
$
−2
260
−4
−6
240
−8
220 −10
1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5
year year
45
Differencing and Trend
Define the lag-1 difference operator, (think ‘first derivative’)
∇Xt = Xt − Xt−1 = (1 − B)Xt ,
where B is the backshift operator, BXt = Xt−1 .
• If Xt = β0 + β1 t + Yt , then
∇Xt = β1 + ∇Yt .
Pk
• If Xt = i=0 βi ti + Yt , then
∇k Xt = k!βk + ∇k Yt ,
where ∇k Xt = ∇(∇k−1 Xt ) and ∇1 Xt = ∇Xt .
46
Differencing and Seasonal Variation
Define the lag-s difference operator,
∇s Xt = Xt − Xt−s = (1 − B s )Xt ,
where B s is the backshift operator applied s times, B s Xt = B(B s−1 Xt )
and B 1 Xt = BXt .
If Xt = Tt + St + Yt , and St has period s (that is, St = St−s for all t), then
∇s Xt = Tt − Tt−s + ∇s Yt .
47
Time Series Modelling
1. Plot the time series.
Look for trends, seasonal components, step changes, outliers.
2. Transform data so that residuals are stationary.
(a) Estimate and subtract Tt , St .
(b) Differencing.
√
(c) Nonlinear transformations (log, ·).
3. Fit model to residuals.
48
