An analysis of a single time series is called a univariate time series.
In this, we extract as much
information as possible from the history of the series. From the history we also mean that each
time series follow a specific data generating process, i.e. AR, MA, ARMA, ARIMA, etc.
The AR model
Simplest, purely statistical time series model is the autoregressive of order one model, or AR(1)
model:
Yt = φYt−1 + ut
where, for simplicity, we do not include a constant and |φ| < 1 and ut is a white noise error term.
The implication behind the AR (1) model is that the time series behavior of Yt is largely
determined by its own value in the preceding period, or what will happen in t is largely
dependent on what happened in t −1, or alternatively what will happen in t + 1 will be
determined by the behavior of the series in the current time t.
A generalization of the AR(1) model is the AR(p) model; the number in parenthesis denotes the
order of the autoregressive process and therefore the number of lagged dependent variables that
the model will have.
Yt = φ1Yt−1 + φ2Yt−2 +· · ·+φpYt−p + ut
or
Yt =ΣφiYt−i + ut
The MA model
The simplest moving average model is that of order one, or the MA (1) model, which has the
form:
Yt = ut + θut−1
Thus, the implication behind the MA (1) model is that Yt depends on the value of the immediate
past error, which is known at time t. The general form of the MA model has the form:
Yt = ut + θ1ut−1 + θ2ut−2+…+ θqut−q
or
Yt = ut + Σ θjut−j
ARMA Models
The general form of the ARMA model is an ARMA(p, q) model of the form:
Yt = φ1Yt−1 + φ2Yt−2 +· · ·+φpYt−p + ut+θ1ut−1 + θ2ut−2 +· · ·+θqut−q
or
Yt = ΣφiYt−i + ut +Σ θjut−j
ARIMA Models
ARMA models can only be made on time series Yt that are stationary.
Stationarity:
A time series is stationary if it’s mean, its variance and its covariance remain constant over time.
Stationarity is important because if the series is non-stationary then all typical results of the
classical regression analysis may not valid.
To avoid this problem, and to induce stationarity, we detrend raw data through differencing.
First differences of a series Yt are given by equation:
ΔYt = Yt − Yt−1
If, after first differencing, series is stationary then the series is also called integrated to order one,
and denoted I(1).In general if stationary after d differences, then called I(d). Thus, we have
ARIMA(p,d,q).
Box and Jenkins Methodology
Box and Jenkins popularized three-stage method aimed at selecting appropriate ARIMA model
for estimating and forecasting a univariate time series.
The three stages are:
(a) Identification,
(b) Estimation, and
(c) Diagnostic checking
Box-Jenkins Approach
Step 1 Calculate ACF and PACF of the raw data, and check whether series is stationary or not. If
series is stationary go to step 3, if not go to step 2.
Step 2 Take logarithm and first differences of raw data and calculate the ACF and PACF for the
first logarithmic differenced series.
Step 3 Examine graphs of the ACF and PACF and determine which models would be good
starting points.
Step 4 Estimate those models.
Step 5 For each of these estimated models:
(a) check to see if parameter of longest lag is significant. If not, you probably have too many
parameters and should decrease the order of p and/or q.
(b) check ACF and PACF of the errors. If model has at least enough parameters, then all error
ACFs and PACFs will be insignificant.
(c) check AIC and SBC together with adj-R2 of estimated models to detect which is the
parsimonious one (i.e. one that minimizes AIC and SBC and has highest adj-R2).
Step 6 If changes in the original model are needed, go back to step 4.
