Econometrics - Exercise set 4

Exercise 1
Consider the following model, 𝑦 = 𝛽𝑋 + 𝜀

Which satisfies all the Gauss-Markov assumptions.

1.1
Explain what leverage is and show the formula

The explained component 𝑦̂ of 𝑦 is:

𝑦̂ = 𝑋𝑏
−1
Where 𝑏 = (𝑋 ′ 𝑋) 𝑋 ′ 𝑦. We can rewrite this as:

𝑦̂ = 𝑋𝑏 = 𝐻𝑦

Where we define the hat-matrix 𝐻 as:

−1
𝐻 = 𝑋(𝑋 ′ 𝑋) 𝑋 ′

The 𝑗th diagonal element of 𝐻 is:

−1
ℎ𝑗 = 𝑥𝑗′ (𝑋 ′ 𝑋) 𝑥𝑗 (5.54)

This value is called the leverage of the 𝑗th observation. A large leverage indicates that the value of 𝑥𝑗 is
unusual compared to the average of the variable 𝑥.

Page 1 af 16
1.2
Define outliers and briefly explain the impact on the OLS estimation

An observation is called an outlier if the value of the dependent variable 𝑦𝑗 differs enough from the general
pattern of the other observations.

Outliers disrupt the regression line and decreases 𝑅2 through changes in the parameter values.

1.3
Illustrate how outliers with high and low leverage affects OLS estimates

1. Outlier but no high leverage as the red dot does not have an extreme value 𝑥𝑗 .

Page 2 af 16
2. No outlier but high leverage. 𝑥𝑗 takes an extreme value but it follows the trend.

3. Both outlier and high leverage.

Page 3 af 16
Exercise 2
Load the dataset E42.RData into RStudio. We have been given a model that shows what effects income,

𝑔𝑑𝑝𝑖 = 𝛽0 + 𝛽1 𝑙𝑎𝑏𝑜𝑢𝑟𝑖 + 𝛽2 𝑒𝑑𝑢𝑖 + 𝛽3 𝑒𝑛𝑒𝑟𝑔𝑦𝑖 + 𝛽4 𝑐𝑎𝑝𝑖𝑡𝑎𝑙𝑖 + 𝜀𝑖

Now, we want to conduct a few tests. An explanation to the variables can be found in the appendix.

2.1
Estimate the model and interpret the results. Are the coefficients as expected?

Our expectations:

• Labour: Positive (higher employment → higher GDP)

• Education: Positive (higher accumulated level of education → higher GDP)
• Energy: Positive (higher energy consumption could resemble a more high-tech
economy)
• Capital: Positive (higher capital stock could lead to higher GDP, as predicted by e.g. the
Solow model).

All coefficients have the expected sign with the exception of capital, which turns out to be an insignificant
predictor of GDP.

Page 4 af 16
2.2
Explain the Breusch-Godfrey, Durbin-Watson, and the Ljung-Box test for autocorrelation and perform
these tests. (Hint: the 𝑯𝟎 -hypothesis is, that we have no autocorrelation)

Let the relation between the dependent and independent variables be specified by:

𝑦𝑖 = 𝑥𝑖′ 𝛽 + 𝜖𝑖 , 𝑖 = 1, … , 𝑛

Disturbances are autocorrelated if: 𝑐𝑜𝑟(𝜖𝑖 , 𝜖𝑗 ) ≠ 0 for 𝑖 ≠ 𝑗

For example, this could happen if we have omitted variables, functional misspecification, or neglected
dynamics.

Durbin-Watson

∑𝑛𝑖=2(𝑒𝑖 − 𝑒𝑖−1 )2
𝑑=
∑𝑛𝑖=1 𝑒𝑖2

The statistics satisfies 0 ≤ 𝑑 ≤ 4, where values close to:

• 0 indicate positive autocorrelation

• 2 indicates no autocorrelation
• 4 indicate negative autocorrelation

The null hypothesis in the DW-test states that autocorrelation is equal to zero:

𝐻0 : 𝜌 = 0, 𝐻1 : 𝜌 ≠ 0, 𝜌 = 𝑐𝑜𝑟(𝜖𝑖 , 𝜖𝑖−1 )

Breusch-Godfrey

The Breusch-Godfrey test is a Lagrange Multiplier test on serial correlation. Particularly, the test statistic is
obtained as 𝐿𝑀 = 𝑛𝑅2 from the auxiliary regression:

𝑒𝑖 = 𝑥𝑖′ 𝛿 + 𝛾𝑒𝑖−1 + ω𝑖 , 𝑖 = 2, … , 𝑛

The null hypothesis is no autocorrelation:

𝐻0 : 𝛾1 = ⋯ = 𝛾𝑝 = 0, 𝐻1 : 𝑓𝑜𝑟 𝑜𝑛𝑒 𝑜𝑟 𝑚𝑜𝑟𝑒 𝛾𝑖 ≠ 0

The test statistic has an asymptotic 𝜒 2 (1) distribution under the null hypothesis of no autocorrelation.

Procedure:

• Step 1: Apply OLS in the model 𝑦 = 𝑋𝛽 + 𝜖 and save residuals

Page 5 af 16
 • Step 2: Perform auxiliary regression: 𝑒𝑖 = 𝑥𝑖′ 𝛿 + 𝛾1 𝑒𝑖−1 + ⋯ + 𝛾𝑝 𝑒𝑖−𝑝 + 𝜔𝑖 , 𝑖 = 𝑝 + 1, … , 𝑛
• Step 3: 𝐿𝑀 = 𝑛𝑅2 of the aux. regression

Ljung-Box

The Ljung-Box test tests for the joint significance of the first 𝑝 autocorrelation coefficients defined.
𝑝
𝑛+2 2
𝐿𝐵 = 𝑛 · ∑ 𝑟 ≈ 𝜒 2 (𝑝)
𝑛−𝑘 𝑘
𝑘=1

Where:

∑𝑛𝑖=1 𝑒𝑖 𝑒𝑖−𝑘
𝑟𝑘 =
∑𝑛𝑖=1 𝑒𝑖2

Measures the correlation between residuals that are 𝑘 observations apart.

The null hypothesis states that there is no autocorrelation:

𝐻0 : 𝜌 = 0, 𝐻1 : 𝜌 ≠ 0

Conducting all three tests reveals the p-values below. We reject the null hypothesis in all three cases,
implying that our model is affected by autocorrelation.

Page 6 af 16
2.3
Explain and perform the tests for heteroskedasticity listed below. (Hint: the 𝑯𝟎 -hypothesis is, that we have
homoscedasticity)

Bresuch-Pagan

The null hypothesis of constant variance corresponds to the (𝑝 − 1) parameter restrictions:

𝛾2 = ⋯ = 𝛾𝑝 = 0

Which implies that all error variances are equal to zero.

Procedure:

• Step 1: Apply OLS in the model 𝑦 = 𝑋𝛽 + 𝜖 and compute residuals.

• Step 2: Perform auxiliary regression: 𝑒𝑖2 = 𝛾𝑖 + 𝛾2 𝑧2𝑖 + ⋯ + 𝛾𝑝 𝑧𝑝𝑖 + 𝜂𝑖
• Step 3: 𝐿𝑀 = 𝑛𝑅2 of the auxiliary regression.

Goldfeld-Quandt

Requires that the data can be ordered with non-decreasing variance and afterwards split into three groups.
The first group consists of 𝑛1 observations (with variance 𝜎12), the second group of the last 𝑛2 observations
(with variance 𝜎22), and the third group of the remaining 𝑛3 = 𝑛 − 𝑛1 − 𝑛2 observations in the middle. The
third group is omitted to create a sharper contrast between the variances of group 1 and 2.

The null and alternative hypotheses are:

𝐻0 : 𝜎12 = 𝜎22 , 𝐻1 : 𝜎22 > 𝜎12

White test (small)

This test is identical to the Breusch-Pagan test. However, in the small White test we include the explanatory
variables squared. The null hypothesis is identical to that of the Breusch-Pagan test.

White test (large)

In addition to the explanatory variables of the small White test, the large White test includes interactions
between all explanatory variables. The null hypothesis is identical to that of the Breusch-Pagan test.

Page 7 af 16
Page 8 af 16
Exercise 3
Load the dataset E43.RData into RStudio.

This dataset includes data for CAPM for the sector of cyclical and non-cyclical consumer goods. Estimate the
following models,

𝑟𝑒𝑛𝑑𝑐𝑦𝑐𝑜 = 𝛽0 + 𝛽1 𝑟𝑒𝑛𝑑𝑚𝑎𝑟𝑘 + 𝜀

𝑟𝑒𝑛𝑑𝑛𝑐𝑐𝑜 = 𝛽0 + 𝛽1 𝑟𝑒𝑛𝑑𝑚𝑎𝑟𝑘 + 𝜀

3.1
Perform tests of heteroskedasticity and serial correlation on both models

To test for serial correlation, I run the Durbin-Watson, Breusch-Godfrey, and the Ljung-Box test. The p-values
obtained for each of the tests (shown below) indicates that we cannot reject the null hypothesis, which
implies that we cannot reject no autocorrelation. Hence, we have no issues with autocorrelation in either of
our models.

To test for heteroskedasticity, I perform the Breusch-Pagan, Goldfeld-Quandt, and the small and large White
test. In general, the null hypothesis is 𝐻0 : 𝜎𝑖2 = 𝜎𝑗2 , ∀𝑖 ≠ 𝑗.

For the Goldfeld-Quandt, the null hypothesis is specified as 𝐻0 : 𝜎12 = 𝜎22 where 𝜎12 contains the first 𝑛1
observations with the smallest variance and 𝜎22 contains the last 𝑛2 observations with the highest variance.

In each of the models, and for each of the tests, we can reject the null hypothesis at a 95% significance level
since the respective p-values are all below 0.05. Hence, we conclude that there is heteroskedasticity in both
of our models.

3.2
Investigate the presence and nature of influential observations, and its effects on OLS. Use Example 5.27
on page 384 in Heij as reference

Page 9 af 16
Model 1

Leverage plots

Studentized residuals

Measures the effect on 𝑦̂ if we delete a specific observation.

𝑒𝑖
𝑒𝑖⋆ =
𝑠𝑖 √1 − ℎ𝑖

Where 𝑠𝑖2 is the residual variance including 𝐷𝑖 . Observation 𝑖 is an outlier if |𝑒𝑖⋆ | > 𝑡(𝑛 − 𝑘 − 1)

Page 10 af 16
Dfbetas

A Method for revealing influence of an observation 𝑖 on OLS is dfbetas. It involves estimating the parameters
(betas) with a dummy-variable (equal to one for observation 𝑖), and without the dummy 𝑏. Taking their
difference yields:

𝑏𝑙 − 𝑏𝑙 (𝑖)
𝑑𝑓𝑏𝑒𝑡𝑎𝑠𝑙𝑖 =
𝑠𝑖 √𝑎𝑙𝑙

−1
Where 𝑎𝑙𝑙 is the 𝑙th diagonal of (𝑋 ′ 𝑋) . Large values of 𝑑𝑓𝑏𝑒𝑡𝑎𝑠𝑙𝑖 indicates observations that are influential
in estimating a given parameter. Generally, we consider 2 as a general cut-off value to indicate influential
observations and 2/√𝑛 as a size-adjusted cutoff.

Page 11 af 16
Dffits
Another method for revealing outliers, which is similar to dfbetas but measures the effect on the fitted
values.

ℎ𝑖
𝑑𝑓𝑓𝑖𝑡𝑠𝑖 = 𝑒𝑖⋆ √
1 − ℎ𝑖

Values larger than 2 · √𝑘/𝑛 indicate outliers.

Model 2

Leverage plots

Page 12 af 16
Studentized residuals

Dfbetas

Page 13 af 16
Dffits

Page 14 af 16
3.3
Discuss the relevance of your findings in the first part on the detection of influential observations

Model 1

We have identified many high-leverage observations, which affect the beta coefficients and fitted values.
Luckily, these are not simultaneously outliers which would alter the model fit substantially.

Model 2
Multiple observations affect the fitted values but, again, these do not necessarily meet both the high-
leverage and outlier criteria simultaneously.

Page 15 af 16
Appendix
Exercise 2Variables
The file “exam_2020_21” contains the following data, observed annually 1959-2000, for Sweden:

Year

Gdp Gross Domestic Product, million Swedish krones

Dgdp First difference of Gross Domestic Product, million Swedish krones

Labour Employment, in thousands people

Capital Capital stock, million Swedish krones

Edu Accumulated years of schooling, in thousands years

Energy Total energy consumption, petajoules

Time Time trend, 1,2,3,…

Page 16 af 16