Definition of Simple Regression Model

Ordinary Least Squares

Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Chapter 2
The Simple Regression Model

Le Van Chon

International University, Vietnam National University HCMC

April 2015

Based on Introductory Econometrics: A Modern Approach

by Wooldridge

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Definition of Simple Regression Model

Applied econometric analysis often begins with 2 variables y
and x.
We want to “study how y varies with changes in x”.
E.g., x is years of education, y is hourly wage.
x is number of police officers, y is a community crime rate.
In the simple linear regression model:
y = β0 + β1 x + u (1)
y is called the dependent variable, the explained variable, or
the regressand.
x is called the independent variable, the explanatory variable,
or the regressor.
u, called error term or disturbance, represents factors other
than x that affect y . u stands for “unobserved”.
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Definition of Simple Regression Model (cont.)

If the other factors in u are held fixed, ∆u = 0, then x has a

linear effect on y : ∆y = β1 ∆x

β1 is the slope parameter. This is of primary interest in

applied economics.

One-unit change in x has the same effect on y , regardless of

the initial value of x. → Unrealistic.

E.g., wage-education example, we might want to allow for

increasing returns.

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Definition of Simple Regression Model (cont.)

An assumption: the average value of u in the population is
zero.
E (u) = 0 (2)
This assumption is not restrictive since we can always use β0
to normalize E (u) to 0.
Because u and x are random variables, we can define
conditional distribution of u given any value of x.
Crucial assumption: average value of u does not depend on
x.
E (u|x) = E (u) (3)

(2) + (3) → the zero conditional mean assumption.

This implies E (y |x) = β0 + β1 x
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Definition of Simple Regression Model (cont.)

Population regression function (PRF): E (y |x) is a linear
function of x. For any value of x, the distribution of y is
centered about E (y |x).

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares

How to estimate population parameters β0 and β1 from a

sample?

Let {(xi , yi ) : i = 1, 2, ..., n} denote a random sample of size n

from the population.

For each observation in this sample, it will be the case that

yi = β0 + β1 xi + ui

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

PRF, sample data points and the associated error terms:

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

To derive the OLS estimates, we need to realize that our main
assumption of E (u|x) = E (u) = 0 also implies that
Cov (x, u) = E (xu) = 0 (4)
Why? Cov (x, u) = E (xu) − E (x)E (u) = Ex [E (xu|x)] =
Ex [xE (u|x)] = 0.
We can write 2 restrictions (2) and (4) in terms of x, y , β0
and β1
E (y − β0 − β1 x) = 0 (5)
E [x(y − β0 − β1 x)] = 0 (6)

(5) and (6) are 2 moment restrictions with 2 unknown

parameters. → They can be used to obtain good estimators
of β0 and β1 .
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

Method of moments approach to estimation implies
imposing the population moment restrictions on the sample
moments.
Given a sample, we choose estimates βˆ0 and βˆ1 to solve the
sample versions:
n
1X
(yi − βˆ0 − βˆ1 xi ) = 0 (7)
n
i=1
n
1 X
xi (yi − βˆ0 − βˆ1 xi ) = 0 (8)
n
i=1
Given the properties of summation, (7) can be rewritten as
ȳ = βˆ0 + βˆ1 x̄ (9)
or βˆ0 = ȳ − βˆ1 x̄ (10)
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

Drop 1/n Pin (8) and plug (10) into (8):

n ˆ ˆ
Pni=1 xi (yi − [ȳ − β1 x̄]P−n β1 xi ) = 0
ˆ
Pn i=1 xi (yi − ȳ ) = β1 Pi=1 xi (xi − x̄)
ˆ n (xi − x̄)2
i=1 (xi − x̄)(yi − ȳ ) = β1 i=1

Provided that
n
X
(xi − x̄)2 > 0 (11)
i=1

the estimated slope is

Pn
(x − x̄)(yi − ȳ )
Pn i
βˆ1 = i=1 2
(12)
i=1 (xi − x̄)

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

Summary of OLS slope estimate:

- Slope estimate is the sample covariance between x and y
divided by the sample variance of x.
- If x and y are positively correlated, the slope will be positive.
- If x and y are negatively correlated, the slope will be
negative.
-Only need x to vary in the sample.

βˆ0 and βˆ1 given in (10) and (12) are called the ordinary least
squares (OLS) estimates of β0 and β1 .

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

To justify this name, for any βˆ0 and βˆ1 , define a fitted value
for y given x = xi :
ŷi = βˆ0 + βˆ1 xi (13)

The residual for observation i is the difference between the

actual yi and its fitted value:
ûi = yi − ŷi = yi − βˆ0 − βˆ1 xi
Intuitively, OLS is fitting a line through the sample points
such that the sum of squared residuals is as small as possible
→ term “ordinary least squares”.
Formal minimization problem:
X n X n
min 2
ûi = (yi − βˆ0 − βˆ1 xi )2 (14)
βˆ0 ,βˆ1 i=1 i=1

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

Sample regression line, sample data points and residuals:

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

To solve (14), we obtain 2 first order conditions, which are the
same as (7) and (8), multiplied by n.

Once we have determined the OLS βˆ0 and βˆ1 , we have the
OLS regression line:

ŷi = βˆ0 + βˆ1 xi (15)

(15) is also called the sample regression function (SRF)

because it is the estimated version of the population
regression function (PRF) E (y |x) = β0 + β1 x.

Remember that PRF is fixed but unknown.

Different samples generate different SRFs.
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

Slope estimate βˆ1 is of primary interest. It tells us the amount
by which ŷ changes when x increases by 1 unit.
∆ŷ = βˆ1 ∆x
E.g., we study the relationship between firm performance and
CEO compensation.
salary = β0 + β1 roe + u
salary = CEOs annual salary in thousands of dollars,
roe = average return (%) on the firm’s equity for previous 3
years.
Because a higher roe is good for the firm, we think β1 > 0.
CEOSAL1 contains information on 209 CEOs in 1990.
OLS regression line:
\ = 963.191 + 18.501roe
salary (16)
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Ordinary Least Squares (cont.)

E.g., for the population of the workforce in 1976, let
y = wage, $ per hour,
x = educ, years of schooling.

Using data in WAGE1 with 526 observations, we obtain the

OLS regression line:
[ = −0.90 + 0.54educ
wage (17)

Implication of the intercept? Why?

Only 18 people in the sample have less than 8 years of
education. → the regression line does poorly at very low
levels.
Implication of the slope?
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Fitted Values and Residuals

Given βˆ0 and βˆ1 , we can obtain the fitted value ŷi for each
observation. Each ŷi is on the OLS regression line.

OLS residual associated with observation i, ûi , is the

difference between yi and its fitted value.
If ûi is positive, the line underpredicts yi .
If ûi is negative, the line overpredicts yi .

In most cases, every ûi 6= 0, none of the data points lie on the
OLS line.

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Algebraic Properties of OLS Statistics

(1) The sum and thus the sample average of the OLS
residuals is zero.
n n
X 1X
ûi = 0 and thus ûi = 0
n
i=1 i=1

(2) The sample covariance between the regressors and the

OLS residuals is zero.
Xn
xi ûi = 0
i=1

(3) The OLS regression line always goes through the mean of
the sample.
ȳ = βˆ0 + βˆ1 x̄
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Algebraic Properties of OLS Statistics (cont.)

We can think of each observation i as being made up of an
explained part and an unexplained part yi = ŷi + ûi
We define the following:
Xn
(yi − ȳ )2 is the total sum of squares (SST),
i=1
n
X
(yˆi − ȳ )2 is the explained sum of squares (SSE),
i=1
n
X
ûi 2 is the residual sum of squares (SSR).
i=1

Then
SST = SSE + SSR (18)
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Algebraic Properties of OLS Statistics (cont.)

Proof:
Xn n
X n
X
2 2
(yi − ȳ ) = [(yi − yˆi ) + (yˆi − ȳ )] = [ûi + (yˆi − ȳ )]2
i=1 i=1 i=1
Xn n
X Xn
2
= ûi + 2 ûi (yˆi − ȳ ) + (yˆi − ȳ )2
i=1 i=1 i=1
n
X
= SSR + 2 ûi (yˆi − ȳ ) + SSE
i=1

and we know that

n
X
ûi (yˆi − ȳ ) = 0
i=1

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Goodness-of-Fit
How well the OLS regression line fits the data?
Divide (18) by SST to get:
SSE SSR
1= +
SST SST

The R-squared of the regression or the coefficient of

determination
SSE SSR
R2 ≡ =1− (19)
SST SST
It implies the fraction of the sample variation in y that is
explained by the model.
0 ≤ R2 ≤ 1
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Goodness-of-Fit (cont.)

E.g., CEOSAL1. roe explains only about 1.3% of the variation

in salaries for this sample.
→ 98.7% of the salary variations for these CEOs is left
unexplained!

Notice that a seemingly low R 2 does not mean that an OLS

regression equation is useless.
It is still possible that (16) is a good estimate of the ceteris
paribus relationship between salary and roe.

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Units of Measurement on OLS Statistics

OLS estimates change when the units of measurement of the
dependent and independent variables change.
E.g., CEOSAL1. Rather than measuring salary in $’000, we
measure it in $, salardol = 1,000×salary .
Without regression, we know that
\ = 963, 191 + 18, 501roe.
salardol (20)

Multiply the intercept and the slope in (16) by 1,000 → (16)

and (20) have the same interpretations.
Define roedec = roe/100 where roedec is a decimal.
\ = 963.191 + 1850.1roedec.
salary (21)
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Units of Measurement on OLS Statistics (cont.)

What happens to R 2 when units of measurement change?

Nothing.

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Nonlinearities in Simple Regression

It is rather easy to incorporate many nonlinearities into simple

regression analysis by appropriately defining y and x.

E.g., WAGE1. βˆ1 of 0.54 means that each additional year of

education increases wage by 54 cents. → maybe not
reasonable.

Suppose that the percentage increase in wage is the same

given one more year of education.
(17) does not imply a constant percentage increase.

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Nonlinearities in Simple Regression (cont.)

New model:
log (wage) = β0 + β1 educ + u (22)
where log(.) denotes the natural logarithm.
For each additional year of education, the percentage change
in wage is the same. → the change in wage increases.
(22) implies an increasing return to education.
Estimating this model and the mechanics of OLS are the
same:
log\
(wage) = 0.584 + 0.083educ (23)
wage increases by 8.3 percent for every additional year of
educ.
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Nonlinearities in Simple Regression (cont.)

Another important use of the natural log is in obtaining a

constant elasticity model.

E.g., CEOSAL1. We can estimate a constant elasticity model

relating CEO salary ($’000) to firm sales ($mil):

log (salary ) = β0 + β1 log (sales) + u (24)

where β1 is the elasticity of salary with respect to sales.

If we change the units of measurement of y , what happens to

β1 ?
Nothing.

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Meaning of Linear Regression

We have seen a model that allows for nonlinear relationships.

So what does “linear” mean?

An equation y = β0 + β1 x + u is linear in parameters, β0 and

β1 .
There are no restrictions on how y and x relate to the original
dependent and independent variables.

Plenty of models cannot be cast as linear regression models

because they are not linear in their parameters.

E.g., cons = 1/(β0 + β1 inc) + u

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS
Unbiasedness of OLS is established under a set of assumptions:
Assumption SLR.1 (Linear in Parameters)
The population model is linear in parameters as

y = β0 + β1 x + u (25)

where β0 and β1 are the population intercept and slope

parameters.

Realistically, y , x, u are all viewed as random variables.

Assumption SLR.2 (Random Sampling)

We can use a random sample of size n, (xi , yi ) : i = 1, 2, , n,
from the population model.
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS (cont.)

Not all cross-sectional samples can be viewed as random

samples, but many may be.

We can write (25) in terms of the random sample as

yi = β0 + β1 xi + ui , i = 1, 2, ..., n (26)

To obtain unbiased estimators of β0 and β1 , we need to

impose
Assumption SLR.3 (Zero Conditional Mean)
E (u|x) = 0
This assumption implies E (ui |xi ) = 0 for all i = 1, 2, ..., n.
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS (cont.)

Assumption SLR.4 (Sample Variation in the Independent
Variable)
In the sample, xi , i = 1, 2, ..., n are not all equal to a constant.
This assumption is equivalent to ni=1 (xi − x̄)2 > 0
P

From (12):
Pn Pn
(xi − x̄)(y i − ȳ ) i=1 (xi − x̄)yi
βˆ1 = i=1 Pn 2
= P n 2
i=1 (xi − x̄) i=1 (xi − x̄)

Plug (26) into this:

Pn Pn
(xi − x̄)(β 0 + β 1 xi + u i ) i=1 (xi − x̄)ui
βˆ1 = i=1 Pn 2
= β1 +
i=1 (xi − x̄) SSTx

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS (cont.)

ui ’s are generally different from 0. → βˆ1 differs from β1 .
The first important statistical property of OLS:
Theorem 2.1 (Unbiasedness of OLS)
Using Assumptions SLR.1 through SLR.4,
E (βˆ0 ) = β0 , and E (βˆ1 ) = β1 (27)
The OLS estimates of β0 and β1 are unbiased.
Proof:
n
X
E (βˆ1 ) = β1 + E [(1/SSTx ) (xi − x̄)ui ]
i=1
n
X
= β1 + (1/SSTx ) (xi − x̄)E (ui ) = β1
i=1
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS (cont.)

(10) implies
β̂0 = ȳ − βˆ1 x̄ = β0 + β1 x̄ + ū − β̂1 x̄ = β0 + (β1 − βˆ1 )x̄ + ū
E (β̂0 ) = β0 + E [(β1 − βˆ1 )x̄] = β0

Remember unbiasedness is a feature of the sampling

distributions of β̂0 and β̂1 . It says nothing about the estimate
we obtain for a given sample.
If any of four assumptions fails, then OLS is not necessarily
unbiased.
When u contains factors affecting y that are also correlated
with x, it can result in spurious correlation.
Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Unbiasedness of OLS (cont.)

E.g., let math10 denote % of tenth graders at a high school
receiving a passing score on a standardized math exam.
Let lnchprg denote % of students eligible for the federally
funded school lunch program.
We expect the lunch program has a positive effect on
performance:
math10 = β0 + β1 lnchprg + u
MEAP93 has data on 408 Michigan high schools for the
1992-1993 school year.
\ = 32.14 − 0.319lnchprg
math10
Why? u contains such as the poverty rate of children
attending school, which affects student performance and is
highly correlated with eligibility in the lunch program.
Le Van Chon Applied Econometrics
 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators

Now we know that the sampling distribution of our estimate is

centered about the true parameter.
How spread out is this distribution? → the variance.

We need to add an assumption.

Assumption SLR.5 (Homoskedasticity)
Var(u|x) = σ 2

This assumption is distinct from Assumption SLR.3:

E (u|x) = 0.
This assumption simplifies the variance calculations for β̂0 and
β̂1 and it implies OLS has certain efficiency properties.

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators (cont.)

Var(u|x) = E (u 2 |x) − [E (u|x)]2 = E (u 2 |x) = σ 2

→ Var(u) = E (u 2 ) = σ 2

σ 2 is often called the error variance.

σ, the square root of the error variance, is called the standard
deviation of the error.

We can say that

E (y |x) = β0 + β1 x (28)
Var(y |x) = σ 2 (29)

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators (cont.)

Homoskedastic case:

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators (cont.)

Heteroskedastic case:

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators (cont.)

Theorem 2.2 (Sampling variances of the OLS estimators)

Under Assumptions SLR.1 through SLR.5,

σ2 σ2
Var(β̂1 ) = Pn 2
= (30)
i=1 (x i − x̄) SSTx
σ 2 n1 ni=1 xi2
P
Var(β̂0 ) = Pn 2
(31)
i=1 (xi − x̄)

Proof:
n
1 X 2 SSTx 2 σ2
Var(β̂1 ) = (xi − x̄) Var(ui ) = σ =
SSTx2 SSTx2 SSTx
i=1

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Variances of the OLS Estimators (cont.)

(30) and (31) are invalid in the presence of heteroskedasticity.

(30) and (31) imply that:

(i) The larger the error variance, the larger are Var(β̂j ).
(ii) The larger the variability in the xi , the smaller are Var(β̂j ).

Problem: the error variance σ 2 is unknown because we don’t

observe the errors, ui .

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Estimating the Error Variance

What we observe are the residuals, ûi . We can use the

residuals to form an estimate of the error variance.

We write the residuals as a function of the errors:

ûi = yi − β̂0 − β̂1 xi = (β0 + β1 xi + ui ) − β̂0 − β̂1 xi

ûi = ui − (β̂0 − β0 ) − (β̂1 − β1 )xi (32)

An unbiased estimator of σ 2 is
n
1 X 2 SSR
σ̂ 2 = ûi = (33)
n−2 n−2
i=1

Le Van Chon Applied Econometrics

Definition of Simple Regression Model

Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Estimating the Error Variance (cont.)

√
σ̂ = σ̂ 2 = standard error of the regression (SER).

Recall that
σ
sd(β̂1 ) = √ ,
SSTx
if we substitute σ̂ 2 for σ 2 , then we have the standard error of
β̂1 :
σ̂ σ̂
se(β̂1 ) = √ = pPn
SSTx 2
i=1 (xi − x̄)

Le Van Chon Applied Econometrics

 Definition of Simple Regression Model
Ordinary Least Squares
Mechanics of OLS
Units of Measurement and Functional Form
Expected Values and Variances of OLS Estimators
Regression through the Origin

Regression through the Origin

In rare cases, we impose the restriction that when x = 0,
E (y |0) = 0.
E.g., if income (x) is zero, income tax revenues (y ) must also
be zero.
Equation y = β̃1 x + ũ (34)
Obtaining (34) is called regression through the origin.
We still use OLS method with the corresponding first order
condition
n Pn
xi yi
xi (yi − β̃ˆ1 xi ) = 0 ⇒ β̃ˆ1 = Pi=1
X
n 2
(35)
x
i=1 i
i=1

If β0 6= 0, then β̃ˆ1 is a biased estimator of β1 .

Le Van Chon Applied Econometrics