FNCE 926

Empirical Methods in CF
Lecture 2 – Linear Regression II

Professor Todd Gormley

Today's Agenda
n  Quick review
n  Finish discussion of linear regression
q  Hypothesis testing
n  Standard errors
n  Robustness, etc.
q  Miscellaneous issues
n  Multicollinearity
n  Interactions

n  Presentations of "Classics #1"

2
Background readings
n  Angrist and Pischke
q  Sections 3.1-3.2, 3.4.1
n  Wooldridge
q  Sections 4.1 & 4.2
n  Greene
q  Chapter 3 and Sections 4.1-4.4, 5.7-5.9, 6.1-6.2

3
Announcements
n  Exercise #1 is due next week
q  You can download it from Canvas
q  If any questions, please e-mail TA, or if
necessary, feel free to e-mail me
q  When finished, upload both typed
answers and DO file to Canvas

4
Quick Review [Part 1]
n  When does the CEF, E(y|x), we approx.
with OLS give causal inferences?
q  Answer = If correlation between error, u, and
independent variables, x's, is zero

n  How do we test for whether this is true?

q  Trick question! You can't test it. The error is
unobserved. Need to rely on sound logic.

5
Quick Review [Part 2]
n  What is interpretation of coefficients
in a log-log regression?
q  Answer = Elasticity. It captures the percent
change in y for a percent change in x

n  What happens if rescale log variables?

q  Answer = The constant will change

6
Quick Review [Part 3]
n  How should I interpret coefficient on x1 in a
multivariate regression? And, what two steps
could I use to get this?
q  Answer = Effect of x1 holding other x's constant
q  Can get same estimates in two steps by first
partialing out some variables and regressing
residuals on residuals in second step

7
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
q  Heteroskedastic versus Homoskedastic errors
q  Hypothesis tests
q  Economic versus statistical significance

n  Miscellaneous issues

8
Hypothesis testing
n  Before getting to hypothesis testing, which
allows us to say something like "our
estimate is statistically significant", it is
helpful to first look at OLS variance
q  Understanding it and the assumptions made to
get it can help us get the right standard errors
for our later hypothesis tests

9
Variance of OLS Estimators
n  Homoskedasticity implies Var(u|x) = σ2
q  I.e. Variance of disturbances, u, doesn't
depend on level of observed x
n  Heteroskedasticity implies Var(u|x) = f(x)
q  I.e. Variance of disturbances, u, does depend
on level of x in some way

10
Variance visually…

Homoskedasticity Heteroskedasticity

11
Which assumption is more realistic?
n  In investment regression, which is more realistic,
homoskedasticity or heteroskedasticity?

Investment = α + βQ + u

q  Answer: Heteroskedasticity seems like a much safer

assumption to make; not hard to come up with stories
on why homoskedasticity is violated

12
Heteroskedasticity (HEK) and bias
n  Does heteroskedasticity cause bias?
q  Answer = No! E(u|x)=0 (which is what we need
for consistent estimates) is something entirely
different. Hetereskedasticity just affects SEs!
q  Heteroskedasticity just means that the OLS
estimate may no longer be the most efficient (i.e.
precise) linear estimator

n  So, why do we care about HEK?

13
Default is homoskedastic (HOK) SEs
n  Default standard errors reported by
programs like Stata assume HOK
q  If standard errors are heteroskedastic,
statistical inferences made from these
standard errors might be incorrect…

q  How do we correct for this?

14
Robust standard errors (SEs)
n  Use "robust" option to get standard
errors (for hypothesis testing ) that are
robust to heteroskedasticity
q  Typically increases SE, but usually won't
make that big of a deal in practice
q  If standard errors go down, could have
problem; use the larger standard errors!
q  We will talk about clustering later…

15
Using WLS to deal with HEK
n  Weighted least squares (WLS) is sometimes
used when worried about heteroskedasticity
q  WLS basically weights the observation of x using
an estimate of the variance at that value of x
q  Done correctly, can improve precision of estimates

16
WLS continued… a recommendation
n  Recommendation of Angrist-Pischke
[See Section 3.4.1]: don't bother with WLS
q  OLS is consistent, so why bother?
Can just use robust standard errors
q  Finite sample properties can be bad [and it may
not actually be more efficient]
q  Harder to interpret than just using OLS [which
is still best linear approx. of CEF]

17
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
q  Heteroskedastic versus Homoskedastic errors
q  Hypothesis tests
q  Economic versus statistical significance

n  Miscellaneous issues

18
Hypothesis tests
n  This type of phrases are common: "The
estimate, βˆ , is statistically significant"
q  What does this mean?
q  Answer = "Statistical significance" is
generally meant to imply an estimate is
statistically different than zero

But, where does this come from?

19
Hypothesis tests[Part 2]
n  When thinking about significance, it is
helpful to remember a few things…
q  Estimates of β1, β2, etc. are functions of random
variables; thus, they are random variables with
variances and covariances with each other
q  These variances & covariances can be estimated
[See textbooks for various derivations]
q  Standard error is just the square root of an
estimate's estimated variance

20
Hypothesis tests[Part 3]
n  Reported t-stat is just telling us how
many standard deviations our sample
estimate, βˆ , is from zero
q  I.e. it is testing the null hypothesis: β = 0
q  p-value is just the likelihood that we would
get an estimate βˆ standard deviations away
from zero by luck if the true β = 0

21
Hypothesis tests[Part 4]
n  See textbooks for more details on how to
do other hypothesis tests; E.g.
q  β1 = β 2

q  β1 = β 2 = β 3 = 0

q  Given these are generally easily done in

programs like Stata, I don't want to
spend time going over the math

22
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
q  Heteroskedastic versus Homoskedastic errors
q  Hypothesis tests
q  Economic versus statistical significance

n  Miscellaneous issues

23
Statistical vs. Economic Significance
n  These are not the same!
q  Coefficient might be statistically
significant, but economically small
n  You can get this in large samples, or when
you have a lot of variation in x (or outliers)

q  Coefficient might be economically large,

but statistically insignificant
n  Might just be small sample size or too little
variation in x to get precise estimate

24
Economic Significance
n  You should always check economic
significance of coefficients
q  E.g. how large is the implied change in y
for a standard deviation change in x?
q  And importantly, is that plausible? If not,
you might have a specification problem

25
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
n  Miscellaneous issues
q  Irrelevant regressors & multicollinearity
q  Binary models and interactions
q  Reporting regressions

26
Irrelevant regressors

n  What happens if include a regressor that

should not be in the model?
q  We estimate y = β0 + β1x1 + β2x2 + u
q  But, real model is y = β0 + β1x1 + u
q  Answer: We still get a consistent of all the β,
where β2 = 0, but our standard errors might
go up (making it harder to find statistically
significant effects)… see next few slides

27
Variance and of OLS estimators
n  Greater variance in your estimates, βˆ j ,
increases your standard errors, making it
harder to find statistically significant estimates

n  So, useful to know what increases Var βˆ j ( )

28
Variance formula
n  Sampling variance of OLS slope is…

σ2
( )
Var βˆ j =
− x j ) (1 − R
∑ (x )
N 2 2
i =1 ij j

for j = 1,…, k, where Rj2 is the R2 from

regressing xj on all other independent variables
including the intercept and σ2 is the variance of
the regression error, u

29
Variance formula – Interpretation
n  How will more variation in x affect SE? Why?
n  How will higher σ2 affect SE? Why?
n  How will higher Rj2 affect SE? Why?

σ
( )
2
Var βˆ j =
∑ i=1( ij j ) ( j )
N 2
x − x 1 − R 2

30
Variance formula – Variation in xj
n  More variation in xj is good; smaller SE!
q  Intuitive; more variation in xj helps us
identify its effect on y!
q  This is why we always want larger samples;
it will give us more variation in xj

31
Variance formula – Effect of σ2
n  More error variance means bigger SE
q  Intuitive; a lot of the variation in y is
explained by things you didn't model
q  Can add variables that affect y (even if not
necessary for identification) to improve fit!

32
Variance formula – Effect of Rj2
n  But, more variables can also be bad if
they are highly collinear
q  Gets harder to disentangle effect of the
variables that are highly collinear
q  This is why we don't want to add variables
that are "irrelevant" (i.e. they don't affect y)

Should we include variables that do explain y and

are highly correlated with our x of interest?

33
Multicollinearity [Part 1]
n  Highly collinear variables can inflate SEs
q  But, it does not cause a bias or inconsistency!
q  Problem is really just one of a having too small
of a sample; with a larger sample, one could get
more variation in the independent variables
and get more precise estimates

34
Multicollinearity [Part 2]
n  Consider the following model
y = β0 + β1 x1 + β 2 x2 + β3 x3 + u
where x2 and x3 are highly correlated

q  ( ) ( )
Var βˆ2 and Var βˆ3 may be large, but
correlation between x2 and x3 has no
( )
direct effect on Var βˆ1
q  If x1 is uncorrelated with x2 and x3, the
( )
R12 = 0 and Var βˆ1 unaffected

35
Multicollinearity – Key Takeaways
n  It doesn't cause bias
n  Don't include controls that are highly
correlated with independent variable of
interest if they aren't needed for
identification [i.e. E(u|x) = 0 without them]
q  But obviously, if E(u|x) ≠ 0 without these
controls, you need them!
q  A larger sample will help increase precision

36
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
n  Miscellaneous issues
q  Irrelevant regressors & multicollinearity
q  Binary models and interactions
q  Reporting regressions

37
Models with interactions
n  Sometimes, it is helpful for identification, to
add interactions between x's
q  Ex. – theory suggests firms with a high value of x1
should be more affected by some change in x2
q  E.g. see Rajan and Zingales (1998)

n  The model will look something like…

y = β0 + β1 x1 + β2 x2 + β3 x1 x2 + u

38
Interactions – Interpretation [Part 1]
n  According to this model, what is the effect of
increasing x1 on y, holding all else equal?
y = β 0 + β1 x1 + β 2 x2 + β3 x1 x2 + u
q  Answer:
Δy = ( β1 + β3 x2 ) Δx1
dy
= β1 + β3 x2
dx1

39
Interactions – Interpretation [Part 2]
n  If β3 < 0, how does a higher x2 affect the
partial effect of x1 on y?
dy
= β1 + β3 x2
dx1
q  Answer: The increase in y for a given change in
x1 will be smaller in levels (not necessarily in
absolute magnitude) for firms with a higher x2

40
Interactions – Interpretation [Part 3]
n  Suppose, β1 > 0 and β3 < 0 … what is
the sign of the effect of an increase in x1
for the average firm in the population?
dy
= β1 + β3 x2
dx1

dy x2 = x2
q  Answer: It is the sign of | = β1 + β3 x2
dx1

41
A very common mistake! [Part 1]

q  Researcher claims that "since β1>0 and β3<0, an

increase in x1 increases y on for the average firm, but
the increase is less for firms with a high x2"
dy x2 = x2
| = β1 + β3 x2
dx1
n  Wrong!!! The average effect of an increase in x1
might actually be negative if x2 is very large!
n  β1 only captures partial effect when x2 = 0, which
might not even make sense if x2 is never 0!

42
A very common mistake! [Part 2]
n  To improve interpretation of β1, you can
reparameterize the model by demeaning
each variable in the model, and estimate
y! = δ 0 + δ 1x!1 + δ 2 x!2 + δ 3 x!1x!2 + u
where y! = y − µ y
x!1 = x1 − µ x
1

x!2 = x2 − µ x
2

43
A very common mistake! [Part 3]
n  You can then show… Δy = (δ 1 + δ 3 x!2 ) Δx1
dy x2 =µ2
and thus, | = δ1 + δ 3 ( x2 − µ2 )
dx1
dy x2 =µ2
| = δ1
dx1

n  Now, the coefficient on the demeaned x1 can

be interpreted as effect of x1 for avg. firm!

44
The main takeaway – Summary
n  If you want to coefficients on non-
interacted variables to reflect the effect
of that variable for the "average" firm,
demean all your variables before
running the specification

n  Why is there so much confusion about this?

Probably because of indicator variables…

45
Indicator (binary) variables

n  We will now talk about indicator variables

q  Interpretation of the indicator variables
q  Interpretation when you interact them
q  When demeaning is helpful
q  When using an indicator rather than a
continuous variable might make sense

46
Motivation
n  Indicator variables, also known as binary
variables, are quite popular these days
q  Ex. #1 – Sex of CEO (male, female)
q  Ex. #2 – Employment status (employed, unemployed)
q  Also see in many diff-in-diff specifications
n  Ex. #1 – Size of firm (above vs. below median)
n  Ex. #2 – Pay of CEO (above vs. below median)

47
How they work
n  Code the information using dummy variable
⎧1 if person i is male
q  Ex. #1: Malei = ⎨
⎩0 otherwise
⎧1 if Ln(assets) of firm i > median
q  Ex. #2: Largei = ⎨
⎩0 otherwise

n  Choice of 0 or 1 is relevant only for interpretation

48
Single dummy variable model
n  Consider wage = β0 + δ 0 female + β1educ + u
n  δ0 measures difference in wage between male
and female given same level of education
q  E(wage|female = 0, educ) = β0 + β1educ
q  E(wage|female = 1, educ) = β0 + δ0 + β1educ
q  Thus, E(wage|f = 1, educ) – E(wage|f = 0, educ) = δ0

n  Intercept for males = β0 , females = β0 + δ0

49
Single dummy just shifts intercept!
n  When δ0 < 0, we have visually…

β1

50
Single dummy example – Wages
n  Suppose we estimate the following wage model

Wage = -1.57 – 1.8female + 0.57educ + 0.03exp + 0.14tenure

q  Male intercept is -1.57; it is meaningless, why?

q  How should we interpret the 1.8 coefficient?
n  Answer: Females earn $1.80/hour less then men
with same education, experience, and tenure

51
Log dependent variable & indicators
n  Nothing new; coefficient on indicator has %
interpretation. Consider following example…
ln( price) = −1.35 + 0.17ln(lotsize) + 0.71ln( sqrft )
+0.03bdrms + 0.054colonial
q  Again, negative intercept meaningless; all other
variables are never all equal to zero
q  Interpretation = colonial style home costs about
5.4% more than "otherwise similar" homes

52
Multiple indicator variables
n  Suppose you want to know how much lower
wages are for married and single females
q  Now have 4 possible outcomes
n  Single & male
n  Married & male
n  Single & female
n  Married & female

q  To estimate, create indicators for three of the

variables and add them to the regression

53
But, which to exclude?
n  We have to exclude one of the four
because they are perfectly collinear with
the intercept, but does it matter which?
q  Answer: No, not really. It just effects the
interpretation. Estimates of included
indicators will be relative to excluded indicator
q  For example, if we exclude "single & male",
we are estimating partial change in wage
relative to that of single males

54
But, which to exclude? [Part 2]

n  Note: if you don't exclude one, then

statistical programs like Stata will just
drop one for you automatically. For
interpretation, you need to figure out
which one was dropped!

55
 Multiple indicators – Example
n  Consider the following estimation results…
ln( wage) = 0.3 + 0.2l marriedMale − .20marriedFemale
−0.11singleFemale + 0.08education

q  I omitted single male; thus intercept is for single males

q  And, can interpret other coefficients as…
n  Married men earn ≈ 21% more than single males, all else equal
n  Married women earn ≈ 20% less than single males, all else equal

56
Interactions with Indicators
n  We could also do prior regression instead
using interactions between indicators
q  I.e. construct just two indicators, 'female' and
'married' and estimate the following

ln( wage) = β 0 + β1 female + β 2 married

+ β3 ( female × married ) + β 4education

q  How will our estimates and interpretation

differ from earlier estimates?

57
Interactions with Indicators [Part 2]
n  Before we had,
ln( wage) = 0.3 + 0.2l marriedMale − .20marriedFemale
−0.11singleFemale + 0.08education
n  Now, we will have,
ln( wage) = 0.3 − 0.11 female + 0.21married
−0.30 ( female × married ) + 0.08education

q  Question: Before, married females had wages

that were 0.20 lower; how much lower are
wages of married females now?

58
Interactions with Indicators [Part 3]
n  Answer: It will be the same!
ln( wage) = 0.32 − 0.11 female + 0.21married
−0.30 ( female × married ) + ...

q  Difference for married female = –0.11+0.21–

0.30 = -0.20; exactly the same as before

n  Bottom line = you can do the indicators

either way; inference is unaffected

59
Indicator Interactions – Example
n  Krueger (1993) found…
ln( wage) = βˆ0 + 0.18compwork + 0.07comphome
+0.02 ( compwork × comphome ) + ...

q  Excluded category = people with no computer

q  How do we interpret these estimates?
n  How much higher are wages if have computer at work? ≈18%
n  If have computer at home? ≈7%
n  If have computers at both work and home? ≈18+7+2=27%

60
Indicator Interactions – Example [part 2]
n  Remember, these are just approximate percent
changes… To get true change, need to convert
q  E.g. % change in wages for having computers at both
home and work is given by
100*[exp(0.18+0.07+0.02) – 1] = 31%

61
Interacting Indicators w/ Continuous
n  Adding dummies alone will only shift
intercepts for different groups
n  But, if we interact these dummies with
continuous variables, we can get different
slopes for different groups as well
q  See next slide for an example of this

62
Continuous Interactions – Example
n  Consider the following
ln( wage) = β0 + δ 0 female + β1educ + δ1 ( female × educ ) + u

q  What is intercept for males? β0

q  What is slope for males? β1
q  What is intercept for females? β0+δ0
q  What is slope for females? β1+δ1

63
Visual #1 of Example
ln( wage) = β0 + δ 0 female + β1educ + δ1 ( female × educ ) + u

In this example…
q  Females earn lower wages
at all levels of education
q  Avg. increase per unit of
education is also lower

64
Visual #2 of Example
ln( wage) = β0 + δ 0 female + β1educ + δ1 ( female × educ ) + u

In this example…
q  Wage is lower for females
but only for lower levels
of education because their
slope is larger

Is it fair to conclude that

women eventually earn
higher wages with
enough education?

65
Cautionary Note on Different Slopes!
n  Crossing point (where women earn higher
wages) might occur outside the data
(i.e. at education levels that don't exist)
q  Need to solve for crossing point before
making this claim about the data
Women : ln( wage) = β 0 + δ 0 + ( β1 + δ1 ) educ + u
Men : ln( wage) = β 0 + β1educ + u

q  They equal when educ = δ0/δ1

66
Cautionary Note on Interpretation!
n  Interpretation of non-interacted terms when
using continuous variables is tricky
n  E.g., consider the following estimates
ln( wage) = 0.39 − 0.23 female + 0.08educ − .01( female × educ )

q  Return to educ is 8% for men, 7% for women

q  But, at the average education level, how much less
do women earn? [–0.23 – 0.01×avg. educ]%

67
Cautionary Note [Part 2]
n  Again, interpretation of non-interacted
variables does not equal average effect unless
you demean the continuous variables
q  In prior example estimate the following:
ln( wage) = β 0 + δ 0 female + β1 ( educ − µeduc )
+δ1 female × ( educ − µeduc )
q  Now, δ0 tells us how much lower the wage is of
women at the average education level

68
Cautionary Note [Part 3]
n  Recall! As we discussed in prior lecture, the
slopes won't change because of the shift
q  Only the intercepts, β0 and β0 + δ0 , and their
standard errors will change

n  Bottom line = if you want to interpret non-

interacted indicators as the effect of indicators
at the average of the continuous variables, you
need to demean all continuous variables

69
Ordinal Variables
n  Consider credit ratings: CR ∈ ( AAA, AA,..., C , D)
n  If want to explain interest rate, IR, with
ratings, we could convert CR to numeric scale,
e.g. AAA = 1, AA = 2, … and estimate

IRi = β 0 + β1CRi + ui
q  But, what are we implicitly assuming, and how
might it be a problematic assumption?

70
Ordinal Variables continued…
n  Answer: We assumed a constant linear
relation between interest rates and CR
q  I.e. Moving from AAA to AA produces same
change as moving from BBB to BB
q  Could take log interest rate, but is a constant
proportional much better? Not really…

n  A better route might be to convert the

ordinal variable to indicator variables

71
Convert ordinal to indicator variables
n  E.g. let CRAAA = 1 if CR = AAA, 0 otherwise;
CRAA = 1 if CR = AA, 0 otherwise, etc.
n  Then, run this regression
IRi = β 0 + β1CRAAA + β 2CRAA + ... + β m−1CRC + ui
q  Remember to exclude one (e.g. "D")

n  This allows IR change from each rating

category [relative to the excluded indicator]
to be of different magnitude!

72
Linear Regression – Outline
n  The CEF and causality (very brief)
n  Linear OLS model
n  Multivariate estimation
n  Hypothesis testing
n  Miscellaneous issues
q  Irrelevant regressors & multicollinearity
q  Binary models and interactions
q  Reporting regressions

73
Reporting regressions
n  Table of OLS outputs should
generally show the following…
q  Dependent variable [clearly labeled]
q  Independent variables
q  Est. coefficients, their corresponding
standard errors (or t-stat), and stars
indicating level of stat. significance
q  Adjusted R2
q  # of observations in each regression

74
Reporting regressions [Part 2]
n  In body of paper…
q  Focus only on variable(s) of interest
n  Tell us their sign, magnitude, statistical &
economic significance, interpretation, etc.

q  Don't waste time on other coefficients

unless they are "strange" (e.g. wrong
sign, huge magnitude, etc)

75
Reporting regressions [Part 3]
n  And last, but not least, don't report
regressions in tables that you aren't going to
discuss and/or mention in the paper's body
q  If it's not important enough to mention in the
paper, it's not important enough to be in a table

76
Summary of Today [Part 1]
n  Irrelevant regressors and multi-
collinearity do not cause bias
q  But, can inflate standard errors
q  So, avoid adding unnecessary controls

n  Heteroskedastic variance doesn't cause bias

q  Just means the default standard errors for
hypothesis testing are incorrect
q  Use 'robust' standard errors (if larger)

77
Summary of Today [Part 2]
n  Interactions and binary variables
can help us get a causal CEF
q  But, if you want to interpret non-interacted
indicators it is helpful to demean continuous var.

n  When writing up regression results

q  Make sure you put key items in your tables
q  Make sure to talk about both economic and
statistical significance of estimates

78
In First Half of Next Class
n  Discuss causality and potential biases
q  Omitted variable bias
q  Measurement error bias
q  Simultaneity bias

n  Relevant readings; see syllabus

79
Assign papers for next week…
n  Fazzari, et al (BPEA 1988) These classic papers
in finance that use
q  Finance constraints & investment rather simple
estimations and
n  Morck, et al (BPEA 1990) 'identification' was
not a foremost
q  Stock market & investment concern

n  Opler, et al (JFE 1999) Do your best to think

about their potential
q  Corporate cash holdings weaknesses…

80
Break Time
n  Let's take our 10 minute break
n  We'll do presentations when we get back

81