INTRODUCTION TO REGRESSION ANALYSIS

(STAT 367)

DEPARMENT OF STATISTICS AND ACTUARIAL SCIENCE

FACULTY OF PHYSICAL AND COMPUTATIONAL SCIENCE

COLLEGE OF SCIENCE

E. O. OWIREDU

January 18, 2025

COURSE OUTLINE

Basic Concepts of Regression and Correlation Analysis.

Simple and Multiple Linear Regression.
Estimation of Parameters by the Least Squares Method.
Inferential Analysis on Regression Parameters.
Use of Qualitative Variables.
Residual Analysis for Testing Models and Assumptions and Adequacy.
Correlation Analysis of Response and Predictor Variable.
Concept of Multicollinearity
Use of Statistical Packages for Regression and Correlation Analysis
(SPSS, R, STATA, etc.)

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 2 / 28

Basic Concepts of Regression and Correlation Analysis

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 3 / 28

COURSE OBJECTIVES

Introduce students to the art of statistical modeling for phenomena

that can be described by a linear function.
Enable students to apply linear regression to model appropriate data
using a statistical package of their preference.
Equip students to carry out inferential procedures.
Develop the ability to properly interpret the results of statistical
analyses.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 4 / 28

CORRELATION
INTRODUCTION
Finding the relationship between two variables without being able to infer
causal relationships.
Correlation is a statistical method used to determine whether a linear
relationship between variables exists.

Real-Life Examples of Correlation

The more hours you spend studying, the higher your test scores tend
to be.
The more water you drink, the more hydrated you feel.
The more sunlight plants get, the faster they grow (up to a certain
limit).
The more you practice a musical instrument, the better you get at
playing it.
E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 5 / 28
CORRELATION CONT’D...

Two Types of Variables here!!!

Dependent = Explained = Regressand = Endogenous = Response =
Outcome = Uncontrolled.
Independent = Explanatory = Regressor = Exogenous = Stimulus =
Predictor = Covariate = Controlled.

NOTE
Correlation does not imply casualty.
Correlation does not distinguish between independent and dependent
variables.
The phrase "correlation does not imply causation" refers to the
inability to legitimately deduce a cause-and-effect relationship
between two events or variables solely on the basis of an observed
association or correlation between them.
E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 6 / 28
CORRELATION CONT’D...

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 7 / 28

Properties of Pearson’s Correlation Coefficient

Measures the relative strength of the linear association between two

variables.

Takes on the same sign as the slope estimate from the linear
regression.

Not affected by linear transformations of y or x.

Does not distinguish between dependent and independent variables

(e.g height and weight).

Population Parameter ρ.

The population parameter ρ is estimated by r .

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 8 / 28

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 9 / 28
CORRELATION CONT’D...

Pearson’s Correlation Coefficient

Unit-less.
Ranges between -1 and 1.
The closer to -1, the stronger the negative linear relationship.
The closer to +1, the stronger the positive linear relationship.
When is 0, there is no linear relationship.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 10 / 28

CORRELATION CONT’D...

Scatter plots of Data with Various Correlation Coefficients.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 11 / 28

Interpretation of Correlation Coefficient

Strength
The greater the absolute value of the correlation coefficient, the
stronger the relationship.
The extreme values of −1 and 1 indicate a perfectly linear
relationship where a change in one variable is accompanied by a
perfectly consistent change in the other.
For these relationships, all of the data points fall on a line. In practice,
you will not see either type of perfect relationship. A zero coefficient
does not represent a linear relationship. As one variable increases,
there is no tendency for the other variable to increase or decrease.
When the value is between 0 and +1/-1, there is a relationship, but
the points don’t all fall on a line. As r approaches −1 or 1, the
strength of the relationship increases and the data points tend to fall
closer to a line.
E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 12 / 28
Interpretation of Correlation Coefficient Cont’d...

Direction
The sign of the correlation coefficient represents the direction of the
relationship.
Positive coefficient indicates that when the value of one variable
increases, the value of the other variable also tends to increase.
Positive relationships produce an upward slope on a scatter plot.

Negative coefficient represents cases when the value of one variable

increases and the value of the other variable tends to decrease.
Negative relationships produce a downward slope.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 13 / 28

CORRELATION CONT’D...

Correlation Coefficient Interpretation Guideline

The correlation coefficient r ranges from -1 (a perfect negative correlation)

to 1 (a perfect positive correlation). In short, -1 ≤ r ≤ 1.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 14 / 28

CORRELATION CONT’D...

Example 1

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 15 / 28

CORRELATION CONT’D...

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 16 / 28

CORRELATION CONT’D...

What is the nature of the Scatter Plot?

Positive relationship.

Negative relationship.

No relationship.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 17 / 28

CORRELATION CONT’D...

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 18 / 28

CORRELATION CONT’D...

Distribution of the Correlation Coefficient:

The sample correlation coefficient follows a T-distribution with n-2

degrees of freedom (since you have to estimate the standard error).

NOTE
Like a proportion, the variance of the correlation coefficient depends on
the correlation coefficient itself. substitute in estimated r.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 19 / 28

Test of Significance of the Correlation Coefficient

The Hypotheses is;

H0 : ρ = 0
H1 : ρ ̸= 0

When the null hypothesis is accepted, it means that there is no linear

relationship between the variables.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 20 / 28

Example 2

Calculate coefficient of correlation for the ages of husband and wife.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 21 / 28

Solution

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 22 / 28

Solution Cont’d...

Since 0.62 is close to +1.0 it appears to be a positive strong correlation.

This implies that the number of hours spent studying for an exam is
positively correlated to the exam score.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 23 / 28

Testing for the Significance of the Correlation Coefficient

Hypothesis:
H0 : The number of hours that a student studies for a final examination is
not positively correlated to the score they will receive on the exam.
H1 : The number of hours that a student studies for a final examination is
positively correlated to the score they will receive on the exam.
Using a T - distribution with a significance level of 5%
Total Number of Observations (n) = 7
s s
n−2 7−2
tcal = r = 0.62 = 1.767
1 − r2 1 − 0.622
From the T - distribution with significance level of α = 5% with df = 5

t α2 ,n−2 = t0.025,5 = 3.163

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 24 / 28

CONCLUSION
Since tcal is less than t0.025,5 we must fail to reject the null hypothesis.
This implies we are 95% confidence that the number of hours studying for
an exam does not in fact positively correlate with scores on the exam.

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 25 / 28

ACTIVITY

Try 1
Calculate the correlation coefficient, the standard error and tcal of the
given data:

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 26 / 28

ACTIVITY

Try 2
Find out the correlation between the marks in Statistics and marks in
Accountancy:

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 27 / 28

E. O. OWIREDU REGRESSION ANALYSIS January 18, 2025 28 / 28