Linear Regression
PT20903 Social Statistics
At the end of this learning:
• Determining the value of the regression
coefficient b and the regression constant a
• Expresses the equation of regression.
• Testing the hypothesis of the linear
relationship of two variables
The Concept Of Prediction And Regression
v In the topic of correlation, relationship or
association between two variables.
v This means, knowing how the variable X
changes when the variable Y changes.
v When these two variables can be
identified in the relationship, we can
predict the value of the variable Y when
the variable X is known.
The Concept Of Prediction And Regression
v This process is called prediction.
v Predicting is the process of estimating the
score of the variable Y as a variable criterion
based on the score of the variable X as a
variable predictor.
v The concept of correlation before is shown
using a scatterplot and when there is a
correlation, the plot point tends to be in a
straight line.
The Concept Of Prediction And Regression
v If a straight line can be drawn, that straight line
represents (on average) how the variable X is
connected with the change of variable Y.
v This line calls it a linear regression.
v It is worth noting that accurate forecasting does
not mean that the variable predictor gives an
effect on the variable criterion.
Linear Regression
There are two steps in the predation.
1. Specifies the regression line that is a
mathematical equation.
2. Using that math to predict scores.
For this topic, the regression line is a straight line
– linear.
Linear Regression
In predicting the Y score of the X score, Y
can be said to be a function of X given as:
Y′ = bX + a
With
Y = predicted score
b = slope of the line
a = Y intercept
Determine The Regression Line
v The regression line is a line that
describes the trend in the data where
the change in X will reflect in the
"average" change Y.
v The line fitted over the data is executed
using the method of least square.
Determine The Regression Line
v Method of least square loads that
regression line where the sum of square
distance between the data and the line is
the most minimal.
v b is also known as the regression
coefficient (regression coefficient).
v a is also known as the regression constant
(regression constant).
Determine The Regression Line
Determine The Regression Line
Determine The Regression Line
Determine The Regression Line
The values b and a can be estimated by
using the formula below
𝑛 ∑ 𝑋𝑌 − ∑ 𝑋 ∑ 𝑌 𝑆"
𝑏= ! 𝑏 = (𝑟)
∑ ! ∑
𝑛 𝑋 − ( 𝑋) 𝑆#
!"#$
a= a= 𝑌 − 𝑏𝑋
%
Example
v Given:
𝑆𝑌 0.52
𝑏= 𝑟 = +0.50 = 0.0021
𝑆𝑋 123.2
𝑎 = 𝑌 − 𝑏𝑋
= 2.57 − 0.0021 545.8
= 2.57 − 1.15 = 1.42
Example
The regression equation for this data can
be written as:
Y′ = bX + a
Y′ = 0.0021X + 1.42
The Y score is predictable. If X = 425,
Y′ = 0.0021X + 1.42
Y′ = 0.0021 (425) + 1.42
Y′ = 2.31
Example
v Get the regression equation for the
following data:
𝑆𝑌 14.83
𝑏= 𝑟 = +0.63 = +0.94
𝑆𝑋 9.97
𝑎 = 𝑌 − 𝑏𝑋 = 100 − 0.94 70 = 100 − 65.8 = 34.2
Example
The regression equation for this data can
be written as:
Y′ = bX + a
Y′ = 0.94X + 34.2
Predict the Y score if X = 55 and X = 82
Example
Standard error of estimate
v Since the regression line is fitted over that
data so that the distance of the data point
from the regression line is minimal.
v The distance between the predicted Y (Y')
score and the actual Y score is a predictive
error that can be given as Y − Y′
Standard error of estimate
v From this prediction error can be
calculated the standard error of the
estimate given as:
v
𝑌 − 𝑌′ 2
𝑆𝑌𝑋 =
𝑛−2
𝑛−1
𝑆𝑌𝑋 = 𝑆𝑌 1 − 𝑟 2
𝑛−2
Standard error of estimate
v Approximate standard error can be
estimated as follows when n is very large:
𝑌 − 𝑌′ 2
𝑆𝑌𝑋 =
𝑛
𝑆𝑌𝑋 = 𝑆𝑌 1 − 𝑟 2
Standard error of estimate
Contoh
v Approximate standard errors for the data
below can be calculated.
𝑆𝑌𝑋 = 𝑆𝑌 1 − 𝑟 2 = 0.52 × 1 − 0.502 = 0.45
Example
v Since SYX is an approximate standard
error, it is expected that 95% of an X
score will have a Y score between Y′ ±
(1.96) SYX (critical value at
v α = 0.05 by using z-distribution)
Contoh
Statistical Testing
Statistical Testing
Standard test statistics = Statistic − Parameters
Statistical standard error
Statistical Testing
v Statistical testing for linear regression
is to determine the regression
coefficient obtained either significantly
or otherwise.
v The specified standard error is the
standard error of the regression
coefficient, Sb which is required to
determine the statistics of the standard
test (for this case it is t).
Test Statistical Calculation
v Standard error regression coefficient,
Sb is calculated using the formula:
𝑆𝑌𝑋
𝑆𝑏 =
𝑆𝑆𝑋
With
SYX = approximate standard error
SSX = sum of squares of X
& (∑ %)!
𝑆𝑆% =∑(𝑋 − 𝑋) = ∑ 𝑋& − *
Test Statistical Calculation
v Standard test statistics are determined
with
t= b−β = b−β
𝑆𝑏 𝑆𝑌𝑋
𝑆𝑆𝑋
Hypothetical testing
Calculate the regression coefficient b and the regression
constant a. Write down the regression equation.
Hypothesis
H0: β = 0 (There is no significant linear relationship between
X and Y)
H1: β ≠ 0 (There is a significant linear relationship between X
and Y)
α = 0.05 or α = 0.01
df = n − 2
Specify critical value, t∗ on α and df
given.
Hypothetical Testing
v Calculate the SYX estimate standard
v Calculate SSX
v Standard error of regression coefficient, Sb
v Calculate the standard test statistics, t
v Compare t with t∗ and determine whether or
not to be in the rejection region
(acceptance).
v Make a decision on the null hypothesis
v Make a conclusion
