What is regression analysis
Regression analysis is a set of statistical methods use to estimate the relationship between dependent
variable and one or two independent variable. It can be used to simulate the future relationship
between variables and evaluate how strongly the variables are related to one another.
y=β 0 + β 1 x +ϵ
Multiple regression
It’s a form of linear regression that is used when there are two or more predictors.
y=β 0 + β 1 x 1+ β 2 x 2+ ϵ
Ordinary least square
The ordinary least squares (OLS) method can be defined as a linear
regression technique that is used to estimate the unknown parameters in a
model. The OLS method minimizes the sum of squared residuals (SSR),
defined as the difference between the actual (observed values of the dependent
variable) and the predicted values from the model. The Ordinary Least Squares
(OLS) method achieves the minimization of the Sum of Squared Residuals
(SSR) by optimizing the values of the coefficients in a regression model so that
the total of the squares of the differences between the observed values and the
values predicted by the model is as small as possible. The resulting line
representing the dependent variable of the linear regression model is called
the regression line. This regression line represents the best fit for the data.
y = β₀ + β₁x₁ + β₂x₂ + ... + βₙxₙ + ε
where:
y is the dependent variable,
β₀ is the y-intercept,
β₁, β₂, ..., βₙ are the coefficients of the independent variables x₁,
x₂, ..., xₙ,
ε is the error term (residual).
Calculating the Coefficients
The coefficients are calculated by deriving the least squares
estimator, which involves taking partial derivatives of the sum
of squared residuals with respect to each coefficient, setting
them to zero, and solving the resulting system of equations.
This process leads to a set of normal equations, which can be
solved to find the coefficient estimates.
The formula for the coefficients in a simple linear regression
(one independent variable) can be represented as:
β₁ = Σ((xᵢ - x̄ )(yᵢ - ȳ)) / Σ((xᵢ - x̄ )²)
β₀ = ȳ - β₁x̄
