Machine
Learning
Section 2
By: Mohamed Fathy
• Linear Regression (one variable)
Table of • Cost Function
• Gradient Descent
content:
• Linear Regression
(with multiple variables)
• Scikit-Learn
• Scikit-Learn
Datasets
• Data
Preprocessing
• Data Cleaning
• Data Split
• Scikit-Learn
Classification Regression Clustering
(Supervised) (Supervised) (Un-Supervised)
• Scikit-Learn
Datasets
• Data
Preprocessing
• Data Cleaning
• Data Split
• Supervised Learning: Given the “right answer” for each example in the
data. And it is divided into: Regression Problem (predict real-valued output),
and Classification Problem (predict discrete-valued output).
• Housing price prediction • Linear
o It is a common problem in the field of machine learning. Regression
o The goal is to build a model that can predict the sale price of a house (one variable)
based on a set of features such as the number of bedrooms, the square
footage of the house, the location of the house, and so on.
• Cost Function
• Gradient
Descent
• Housing price prediction
“best fit line” refers to the line that best represents the relationship between two
variables.
• Linear
Regression
(one variable)
• Cost Function
• Gradient
Descent
• Linear Regression is a statistical method used to model the relationship
between a dependent variable (also known as the response variable) and
one or more independent variables (also known as predictor variables or
features).
• Linear
• The relationship is assumed to be linear, which means that the dependent Regression
variable can be predicted by a linear combination of the independent (one variable)
variables, plus a constant term called the intercept.
• Cost Function
• Gradient
Descent
• The goal of linear regression is to find the line of best fit that describes the
relationship between the dependent and independent variables. This line is
typically represented by a linear equation of the form:
• Linear
Regression
(one variable)
• Cost Function
• Gradient
Descent
• Linear
Regression
(one variable)
• Cost Function
• Gradient
Descent
• Linear
Regression
(one variable)
• Cost Function
Gradient
• The goal of linear regression is to find the parameters (slope and intercept) •
Descent
of a linear equation that best fit the given data.
• The cost function is used to measure the difference between the predicted
values and the actual values of the dependent variable.
• The most used cost function in linear regression is the Mean Squared Error
(MSE) function.
• The MSE function is defined as the average of the squared differences • Linear
between the predicted values and the actual values. Mathematically, it can Regression (one
variable)
be expressed as:
• Cost Function
• The goal of linear regression is to minimize the value of the cost function.
• This is achieved by adjusting the parameters of the linear equation until the
• Gradient
value of the cost function is minimized.
Descent
• Once the parameters are optimized, they can be used to make predictions on
new data.
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
𝒉𝜽 𝒙 = 𝜽𝟎 + 𝜽𝟏 𝒙
3.5
3
1.2
• Linear
2.5
Regression (one
1
variable)
2
y
0.8
1.5
𝐽(𝜃1 )
0.6
1 • Cost Function
0.5 0.4
0 0.2
• Gradient
0 1 2 3 4
X Descent
0
0 0.2 0.4 0.6 0.8 1 1.2
𝜽𝟎 = 𝟎 𝜃1
𝒉𝜽 𝒙 = 𝜽𝟏 𝒙
3.5
• Linear
2.5
Regression (one
variable)
2
y
1.5 𝐽(𝜃1 )
1 • Cost Function
0.5
0
• Gradient
0 1 2 3 4
X Descent
𝜽𝟎 = 𝟎
𝜃1
• Contour plot/figure
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Gradient Decent Optimization
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Linear
Regression (one
variable)
Learning Rate
Derivative
• Cost Function
• Gradient
Descent
• Learning Rate (α)
o which basically defines how big the steps during the descent .
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Derivatives
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
• Example:
• Linear
Regression (one
variable)
• Cost Function
• Gradient
Descent
LR-One Variable Codes
• Linear
Regression
(with multiple
variables)
• Multiple features
𝑋0 𝑋1 𝑋2 𝑋𝑛 𝑦
𝑋0
𝑋1
𝑋2
• Linear
𝑚
𝑋 Regression
(with multiple
variables)
𝑿𝟐𝟏 = ?
• Linear regression with multiple variables (multiple linear
regression):
o it is a statistical method used to analyze the relationship between two or
more independent variables and a dependent variable.
o The goal of multiple linear regression is to create a linear equation that
can predict the value of the dependent variable based on the values of
the independent variables. • Linear
o In contrast to simple linear regression, which involves only one Regression
independent variable, multiple linear regression involves two or more (with multiple
variables)
independent variables. The equation for multiple linear regression takes
the form:
ℎ𝜃 (𝑥) : is the dependent variable.
𝑋1 , 𝑋2 , 𝑋3 , … , 𝑋𝑛 : are the independent variables. • Linear
𝜃1 , 𝜃2 , 𝜃3 , … , 𝜃𝑛 : are the coefficients of the independent variables. Regression
(with multiple
variables)
ℎ𝜃 𝑥 = 𝜃 𝑇 ∗ 𝑋
𝑋0 (𝐵𝑖𝑎𝑠) = 1
Hypothesis:
• Linear
Regression
Parameters: (with multiple
variables)
Cost function
• Gradient descent for multiple variables
• Linear
Regression
(with multiple
variables)
Implementation:
The implementation can be broken down into 3 parts:
▪ Loading Dataset and Preprocessing.
▪ Estimating Coefficients (Learning) using an optiomization
Algorithm (such as griedent decent). • Linear
Regression
o It requires two parameters: (with multiple
o Learning Rate: Used to limit the amount each coefficient is variables)
corrected each time it is updated.
o Iterations: The number of times to run through the training data
while updating the coefficients.
▪ Evaluate Algorithm.
LR-Multi Variable Codes
