Module-2
Lecture-1
Linearity Vs Non-Linearity
Linearity refers to the property of a system or model where the output is directly
proportional to the input.
• In many cases, linear models are often the simplest and most effective approach.
• A linear model essentially fits a straight line to the data, allowing it to make predictions based on a linear
relationship between the input features and the output variable.
• In a regression problem, a linear model is used to predict a continuous target variable based on one or more
input features, such as the size and age of a tree.
Nonlinearity implies that the relationship between input and output is more complex
and cannot be expressed as a simple linear function.
• Nonlinear models can take many forms, from polynomial models that fit curves to the data, to neural
networks that can learn complex patterns in high-dimensional data.
• These models are often more powerful than linear models because they can capture more complex
relationships between variables.
Multiclass Classification Vs Multi-label classification
Multiclass classification is a machine learning challenge focused on categorizing data into more than two classes.
A binary classification involves distinguishing between only two classes, multiclass classification expands this scope
to involve distinguishing between multiple classes.
Multiclass classification finds application in a wide range of real-world scenarios.
Consider email categorization, where emails need to be sorted into categories like spam, non-spam, or important.
Multi-label classification is a machine learning paradigm where instances can be associated with multiple labels
simultaneously.
Unlike traditional classification tasks, where an instance is assigned a single exclusive label, multi-labe
classification recognizes the possibility for instances to exhibit characteristics that span across various categories.
Regression
• Regression analysis is a statistical technique for investigating and
modelling the relationship between variables
Simple Linear Regression
It is used to quantify the relationship between a predictor variable and a
response variable
The linear regression equation is represented by
Y = mx + c
Or
Y= ax + b
Example-1
The following data set shows the weight of 7 individuals.
Wt Height
140 60
155 62
159 67
179 70
192 71
200 72
212 75
Predict the height of a person who is 180Kgs
Step 1 Calculate x*y, x2,y2
Step 2 ∑x , ∑y , ∑x*y, ∑x2 ,∑y2
Wt Height
x y x*y x2 y2
140 60
155 62
159 67
179 70
192 71
200 72
212 75
∑x = ∑y= ∑x *y = ∑x2 = ∑y2 =
Wt Height
x y x*y x2 y2
140 60 8400 19600 3600
155 62 9610 24025 3844
159 67 10653 25281 4489
179 70 12530 32041 4900
192 71 13632 36864 5041
200 72 14400 40000 5184
212 75 15900 44944 5625
∑x = 1237 ∑y=477 ∑x *y =85125 ∑x2 =222755 ∑y2 =32683
a b
a= 0.20009 b= 32.783
Y=ax+b
Y=0.20009x+32.783
Interpretation
b=32.783
When weight (x) is zero kg, the predicted height is 32.783 inches
a= 0.2009
A one kg increase in weight is associated with a 0.2001 inch increase in height
Predicted height of a person who is 180Kgs is = ?
Error
• Now predict the height for the same weights with your linear regression
equation
• What is your observation?
Example-1
Multiple Linear Regression
It is used to derive the relationship between a multiple predictor
variables and a response variable
If we have a P predictor variables, then a multiple linear regression
takes the from as
Y= β0+ β1 x1+ β2 x2+…….. βP xP+e
Where,
Y= the response variable
xp= the pth predictor Variable
β j= the average effect on Y for a one unit increase in xp holding all other
predictors fixed
e= error term
Multiple Linear Regression
Step 1
Y X1 X2 X12 X22 X1*Y X2*Y X1*X2
140 60 22
155 62 25
159 67 24
179 70 20
192 71 15
200 72 14
212 75 14
215 78 11
∑= ∑= ∑= ∑= ∑= ∑= ∑= ∑=
Y X1 X2 X12 X22 X1*Y X2*Y X1*X2
140 60 22 3600 484 8400 3080 1320
155 62 25 3844 625 9610 3875 1550
159 67 24 4489 576 10653 3816 1608
179 70 20 4900 400 12530 3580 1400
192 71 15 5041 225 13632 2880 1065
200 72 14 5184 196 14400 2800 1008
212 75 14 5625 196 15900 2968 1050
215 78 11 6084 121 16770 2365 858
∑= 1452 ∑ 555 ∑145 ∑38767 ∑2823 ∑101895 ∑25364 ∑9859
Step:2
Calculate
Regression Sums
Step: 3 β1 , β2 and β0
Step 4 Find Y
Home work Problem 1
x1 x2 y
3 8 -3.7
4 5 3.5
5 7 2.5
6 3 11.5
2 1 5.7
Home work Problem 2
Math behind solving Linear Regression
using Matrix Multiplication
Home Work (Problem 3)
