Logistic Regression
Shilpa Das
Logistic Regression
• Like the multiple regression, logistic regression is a statistical analysis
used to examine relationships between independent variables
(predictors) and a dependant variable (criterion).
• The main difference is in logistic regression, the criterion is nominal
(predicting group membership). For example, do age and gender
predict whether one signs up for swimming lessons (yes/no).
Logistic Regression
• The logistic regression model arises from the desire to model the posterior
probabilities of the K classes via linear functions in x, while, at the same time
ensuring that they sum to one and remain in [0,1]. Logistic regression is named for
the function used at the core of the method, the logistic function.
• The logistic function, also called the sigmoid function was developed by statisticians
to describe properties of population growth in ecology, rising quickly and maxing out
at the carrying capacity of the environment. It’s an S-shaped curve that can take any
real-valued number and map it into a value between 0 and 1, but never exactly at
those limits.
• F(x) = output between 0 and 1 (probability estimate)
• z = input to the function (your algorithm’s prediction e.g. mx + b)
• e = base of natural log
Logistic Regression
• Below is a plot of the numbers between -5 and 5 transformed into
the range 0 and 1 using the logistic function.
Types of Logistic Regression
• There are primarily 2 types of logistic regression: (1) Binary and (2)
Multinomial models. The difference lies in the types of the criterion
variable.
• Binary logistic regression is for a dichotomous criterion (i.e., 2-level
variable)
• Multinomial logistic regression is for a multicategorical criterion (i.e., a
variable with more than 2 levels)
Logistic Regression
Definition: Logistic Regression is a statistical model used for binary classification problems.
Despite its name, it is a classification algorithm, not a regression algorithm.
Why Logistic Regression?
Linear regression isn't suitable for predicting probabilities because it can predict values
outside the range of 0 and 1.
Logistic Regression
Model
Logistic Regression
Model
Clearly, we should not fit a linear regression model to these data. The
outcomes of a linear regression model can take any numerical value, but these
data can only take on outcomes of 0 or 1, so the predictions of a linear model
may not be meaningful.
data. The values of this function can be
interpreted as probabilities, as the values range
between 0 and 1. We can interpret the line as the
probability of a sunny day given a particular
temperature.
What is Log
Loss?
• Log Loss (or Logistic Loss) is the cost function used in Logistic Regression.
• It measures the performance of a classificati on model where the prediction is a
probability value between 0 and 1.
• The goal is to minimize the log loss to make the model more accurate.
Log Loss
Problem - I
• i.Classify whether (2.5, 3.5) is a malicious or benign form of cancer
based on the following dataset, if P()>=0.5, then, class 1 else 0 –
• Size Volume Type
• 2.7 2.5 0
• 1.4 2.3 0
• 3.3 4.4 0
• 3.04 3.05 0
• 5.3 2.75 1
Problem - II
• ii.Predict the following cases based on the given dataset, given Z, log (odds) =-
64+2*hrs –
• a.Find out the pass percentage of the student if he/she studies for 33 hours
• b.If the pass percentage is 95%, how many hours did the student study?
• Hours Results
• 29 0
• 15 0
• 33 1
• 28 1
• 39 1
