Binary classification
A binary classification model is a model that can only distinguish between two
classes, such as yes or no, positive or negative, spam or not spam, etc. For
example, if you want to build a model that can detect whether an email is spam or
not, you would use a binary classification model. A binary classification model
usually outputs a probability score between 0 and 1 for each input, and then applies
a threshold to decide which class to assign. For example, if the score is greater than
0.5, the input is classified as positive, otherwise as negative.
Multi-class classification
A multi-class classification model is a model that can distinguish between more
than two classes, such as red, green, or blue, dog, cat, or bird, etc. For example, if
you want to build a model that can recognize the type of animal in an image, you
would use a multi-class classification model. A multi-class classification model
usually outputs a vector of probability scores for each input, one for each class, and
then selects the class with the highest score as the prediction. For example, if the
scores are [0.2, 0.6, 0.2] for red, green, and blue, the input is classified as green.
Multi-class classification is the classification technique that allows us to categorize
the test data into multiple class labels present in trained data as a model prediction.
There are mainly two types of multi-class classification techniques:-
One vs. All (one-vs-rest)
One vs. One
1. One vs. All (One-Vs-Rest)
In one-Vs-All classification, for the N-class instances dataset, we have to generate
the N-binary classifier models. The number of class labels present in the dataset
and the number of generated binary classifiers must be the same.
Now, as I told you earlier that we have to generate the same number of classifiers
as the class labels are present in the dataset, So we have to create three classifiers
here for three respective classes.
Classifier 1:- [Green] vs [Red, Blue]
Classifier 2:- [Blue] vs [Green, Red]
Classifier 3:- [Red] vs [Blue, Green]
Now to train these three classifiers, we need to create three training datasets.
After the training model, when we pass input test data to the model, then that data
is considered as input for all generated classifiers. If there is any possibility that
our input test data belongs to a particular class, then the classifier created for that
class gives a positive response in the form of +1, and all other classifier models
provide an adverse reaction in the way of -1. Similarly, binary classifier models
predict the probability of correspondence with concerning classes.
By analyzing the probability scores, we predict the result as the class index having
a maximum probability score.
2. One vs. One (OvO)
In One-vs-One classification, for the N-class instances dataset, we have to generate
the N* (N-1)/2 binary classifier models. Using this classification approach, we split
the primary dataset into one dataset for each class opposite to every other class.
we have a classification problem having three types: Green, Blue, and Red (N=3).
We divide this problem into N* (N-1)/2 = 3 binary classifier problems:
Classifier 1: Green vs. Blue
Classifier 2: Green vs. Red
Classifier 3: Blue vs. Red
Each binary classifier predicts one class label. When we input the test data to the
classifier, then the model with the majority counts is concluded as a result.
Binary classification vs. Multi-class classification
Binary Classification
Only two class instances are present in the dataset.
It requires only one classifier model.
Confusion Matrix is easy to derive and understand.
Example:- Check email is spam or not, predicting gender based on height
and weight.
Multi-class Classification
Multiple class labels are present in the dataset.
The number of classifier models depends on the classification technique we
are applying to.
One vs. All:- N-class instances then N binary classifier models
One vs. One:- N-class instances then N* (N-1)/2 binary classifier models
The Confusion matrix is easy to derive but complex to understand.
Example:- Check whether the fruit is apple, banana, orange
Different Types of Linear Regression are:
1. Univariate linear Regression
2. Multivariate linear Regression
Univariate linear regression
Univariate linear regression focuses on determining relationship between one
independent (explanatory variable) variable and one dependent variable.
Regression comes handy mainly in situation where the relationship between two
features is not obvious to the naked eye.
In the preceding equation, we have the following:
yi represents the dependent variable for the ith observation
xi represents the single independent variable for the ith observation
εi represents the error term for the ith observation
β0 is the intercept coefficient
β1 is the regression coefficient for the single independent variable
Multivariate linear regression
Multivariate regression is a technique used to measure the degree to which the
various independent variable and various dependent variables are linearly related
to each other. The relation is said to be linear due to the correlation between the
variables. Once the multivariate regression is applied to the dataset, this method is
then used to predict the behavior of the response variable based on its
corresponding predictor variables.
Multivariate regression is commonly used as a supervised algorithm in machine
learning, a model to predict the behavior of dependent variables and multiple
independent variables.
Multivariate regression allows one to have a different view of the
relationship between various variables from all the possible angles.
It helps you to predict the behavior of the response variables depending on
how the predictor variables move.
Multivariate regression can be applied to various machine learning fields,
economic, science and medical research studies.
Example of multivariate regression
An agriculture expert decides to study the crops that were ruined in a certain
region. He collects the data about recent climatic changes, water supply, irrigation
methods, pesticide usage, etc. To understand why the crops are turning black, do
not yield any fruits and dry out soon.
In the above example, the expert decides to collect the mentioned data, which act
as the independent variables. These variables will affect the dependent variables
which are nothing but the conditions of the crops. In such a case, using single
regression would be a bad choice and multivariate regression might just do the
trick.
