Naïve Bayes Classifier
Index:
• Introduction to Naïve Bayes Classifier
• Working of Naïve Bayes Classifier with example
• Pros and Cons of Naïve Bayes
• Application
• Types of Naïve Bayes Classifier
• References
Naïve Bayes Classifier Algorithm
▪ Naïve Bayes algorithm is a supervised learning algorithm, which is based
on Bayes theorem and used for solving classification problems.
▪ It is mainly used in text classification that includes a high-dimensional training
dataset.
▪ Naïve Bayes Classifier is one of the simple and most effective Classification
algorithms which helps in building the fast machine learning models that can
make quick predictions.
▪ It is a probabilistic classifier, which means it predicts on the basis of the
probability of an object.
Why is it called Naïve Bayes?
▪ The Naïve Bayes algorithm is comprised of two words Naïve and Bayes, Which can be described
as:
▪ Naïve: It is called Naïve because it assumes that the occurrence of a certain feature is independent of
the occurrence of other features. Such as if the fruit is identified on the bases of color, shape, and
taste, then red, spherical, and sweet fruit is recognized as an apple. Hence each feature individually
contributes to identify that it is an apple without depending on each other.
▪ Bayes: It is called Bayes because it depends on the principle of Bayes' Theorem.
Bayes' Theorem:
▪ Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to
determine the probability of a hypothesis with prior knowledge. It depends on the
conditional probability.
▪ The formula for Bayes' theorem is given as:
▪
Where,
▪ P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.
▪ P(B|A) is Likelihood probability: Probability of the evidence given that the
probability of a hypothesis is true.
• Bayes' Theorem: The foundation of Naive Bayes is Bayes' theorem,
which describes the probability of an event, based on prior
knowledge of conditions that might be related to the event.
• In classification tasks, it helps us determine the probability of a label
given some observed features.
Working of Naïve Bayes' Classifier:
• Consider a fictional dataset that describes the weather conditions for playing a
game of golf. Given the weather conditions, each tuple classifies the conditions as
fit(“Yes”) or unfit(“No”) for playing golf.
• Here is a tabular representation of our dataset shown in Table 1,Table 2 and Table
3
Outlook Play
0 Rainy Yes
1 Sunny Yes
2 Overcast Yes
3 Overcast Yes
4 Sunny No
5 Rainy Yes
6 Sunny Yes
7 Overcast Yes
8 Rainy No
9 Sunny No
10 Sunny Yes
11 Rainy No
12 Overcast Yes
13 Overcast Yes
Table 1
Frequency table for the Weather Conditions:
Weather Yes No
Overcast 5 0
Rainy 2 2
Sunny 3 2
Total 10 4
Table 2
Likelihood table weather condition:
Weather No Yes
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71
Table 3
Applying Bayes'theorem:
• P(Yes|Sunny)= P(Sunny|Yes)*P(Yes)/P(Sunny)
• P(Sunny|Yes)= 3/10= 0.3
• P(Sunny)= 0.35
• P(Yes)=0.71
So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60
• P(No|Sunny)= P(Sunny|No)*P(No)/P(Sunny)
• P(Sunny|NO)= 2/4=0.5
Applying Bayes'theorem:
• P(No)= 0.29
• P(Sunny)= 0.35
• So P(No|Sunny)= 0.5*0.29/0.35 = 0.41
• So as we can see from the above calculation that P(Yes|Sunny)>P(No|Sunny)
• Hence on a Sunny day, Player can play the game.
Advantages & Disadvantages of Naïve Bayes Classifier:
• Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets.
• It can be used for Binary as well as Multi-class Classifications.
• It performs well in Multi-class predictions as compared to the other Algorithms.
• It is the most popular choice for text classification problems.
Disadvantages of Naïve Bayes Classifier:
• Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the
relationship between features.
Applications of Naïve Bayes Classifier:
• It is used for Credit Scoring.
• It is used in medical data classification.
• It can be used in real-time predictions because Naïve Bayes Classifier is an eager learner.
• It is used in Text classification such as Spam filtering and Sentiment analysis.
• Despite its simplicity and the "naive" assumption, Naive Bayes can
often perform surprisingly well, especially on text classification tasks
where the independence assumption might not be severely violated.
• However, it might not be suitable for datasets with highly correlated
features.
Types of Naïve Bayes Model:
There are three types of Naive Bayes Model, which are given below:
• Gaussian: The Gaussian model assumes that features follow a normal distribution. This means if
predictors take continuous values instead of discrete, then the model assumes that these values are
sampled from the Gaussian distribution.
• Multinomial: The Multinomial Naïve Bayes classifier is used when the data is multinomial
distributed. It is primarily used for document classification problems, it means a particular
document belongs to which category such as Sports, Politics, education, etc.
The classifier uses the frequency of words for the predictors.
• Bernoulli: The Bernoulli classifier works similar to the Multinomial classifier, but the predictor
variables are the independent Booleans variables. Such as if a particular word is present or not in a
document. This model is also famous for document classification tasks.
