CHAPTER 3: CLASSIFICATION
1. Decision Tree Classifier
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn import tree
# Read the CSV file
df = pd.read_csv("salaries.csv")
print(df)
# Prepare inputs and target
inputs = df.drop('salary_more_then_100k', axis='columns')
target = df['salary_more_then_100k']
# Label encode categorical features
le_company = LabelEncoder()
le_job = LabelEncoder()
le_degree = LabelEncoder()
inputs['company_n'] = le_company.fit_transform(inputs['company'])
inputs['job_n'] = le_job.fit_transform(inputs['job'])
inputs['degree_n'] = le_degree.fit_transform(inputs['degree'])
print(inputs)
inputs_n = inputs.drop(['company', 'job', 'degree'], axis='columns')
# Create and fit the decision tree model
model = tree.DecisionTreeClassifier()
model.fit(inputs_n, target)
# Print model score and make predictions
print("Model Score:", model.score(inputs_n, target))
#Is salary of Google, Computer Engineer, Bachelors degree > 100 k ?
print("Prediction for [2, 1, 0]:", model.predict([[2, 1, 0]]))
#Is salary of Google, Computer Engineer, Masters degree > 100 k ?
print("Prediction for [2, 1, 1]:", model.predict([[2, 1, 1]]))
Output:
Model Score: 1.0
Prediction for [2, 1, 0]: [0]
Prediction for [2, 1, 1]: [1]
2. NAIVE BAYES CLASSIFICATION:[pg.no:51-53]
Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem and used
for solving classification problems. It is mainly used in text classification that includes a high-
dimensional training dataset.
The Bayesian method of calculating conditional probabilities is used in machine learning applications
that involve classification tasks. A simplified version of the Bayes Theorem, known as the Naive Bayes
Classification, is used to reduce computation time and costs.
Applications of Naïve Bayes Classifier:
It is used for Credit Scoring.
It is used in medical data classification.
It can be used in real-time predictions because Naïve Bayes Classifier is an eager learner.
It is used in Text classification such as Spam filtering and Sentiment analysis.
PROGRAM:
from sklearn import preprocessing
from sklearn.naive_bayes import GaussianNB
age = ['youth', 'youth', 'middle-aged', 'senior', 'senior',
'senior', 'middle-aged', 'youth', 'youth', 'senior', 'youth',
'middle-aged', 'middle-aged', 'senior']
income = ['high', 'high', 'high', 'medium', 'low', 'low',
'low', 'medium', 'low', 'medium', 'medium', 'medium',
'high', 'medium']
student = ['no', 'no', 'no', 'no', 'yes', 'yes', 'yes',
'no', 'yes', 'yes', 'yes', 'no', 'yes', 'no']
credit_rating = ['fair', 'excellent', 'fair', 'fair', 'fair',
'excellent', 'excellent', 'fair', 'fair', 'fair',
'excellent', 'excellent', 'fair', 'excellent']
buys_computer = ['no', 'no', 'yes', 'yes', 'yes', 'no',
'yes', 'no', 'yes', 'yes', 'yes', 'yes', 'yes', 'no']
# Create Label Encoder object
le = preprocessing.LabelEncoder()
# Converting string labels into numbers
age_encoded = le.fit_transform(age)
print(age_encoded)
income_encoded = le.fit_transform(income)
print(income_encoded)
student_encoded = le.fit_transform(student)
print(student_encoded)
credit_encoded = le.fit_transform(credit_rating)
print(credit_encoded)
# Converting string labels into numbers
label = le.fit_transform(buys_computer)
print(label)
# Combining age, income, student, and credit rating into a single list of tuples
features = list(zip(age_encoded, income_encoded, student_encoded, credit_encoded))
# Create a Gaussian Naive Bayes model
model = GaussianNB()
# Train the model using the training sets
model.fit(features, label)
# Predict output
predicted = model.predict([[2, 2, 1, 1]]) # 2: youth, 2: medium, 1: yes, 1: fair
print("Predicted Value:", predicted)
Output:
3. MULTINOMINAL NAIVE BAYES CLASSIFICATION:[pg.no:54-55]
Multinomial Naive Bayes is one of the most popular supervised learning classifications that is used
for the analysis of the categorical text data. Text data classification is gaining popularity because
there is an enormous amount of information available in email, documents, websites, etc. that needs
to be analyzed.
Examples of categorical variables are race, sex, age group, and educational level. While the latter two
variables may also be considered in a numerical manner by using exact values for age and highest
grade completed, it is often more informative to categorize such variables into a relatively small
number of groups.
This dataset is the result of a chemical analysis of wines grown in the same region in Italy but
derived from three different plant varieties. Dataset comprises of 13 features (alcohol, malic_acid,
ash,alkalinity_of_ash,magnesium,total_phenols,flavonoids,nonflavanoid_phenols,proanthocyanins,c
olor_intensity,hue,od280/od315_of_diluted_wines,proline) and type of wine plant variety. This data
has three type of wine Class_0, Class_1, Class_2.
PROGRAM:
# Import scikit-learn dataset library
from sklearn import datasets
# Load dataset
wine = datasets.load_wine()
# Print the names of the 13 features
print("Features:", wine.feature_names)
# Print the label type of wine (class 0, class 1, class 2)
print("Labels:", wine.target_names)
# Print data (feature) shape
print(wine.data.shape)
# Print the wine data features (top 5)
print(wine.data[:5])
print(wine.target)
# Import train test split function from sklearn.model_selection
from sklearn.model_selection import train_test_split
# Split dataset into training set and test set (70% training and 30% test)
X_train, X_test, y_train, y_test =train_test_split(wine.data, wine.target, test_size=0.3,
random_state=109)
# Import Gaussian Naive Bayes model
from sklearn.naive_bayes import GaussianNB
# Create a Gaussian Classifier
gnb = GaussianNB()
# Train the model using the training sets
gnb.fit(X_train, y_train)
# Predict the response for the test dataset
y_pred = gnb.predict(X_test)
print("Predicted Labels:",y_pred)
# Import scikit-learn metrics module for accuracy calculation
from sklearn import metrics
# Model Accuracy to find out how accurate is the classifier
print("Accuracy:", metrics.accuracy_score(y_test, y_pred) *100)
Output:
Classify 1: According to color.
Classify 2: According to the carbon dioxide pressure.
Classify 3: According to the sugar content.
4. LINEAR KERNEL:[pg.no:59-61]
Kernels, also known as kernel techniques or kernel functions, are a collection of distinct forms of
pattern analysis algorithms, using a linear classifier, they solve an existing non-linear problem. SVM
(Support Vector Machines) uses Kernels Methods in ML to solve classification and regression issues.
Linear Kernel is used when the data is Linearly separable, that is, it can be separated using a single
Line. It is one of the most common kernels to be used. It is mostly used when there are a Large
number of Features in a particular Data Set.
PROGRAM:
# Load libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,classification_report, confusion_matrix
import matplotlib.pyplot as plt
import seaborn as sn
from sklearn.svm import SVC
# Assign column names to the dataset
column_names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'Class']
# Load dataset
df = pd.read_csv("iris.csv", names=column_names)
# Split dataset into features and target
X = df.drop('Class', axis=1) # Features
y = df['Class'] # Target variable
# Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1)
# Import support vector classifier from sklearn.svm
clf = SVC(kernel='linear')
# Fit the classifier to the training data
clf.fit(X_train, y_train)
# Predict the classes on test set
y_pred = clf.predict(X_test)
print(y_pred)
# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy * 100)
# Print classification report and confusion matrix
print(classification_report(y_test, y_pred))
# Generate and display confusion matrix heatmap
cm = pd.crosstab(y_test,y_pred,rownames=['Actual'],colnames=['Predicted'])
ax = sn.heatmap(cm, annot=True)
plt.show()
Output:
5. POLYNOMINAL KERNEL:[pg.no:61-63]
PROGRAM:
# Import libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,classification_report, confusion_matrix
import matplotlib.pyplot as plt
import seaborn as sn
# Assign column names to the dataset
colnames = ['sepal-length', 'sepal-width', 'petal-length',
'petal-width', 'Class']
df = pd.read_csv("iris.csv", names=colnames)
# Split dataset into features and target variable
X = df.drop('Class', axis=1) # Features
y = df['Class'] # Target variable
# Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1)
# Import support vector classifier from sklearn
from sklearn.svm import SVC
clf = SVC(kernel='poly', degree=8) # Polynomial kernel with degree 8
# Fit the classifier to the training data
clf.fit(X_train, y_train)
# Make predictions on the test data
y_pred = clf.predict(X_test)
# Print the predicted labels
print(y_pred)
# Calculate and print the accuracy score
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy*100)
# Print the classification report
print(classification_report(y_test, y_pred))
# Generate and display the confusion matrix as a heatmap
confusion_matrix = pd.crosstab(y_test, y_pred,
rownames=['Actual'],colnames=['Predicted'])
ax = sn.heatmap(confusion_matrix, annot=True)
plt.show()
Output:
6. RADIAL BASIS FUNCTION KERNEL:[pg.no:63-65]
PROGRAM:
# Load libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,classification_report, confusion_matrix
import matplotlib.pyplot as plt
import seaborn as sn
from sklearn.svm import SVC
# Assign column names to the dataset
col_names = ['sepal-length', 'sepal-width', 'petal-length',
'petal-width', 'Class']
# Load the dataset
dataset = pd.read_csv("iris.csv", names=col_names)
# Separate features and target variable
X = dataset.drop('Class', axis=1)
y = dataset['Class']
# Split dataset into training set and test set (70% training and 30% test)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1)
# Import support vector classifier
clf = SVC(kernel='rbf', gamma=0.1)
# Fit the classifier to the training data
clf.fit(X_train, y_train)
# Predict the labels for the test set
y_pred = clf.predict(X_test)
# Print the predicted labels and accuracy
print("Predicted labels:", y_pred)
print("Accuracy:", accuracy_score(y_test, y_pred))
# Print the classification report
print(classification_report(y_test, y_pred))
# Generate and display confusion matrix as a heatmap
confusion_matrix = pd.crosstab(y_test, y_pred,
rownames=['Actual'], colnames=['Predicted'])
sn.heatmap(confusion_matrix, annot=True)
plt.show()
7. K-NEAREST NEIGHBOURS
The k-nearest neighbors algorithm, also known as KNN or k-NN, is a non-parametric, supervised
learning classifier, which uses proximity to make classifications or predictions about the grouping of
an individual data point.
# The make_blobs function from sklearn.datasets is used to generate a synthetic dataset with 500
samples, 2 features, 4 centers, and a cluster standard deviation of 1.5. The X variable contains the
feature vectors, and the y variable contains the corresponding labels.
PROGRAM:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
X, y = make_blobs(n_samples = 500, n_features = 2, centers = 4,cluster_std = 1.5, random_state = 4)
plt.style.use('seaborn')
plt.figure(figsize = (10,10))
plt.scatter(X[:,0], X[:,1], c=y, marker= '*',s=100,edgecolors='black')
plt.show()
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 0)
knn5 = KNeighborsClassifier(n_neighbors = 5)
knn1 = KNeighborsClassifier(n_neighbors=1)
knn5.fit(X_train, y_train)
knn1.fit(X_train, y_train)
y_pred_5 = knn5.predict(X_test)
y_pred_1 = knn1.predict(X_test)
from sklearn.metrics import accuracy_score
print("Accuracy with k=5", accuracy_score(y_test, y_pred_5)*100)
print("Accuracy with k=1", accuracy_score(y_test, y_pred_1)*100)
plt.figure(figsize = (15,5))
plt.subplot(1,2,1)
plt.scatter(X_test[:,0], X_test[:,1], c=y_pred_5, marker= '*', s=100,edgecolors='black')
plt.title("Predicted values with k=5", fontsize=20)
plt.subplot(1,2,2)
plt.scatter(X_test[:,0], X_test[:,1], c=y_pred_1, marker= '*', s=100,edgecolors='black')
plt.title("Predicted values with k=1", fontsize=20)
plt.show()
Output:
Accuracy with k=5 93.60000000000001
Accuracy with k=1 90.4
8. RANDOM FOREST :[pg.no:70-71]
It builds decision trees on different samples and takes their majority vote for classification and
average in case of regression.
PROGRAM:
# Load libraries
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score,classification_report, confusion_matrix
import matplotlib.pyplot as plt
import seaborn as sn
# Load dataset
df = pd.read_csv("iris.csv")
# Assign column names to the dataset
colnames = ['sepal-length', 'sepal-width', 'petal-length',
'petal-width', 'Class']
df.columns = colnames
# Split dataset into features and target
X = df.drop('Class', axis=1) # Features
y = df['Class'] # Target variable
# Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.3, random_state=1)
# Import random forest classifier and fit the model
clf = RandomForestClassifier(n_estimators=100)
clf.fit(X_train, y_train)
# Make predictions on the test set
y_pred = clf.predict(X_test)
# Print predictions and accuracy score
print("Predictions:", y_pred)
print("Accuracy:", accuracy_score(y_test, y_pred) * 100)
# Print classification report and confusion matrix
print("Classification Report:")
print(classification_report(y_test, y_pred))
print("Confusion Matrix:")
cm = confusion_matrix(y_test, y_pred)
print(cm)
# Generate heatmap and display it
plt.figure(figsize=(8, 6))
sn.heatmap(cm, annot=True, fmt='d')
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.show()
