Machine Learning
Engr. Ejaz Ahmad
Common terms
Accuracy
% of correct prediction made by model to the total
observation
Algorithm
A method, function or set of instruction to generate a
machine learning algorithm
Attribute
A quality describing an observation
Common terms
Bias
Bias is error due to overly simplistic in training
data. causes under fitting
Bias matric
Average difference between prediction and
Observed values
Bias Terms
Allow model to represent patterns that do not pass
through the origin
Common terms
Categorical Variables
Variable with a discrete possible set of values
Classification_report
Recall
Amount of true positive rate with compare to the
actual positive event throughout data
Precision
It is the amount of positive predictive value and it is a
measure of the amount of accurate positives your model
claims compared to the number of positives it actually
claims
Classification_report
F1 Error
It is the measure of model performance, it is the
average weight of the precision and recall of the model
From sklearn.metrics import classification_report
classification_report(y_test, y_predict)
ROC_curve
Receiver operating characteristics is used for visual
comparison of classification model
Common terms
Convergence
State reach during the training model when the
Loss change very little during each titration
Dimension
Number of feature have in data sets
Extrapolation
Making prediction out side the range of data sets
Common terms
Feature Selection
It is a process of selection of relevant feature from
data set for model
Hyper parameter
it is higher level of model property such as ,learning
rate, depth of tree, number of hidden layers
Normalization
Restriction of the value of the weight in regression
to avoid over fitting
Common terms
Noise
Any irrelevant information or randomness in
dataset
Outliers
An observation that deviates significantly from
other observation
Bayes Theorem
Bayes Theorem
it gives the posterior probability of an event on the
bases of prior knowledge
Generative Model
Generative Model will learn categories of data
Discriminative model
It simply learn the distinction between different
categories of data
Split train and test data
From sklearn.model_selection import train_test_split
Accuracy Measure
From sklearn.matrics import accuracy_score
Data Preprocessing
Feature Scaling
Pipeline
Polynomial Feature
Regression
Regression
Statistical Model for making relationship between a
dependent variable with a given set of independent
variables
1.Simple Linear Regression
Prediction a response using a single feature
From sklearn.linear_model import LinearRegression
Model=LinearRegression()
Model.fit()
Model.intercept_,model.cofe_
Model.predict()
Gradient Descent
It is a very effective and simple approach to fit linear
models. The general idea for GD is tweak parameters
iteratively in order to minimize the cost function
Types
1. Batch Gradient DescentIt use full training Batch at
each step
2. Stochastic Gradient Descent It pick any random
instance in training set and calculate the gradient
3. Mini batch Gradient Descent it calculate gradient
on small random set of instance called mini batch
Stochastic Gradient Descent
From sklearn.linear_model import SGDRegressor
Model=SGDRegressor(max_iter=1000,tol=1e-
3,eta0=0.1,penalty=None)
Model.fit()
Model.predict()
Regularized Linear Models
For linear models regularization achieved by
constraining the weight of the model
Following is the regularized version of linear
Regression
1. Ridge Regression
2. Lasso Regression
3. Elastic Net
Ridge Regression
From sklearn.linear_model import Ridge
Model=Ridge(alpha=1,solver=‘cholesky’)
Model.fit()
Model.predict()
Same task can be accomplish using SGDRegression
Model=SGDRegression(penalty=‘l2’)
Here penalty l2 means Ridge Regression
Lasso class
From sklearn.linear_model import Lasso
Model=Lasso(alpha=0.1)
Model.fit()
Model.predict()
Elastic Net
From sklearn.linear_model import ElasticNet
Model=ElasticNet(alpha=0.1 , l1_ratio=0.5)
Model.fit()
Model.predict()
Logistic Regression
It produce result in binary format which is used to
predict the outcome of the categorical dependent
variable.so its outcome should be
discrete/categorical, such as 0 or 1,yes or no etc.
From sklearn.linear_model import
LogisticRegression
Classification
Classification
It is the process of categories a give data into classes,
it can be performed on both structured and
unstructured data
Decision Tree
It split the dataset into small segment until the target
variable are the same or until the dataset can no
longer be split
From sklearn.tree import DecisionTreeClassifier
Model=DecisionTreeClassifier()
Model. fit()
Model.predict()
Decision Tree
Save model result
From sklearn.externals import joblib
Joblib.dump(model, ”filename.joblib”)
model=joblib.load(‘filename.joblib’)
K.Nearest Neighbor
It is Supervised learning both for Regression and
classification. The principal is to find the predefined
number of training sample closest to the new point
Support Vector Machine
SVM is a very powerful and versatile Machine
Learning model capable for linear and non linear
classification and regression and even outliear
detection
Linear SVM Classification
From sklearn.pipeline import Pipeline
From sklearn.preprocessing import StandardScale
From sklearn.svm import LinearSVC(support vector classifier)
Svm_clf=Pipeline([(‘scaler,StandardScale),
(linear_csv,LinearSVC(C=1,loss=‘hinge’))])
Scm_clf.fit(x, y)
We can regulate the model by decreasing the value of C hyper
parameter
Anotheroption is to use the SGDClassifier class, with
SGDClassifier(loss="hinge",
alpha=1/(m*C)).
Nonlinear SVM Classification
There are two method for classification in SVM
1. Make the data linear by using Polynomial Feature
2. Use the Kernel trick
1.Method PolynomialFeatures
polynomial_svm_clf = Pipeline([
("poly_features", PolynomialFeatures(degree=3)),
("scalar", StandardScaler()),
("svm_clf", LinearSVC(C=10, loss="hinge"))
])
polynomial_svm_clf.fit(X, y)
2.Mathod Polynomial Kernel
from sklearn.svm import SVC
poly_kernel_svm_clf = Pipeline([
("scaler", StandardScaler()),
("svm_clf", SVC(kernel="poly", degree=3, coef0=1,
C=5))
])
poly_kernel_svm_clf.fit(X, y)
2.Method Gaussian RBF Kernel
rbf_kernel_svm_clf = Pipeline([
("scaler", StandardScaler()),
("svm_clf", SVC(kernel="rbf", gamma=5, C=0.001))
])
rbf_kernel_svm_clf.fit(X, y)
Increasing gamma makes the bell-shape curve narrower
SVM Regression
Linear Regression
from sklearn.svm import LinearSVR
svm_reg = LinearSVR(epsilon=1.5)
svm_reg.fit(X, y)
Nonlinear Regression
from sklearn.svm import SVR
svm_poly_reg = SVR(kernel="poly", degree=2, C=100,
epsilon=0.1)
svm_poly_reg.fit(X, y)
SGD classifier
It is very strong classifier for handling large data
and suitable for noline learning
from sklearn.linear_model import SGDClassifier
sgd_clf = SGDClassifier(random_state=42)
sgd_clf.fit(X_train, y_train_5)
Naïve Bayes
It is a classification algorithm based on Bayes’s
Theorem
Random Forest
Artificial NeuralNetwork
Performance
Measures
Classification models
Accuracy Measure using K_Fold
K-fold cross validation means splitting the training set
into K-folds (i.e. 2,3,4), then making predictions and
evaluating them on each fold using a model trained
on the remaining folds
from sklearn.model_selection import cross_val_score
cross_val_score(sgd_clf, X_train, y_train_5, cv=3,
scoring="accuracy")
Confusion Matrix
It is a performance measurement technique of classification
It has 4 parameter,TP,TN,FP,FN
From sklearn.matrics import confusion_matrix
Confusion_matrix(y_train,y_predict)
C matrix tack two argument train and predict,we can find
y_predict for it using followni
from sklearn.model_selection import cross_val_predict
y_train_pred = cross_val_predict(sgd_clf, X_train, y_train_5,
cv=3)
Precision
Precision
It is the accuracy of positive prediction
Precision=TP/(TP+FP)
from sklearn.metrics import precision_score
precision_score(y_train_5, y_train_pred)
Recall
It is the ration of positive instance that are correctly
detected by classifier
Recall=TP/(TP+FN)
from sklearn.metrics import recall_score
recall_score(y_train_5, y_train_pred)
Confusion Matrix
Type I Error
It is False Positive error ,means calming that
something has happened but in actual it hasn't
Type II Error
It is False Negative error, means calming that
something has not happened but in actual it happened
