Machine Learning

ICT-4261

By-
Dr. Jesmin Akhter
Professor
Institute of Information Technology
Jahangirnagar University
 Grading Policy

 Attendance =10%
 Exercise test =10%
– Instant test
– Assignment
– Presentation
 Class Test (Average of two) =20%
 Final Examination =60%
================================
=100%
 Exam schedule

Tutorial Date and Time

Tutorial-01 After 4 classes

Tutorial-02 After 8 classes
Tutorial-03 After 12 classes
Tutorial-04 After 18 classes

NB: Schedule may change

Contents
The course will mainly cover the following topics:
 A Gentle Introduction to Machine Learning
 Linear Regression
 Logistic Regression
 Naive Bayes
 Support Vector Machines
 Decision Trees and Ensemble Learning
 Clustering Fundamentals
 Hierarchical Clustering
 Neural Networks and Deep Learning
 Unsupervised Learning……
Outline

 A Gentle Introduction to Machine Learning

– What is Machine Learning
– classic and adaptive machines
– Supervised learning
– Unsupervised learning
What is Machine Learning

 Machine learning is a sub-field of artificial intelligence (AI) , which enables the machine
– To automatically learn from data,
– Improve performance from past experiences without being explicitly programmed, and
– Make predictions.
 Machine learning focuses on the use of huge amount of data and a set of algorithms to
imitate the way that human learns, gradually improving its accuracy.
 Data is fed to these algorithms to train them, and on the basis of training, they build the model and
perform a specific task.
 For the process of learning, model fitting, we need to have available data in order to
explore potential underlying patterns, hidden in our data.
– These learned patterns are some functions or decision boundaries.
 These patterns are learned by the systems automatically without human intervention or input.
 Machine learning algorithms are typically created using frameworks TensorFlow and PyTorch that
accelerate solution. 6
 Classic machines

 This is the generic representation of a classical system that receives some input values,
processes them, and produces output results:

 Classic machines examine data inputs according to a predetermined set of rules, finding
patterns and relationships that can be used to generate predictions or choices.
– Support vector machines, decision trees, and logistic regression are some of the most used
classical machine-learning techniques.
 Adaptive machines
 Here's a schematic representation of an adaptive system:

 Such a system isn't based on static or permanent structures (model parameters and architectures) but
rather on a continuous ability to adapt its behavior to external signals (datasets or real-time inputs) and,
like a human being, to predict the future using uncertain and fragmentary pieces of information.
 Artificial neural networks, which are designed after the composition and operation of the human brain, are
used by adaptive machines. These neural networks are made up of layers of connected nodes, or neurons,
where each one carries out a straightforward calculation. The neurons are arranged in layers, and each
layer processes the input data in a unique way.
 Types of machine learning
 Based on the methods and way of learning, Machine learning models fall into four primary categories:
– Supervised Machine Learning:
– Unsupervised Machine Learning: which goods are usually purchased together is a real-world example of
market basket analysis
– Semi-Supervised Machine Learning: Text Classification-Sentiment analysis is often performed on textual
data to help businesses to monitor brand and product from the sentiment in customer feedback, and
understand customer needs.
– Reinforcement Learning: It can enable self-driving cars to learn from their own experience and adapt to
changing situations
 Supervised machine learning

 Supervised learning, also known as supervised machine learning, is defined by its use of
labeled datasets to train algorithms to classify data or predict outcomes accurately.
 Supervised machine learning has two key components: first is input data and second
corresponding output labels. The goal is to build a model that can learn from this labeled data
to make predictions or classifications on new, unseen data.
 The labeled data consists of input features (also known as independent variables or
predictors) and the corresponding output labels (also known as dependent variables or
targets).
 The model’s objective is to capture patterns and relationships between the input features and
the output labels, allowing it to generalize and make accurate predictions on unseen data.
 Supervised machine learning

 As input data is fed into the model, the model adjusts its weights until it has been fitted
appropriately.
 This occurs as part of the cross validation process to ensure that the model
avoids overfitting or underfitting. Supervised learning helps organizations to solve a variety
of real-world problems at scale, such as classifying spam in a separate folder from your inbox.
 There are two main types of supervised learning problems: they are classification that involves
predicting a class label and regression that involves predicting a numerical value.
 Classification algorithms are used to predict/Classify the discrete values such as Male or
Female, True or False, Spam or Not Spam, etc
 Regression algorithms are used to predict the continuous values such as price, salary, age,
etc.
 Classification and Regression
 The classification process deals with  Regression is the process of finding a model or
problems where the data can be divided into function for distinguishing the data into continuous
binary or multiple discrete labels. real values instead of using classes or discrete values.
 Suppose we want to predict the possibility of It can also identify the distribution movement
depending on the historical data.
the winning of a match by Team A on the
basis of some parameters recorded earlier.  An example in regression, where we are finding the
possibility of rain in some particular regions with the
Then there would be two labels Yes and No.
help of some parameters recorded earlier. Then
there is a probability associated with the rain.
 How Does Supervised Learning Work?

Supervised machine learning typically follows a series of steps to train a model and make predictions.
 Data Collection and Labeling
– The first step in supervised machine learning is collecting a representative and diverse
dataset. This dataset should include a sufficient number of labeled examples that cover the
range of inputs and outputs the model will encounter in real-world scenarios.
– The labeling process involves assigning the correct output label to each input example in the
dataset. This can be a time-consuming and labor-intensive task, depending on the
complexity and size of the dataset.
 Training and Test Sets
– Once the dataset is collected and labeled, it is divided into two subsets: the training set and
the test set.
• The training set is used to train the model and it serves as the basis for the model to
learn patterns and relationships between the input features and the output labels,
• While the test set is used to assess the model’s generalization ability and its performance
on new, unseen data.
 How Does Supervised Learning Work?

 Feature Extraction
– Before training the model, it is essential to extract relevant features from the input
data. Feature extraction involves selecting or transforming the input features to capture
the most relevant information for the learning task. This process can enhance the model’s
predictive performance and reduce the dimensionality of the data.
 Model Selection and Training
– Choosing an appropriate machine learning algorithm is crucial for the success of supervised
learning. Different algorithms have different strengths and weaknesses, making it important to
select the one that best fits the problem at hand.
– Once the algorithm is selected, the model is trained using the labeled training data. During
the training process, the model learns the underlying patterns and relationships in
the data by adjusting its internal parameters.
– The objective is to minimize the difference between the predicted outputs and the
true labels in the training data.
How Does Supervised Learning Work?
 How Does Supervised Learning Work?

 Prediction and Evaluation

– Once the model is trained, it can be used to make predictions on new, unseen data. The input
features of the unseen data are fed into the trained model, which generates predictions or
classifications based on the learned patterns.
– To evaluate the model’s performance, the predicted outputs are compared against the true
labels of the unseen data.
– Common evaluation metrics include accuracy, precision, recall, and F1 score, depending on the
nature of the learning task.
 Supervised Learning Algorithms

Supervised machine learning includes various algorithms, each suited for different types of problems.
Commonly used algorithms are:
 Linear Regression
– Linear regression is a popular algorithm used for predicting continuous output values. It establishes a
linear relationship between the input features and the target variable, allow us to make predictions
based on this relationship.
 Logistic Regression
– Logistic regression is employed when the output variable is binary or categorical. It models the
relationship between the input features and the probability of a particular outcome using a logistic
function. Logistic Regression generates a value where output is always either 0 or 1. So it is a
classification algorithm.
 Decision Trees
– Decision trees are tree-like models that use a hierarchical structure to make decisions. They split the
data based on different features and create a tree-like structure, enabling classification or
regression tasks. It is also used in Random Forest to train on different subsets of training data
 Supervised Learning Algorithms

 Random Forests
– Random forests are an ensemble learning method that combines multiple decision trees.
They improve the predictive accuracy by aggregating predictions from multiple trees,
reducing overfitting and increasing robustness.
 Support Vector Machines (SVM)
– Support Vector Machines are effective for both classification and regression tasks. They create
hyperplanes or decision boundaries that maximize the margin between different classes, allowing
for accurate predictions. The hyperplane tries that the margin between the closest points of
different classes should be as maximum as possible.
 Naive Bayes
– Naive Bayes algorithms are based on Bayes’ theorem and are commonly used for classification
tasks. They assume that the input features are independent, making predictions based on the
probability of each class.
 Supervised Learning Algorithms

 K-Nearest Neighbors (KNN)

– K-Nearest Neighbors is a non-parametric algorithm that classifies new instances based on their
proximity to the labeled instances in the training data. It assigns a class label based on the
majority vote of its k nearest neighbors.
 Neural Networks
– Neural networks are a powerful class of algorithms inspired by the human brain’s structure and
functioning.They consist of interconnected nodes (neurons) organized in layers, enabling them to
learn complex patterns and relationships.
 Gradient Boosting Algorithms
– Gradient boosting algorithms, such as Gradient Boosted Trees and XGBoost, are ensemble
methods that sequentially build models, each focusing on the errors of the previous models. They
are effective for classification and regression tasks, providing high predictive accuracy.
 Applications

Supervised machine learning finds application in various domains. Here are some examples:
 House prices
– One practical example of supervised learning problems is predicting house prices. How is this
achieved?
– First, we need data about the houses: square footage, number of rooms, whether a house has a
garden or not, location and so on. We then need to know the prices of these houses, i.e. the
corresponding labels. By leveraging data/ analyzing data coming from thousands of houses, their
features and prices, we can now train a supervised machine learning model to predict a new
house’s price based on the examples observed by the model.
 Applications

 How’s the weather today?

– One particularly interesting problem which requires considering a lot of different parameters, is
predicting weather conditions in a particular location.To make correct predictions for the
weather, we need to take into account various parameters, including historical temperature data,
precipitation, wind, humidity, and so on.
– This particularly interesting and challenging problem may require developing complex supervised
models that include multiple tasks. Predicting today’s temperature is a regression problem, where
the output labels are continuous variables. By contrast, predicting whether it is going to snow or
not the next day is a binary classification problem.
 Applications

 Spam Email Detection

– Supervised learning can be used to classify emails as spam or legitimate. By training a model on a
labeled dataset of spam and non-spam emails, it can accurately predict whether an incoming email
is spam, helping filter unwanted messages.
 Sentiment Analysis
– One particularly popular topic in text classification is to predict the sentiment of a piece of text,
like a tweet or a product review. Sentiment analysis involves determining the sentiment or
opinion expressed in text data. By training a model on labeled data that associates text with
positive, negative, or neutral sentiments, it can automatically analyze large volumes of text, such as
social media posts or customer reviews. This is widely used in the e-commerce industry to help
companies to determine negative comments made by customers
 Applications

 Image Classification
– Supervised learning enables image classification tasks, where the goal is to assign a label to an
image based on its content. By training a model on a dataset of labeled images, it can accurately
classify new images, enabling applications like object recognition and autonomous driving.
– Is it a cat or a dog?
– Image classification is a popular problem in the computer vision field. Here, the goal is to predict
what class an image belongs to. In this set of problems, we are interested in finding the class label
of an image. More precisely: is the image of a car or a plane? A cat or a dog?
 Applications

 Credit Scoring
– In the finance industry, supervised learning is used to assess creditworthiness. By training a model
on historical data that includes borrower information and their credit outcomes, it can predict
the probability of default or repayment behavior for new loan applications, aiding in risk
assessment.
 Medical Diagnosis
– Supervised machine learning plays a crucial role in medical diagnosis. By training models on
labeled medical data, such as patient symptoms and corresponding diagnoses, it can assist
healthcare professionals in diagnosing diseases, identifying patterns, and recommending
appropriate treatments.
 Stock Market Prediction
– Supervised learning can be applied to predict stock market trends and make investment decisions.
By training a model on historical stock data and relevant market indicators, it can provide insights
into potential price movements, aiding investors in making informed decisions.
 Benefits

Supervised machine learning offers several benefits, including:

 Accurate predictions: Supervised learning models can provide highly accurate predictions or
classifications when trained on a diverse and representative dataset.
 Versatility: It can be applied to a wide range of problem domains, making it a flexible approach for
various industries and applications.
 Interpretable results: Unlike some other machine learning approaches, supervised learning models
often provide interpretable results(detailed explanations for their decisions, such as which features
are most important for predicting results), allowing users to understand the reasoning behind
predictions.
 Limitations

However, it’s important to consider the limitations:

 Dependency on labeled data: Supervised learning relies heavily on labeled data, which can be
expensive and time-consuming to collect, especially for complex problems.
 Limited generalization: Models trained on specific datasets may struggle to generalize well to new
or unseen data that differ significantly from the training data distribution.
 Overfitting: If a model becomes overly complex or is trained on limited data, it may memorize the
training examples instead of learning underlying patterns, leading to poor performance on unseen
data.
 Unsupervised machine learning

 Unsupervised learning is a type of machine learning where the algorithm learns from
unlabeled data without any predefined outputs or target variables.
 The unsupervised learning finds patterns, similarities, or groupings within the data to get insights
and make data-driven decisions.
 These algorithms discover hidden patterns or data groupings without the need for human
intervention.
 It is particularly useful when dealing with large datasets where manual labeling would be
impractical or costly.
 Imagine you are in a foreign country and you are visiting a food market, You see a stall selling a
fruit that you cannot identify.You don’t know the name of this fruit. However, you can use your
observations as a reference. In this case, you can easily apart the fruit from nearby vegetables or
other food by identifying its various features like its shape, color, or size.
 How does Unsupervised Learning Work?
 Suppose an input dataset containing images of different types of cats and dogs. Here, we have taken an
unlabeled input data, which means it is not categorized and corresponding outputs are not given.
 Now, this unlabeled input data is fed to the machine learning model in order to train it.
 Firstly, it will interpret the raw data to find the hidden patterns from the data and then will apply suitable
algorithms such as K-means clustering algorithms.
 Once it applies the suitable algorithm, the algorithm divides the data objects into groups according to the
similarities and difference between the objects.
Thank You

30