DR. S. & S. S.

GHANDHY GOVERNMENT ENGINEERING COLLEGE

SURAT
ELECTRICAL ENGINEERING DEPARTMENT

AI and Machine learning (3170924)

Clustering in Machine Learning

Clustering or cluster analysis is a machine learning technique, which groups the unlabelled
dataset. It can be defined as "A way of grouping the data points into different clusters,
consisting of similar data points. The objects with the possible similarities remain in a group
that has less or no similarities with another group."

It does it by finding some similar patterns in the unlabelled dataset such as shape, size, color,
behavior, etc., and divides them as per the presence and absence of those similar patterns.

It is an unsupervised learning

method, hence no supervision is provided to the algorithm, and it deals with the unlabeled dataset.

After applying this clustering technique, each cluster or group is provided with a cluster-ID.
ML system can use this id to simplify the processing of large and complex datasets.

Example: Let's understand the clustering technique with the real-world example of Mall: When
we visit any shopping mall, we can observe that the things with similar usage are grouped
together. Such as the t-shirts are grouped in one section, and trousers are at other sections,
similarly, at vegetable sections, apples, bananas, Mangoes, etc., are grouped in separate
sections, so that we can easily find out the things. The clustering technique also works in the
same way. Other examples of clustering are grouping documents according to the topic.

The clustering technique can be widely used in various tasks. Some most common uses of this
technique are:

o Market Segmentation
o Statistical data analysis
o Social network analysis
o Image segmentation
o Anomaly detection, etc.

Apart from these general usages, it is used by the Amazon in its recommendation system to
provide the recommendations as per the past search of products. Netflix also uses this technique
to recommend the movies and web-series to its users as per the watch history.

The below diagram explains the working of the clustering algorithm. We can see the different
fruits are divided into several groups with similar properties.

1
Types of Clustering Methods
The clustering methods are broadly divided into Hard clustering (datapoint belongs to only
one group) and Soft Clustering (data points can belong to another group also). But there are
also other various approaches of Clustering exist. Below are the main clustering methods used
in Machine learning:

1. Partitioning Clustering
2. Density-Based Clustering
3. Distribution Model-Based Clustering
4. Hierarchical Clustering
5. Fuzzy Clustering

K-Means Clustering Algorithm

K-Means Clustering is an unsupervised learning algorithm that is used to solve the clustering
problems in machine learning or data science. In this topic, we will learn what is K-means
clustering algorithm, how the algorithm works, along with the Python implementation of k-
means clustering.

What is K-Means Algorithm?

K-Means Clustering is an Unsupervised Learning algorithm

, which groups the unlabeled dataset into different clusters. Here K defines the number of pre-defined
clusters that need to be created in the process, as if K=2, there will be two clusters, and for K=3, there
will be three clusters, and so on.

It is an iterative algorithm that divides the unlabeled dataset into k different clusters in such a way that
each dataset belongs only one group that has similar properties.
2
It allows us to cluster the data into different groups and a convenient way to discover the
categories of groups in the unlabeled dataset on its own without the need for any training.

It is a centroid-based algorithm, where each cluster is associated with a centroid. The main aim
of this algorithm is to minimize the sum of distances between the data point and their
corresponding clusters.

The algorithm takes the unlabeled dataset as input, divides the dataset into k-number of clusters,
and repeats the process until it does not find the best clusters. The value of k should be
predetermined in this algorithm.

The k-means clustering

algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative process.
o Assigns each data point to its closest k-center. Those data points which are near to the particular
k-center, create a cluster.

Hence each cluster has datapoints with some commonalities, and it is away from other clusters.

The below diagram explains the working of the K-means Clustering Algorithm:

3
How does the K-Means Algorithm Work?
The working of the K-Means algorithm is explained in the below steps:

Step-1: Select the number K to decide the number of clusters.

Step-2: Select random K points or centroids. (It can be other from the input dataset).

Step-3: Assign each data point to their closest centroid, which will form the predefined K
clusters.

Step-4: Calculate the variance and place a new centroid of each cluster.

Step-5: Repeat the third steps, which means reassign each data point to the new closest centroid
of each cluster.

Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.

Step-7: The model is ready.

Hierarchical Clustering in Machine Learning

Hierarchical clustering is another unsupervised machine learning algorithm, which is used to
group the unlabeled datasets into a cluster and also known as hierarchical cluster analysis or
HCA.

In this algorithm, we develop the hierarchy of clusters in the form of a tree, and this tree-shaped
structure is known as the dendrogram.

Sometimes the results of K-means clustering and hierarchical clustering may look similar, but
they both differ depending on how they work. As there is no requirement to predetermine the
number of clusters as we did in the K-Means algorithm.

The hierarchical clustering technique has two approaches:

4
 1. Agglomerative: Agglomerative is a bottom-up approach, in which the algorithm starts with
taking all data points as single clusters and merging them until one cluster is left.
2. Divisive: Divisive algorithm is the reverse of the agglomerative algorithm as it is a top-down
approach.

Why hierarchical clustering?

As we already have other clustering

algorithms such as K-Means Clustering

, then why we need hierarchical clustering? So, as we have seen in the K-means clustering that there
are some challenges with this algorithm, which are a predetermined number of clusters, and it always
tries to create the clusters of the same size. To solve these two challenges, we can opt for the
hierarchical clustering algorithm because, in this algorithm, we don't need to have knowledge about
the predefined number of clusters.

In this topic, we will discuss the Agglomerative Hierarchical clustering algorithm.

Agglomerative Hierarchical clustering

The agglomerative hierarchical clustering algorithm is a popular example of HCA. To group
the datasets into clusters, it follows the bottom-up approach. It means, this algorithm
considers each dataset as a single cluster at the beginning, and then start combining the closest
pair of clusters together. It does this until all the clusters are merged into a single cluster that
contains all the datasets.

This hierarchy of clusters is represented in the form of the dendrogram.

How the Agglomerative Hierarchical clustering Work?

The working of the AHC algorithm can be explained using the below steps:

o Step-1: Create each data point as a single cluster. Let's say there are N data points, so the
number of clusters will also be N.

5
 o Step-2: Take two closest data points or clusters and merge them to form one cluster. So, there
will now be N-1 clusters.

o Step-3: Again, take the two closest clusters and merge them together to form one cluster. There
will be N-2 clusters.

Step-4: Repeat Step 3 until only one cluster left. So, we will get the following clusters. Considerthe below
images

6
o Step-5: Once all the clusters are combined into one big cluster, develop the dendrogram to
divide the clusters as per the problem.

7
Introduction to Semi-Supervised Learning
Semi-Supervised learning is a type of Machine Learning algorithm that represents the
intermediate ground between Supervised and Unsupervised learning algorithms. It uses the
combination of labeled and unlabeled datasets during the training period.

Before understanding the Semi-Supervised learning, you should know the main categories
of Machine Learning algorithms. Machine Learning consists of three main categories:
Supervised Learning, Unsupervised Learning, and Reinforcement Learning. Further, the
basic difference between Supervised and unsupervised learning is that supervised learning
datasets consist of an output label training data associated with each and unsupervised
datasets do not consist the same. Semi-supervised learning is an important category that lies
between the Supervised and Unsupervised machine learning. Although Semi- supervised
learning is the middle ground between supervised and unsupervised learning and operates on
the data that consists of a few labels, it mostly consists of unlabeled data. As labels are costly,
but for the corporate purpose, it may have few labels.

The basic disadvantage of supervised learning is that it requires hand-labeling by ML

specialists or data scientists, and it also requires a high cost to process. Further unsupervised
learning also has a limited spectrum for its applications. To overcome these drawbacks of
supervised learning and unsupervised learning algorithms, the concept of Semi-
supervised learning is introduced. In this algorithm, training data is a combination of both
labeled and unlabeled data. However, labeled data exists with a very small amount while it
consists of a huge amount of unlabeled data. Initially, similar data is clustered along with an
unsupervised learning algorithm, and further, it helps to label the unlabeled data into labeled
data. It is why label data is a comparatively, more expensive acquisition than unlabeled data.

We can imagine these algorithms with an example. Supervised learning is where a student is
under the supervision of an instructor at home and college. Further, if that student is self-
analyzing the same concept without any help from the instructor, it comes under unsupervised
learning. Under semi-supervised learning, the student has to revise itself after analyzing the
same concept under the guidance of an instructor at college.

8
Assumptions followed by Semi-Supervised Learning

To work with the unlabeled dataset, there must be a relationship between the objects. To
understand this, semi-supervised learning uses any of the following assumptions:

o ContinuityAssumption:

As per the continuity assumption, the objects near each other tend to share the same group or
label. This assumption is also used in supervised learning, and the datasets are separated by the
decision boundaries. But in semi-supervised, the decision boundaries are added with the
smoothness assumption in low-density boundaries.

o Cluster assumptions- In this assumption, data are divided into different discrete clusters.
Further, the points in the same cluster share the output label.

o Manifold assumptions- This assumption helps to use distances and densities, and this data lie
on a manifold of fewer dimensions than input space.

o The dimensional data are created by a process that has less degree of freedom and may be hard
to model directly. (This assumption becomes practical if high).

Working of Semi-Supervised Learning

Semi-supervised learning uses pseudo labeling to train the model with less labeled training data
than supervised learning. The process can combine various neural network models and training
ways. The whole working of semi-supervised learning is explained in the below points:

o Firstly, it trains the model with less amount of training data similar to the supervised learning
models. The training continues until the model gives accurate results.
o The algorithms use the unlabeled dataset with pseudo labels in the next step, and now the result
may not be accurate.
o Now, the labels from labeled training data and pseudo labels data are linked together.
o The input data in labeled training data and unlabeled training data are also linked.
o In the end, again train the model with the new combined input as did in the first step. It will
reduce errors and improve the accuracy of the model.

Difference between Semi-supervised and Reinforcement Learning.

Reinforcement learning is different from semi-supervised learning, as it works with rewards
and feedback. Reinforcement learning aims to maximize the rewards by their hit and trial
actions, whereas in semi-supervised learning, we train the model with a less labeled dataset.

9
Real-world applications of Semi-supervised Learning-
Semi-supervised learning models are becoming more popular in the industries. Some of the
main applications are as follows.

o Speech Analysis- It is the most classic example of semi-supervised learning applications.

Since, labeling the audio data is the most impassable task that requires many human resources,
this problem can be naturally overcome with the help of applying SSL in a Semi-supervised
learning model.
o Web content classification- However, this is very critical and impossible to label each page
on the internet because it needs mode human intervention. Still, this problem can be reduced
through Semi-Supervised learning algorithms. Further, Google also uses semi-supervised
learning algorithms to rank a webpage for a given query.
o Protein sequence classification- DNA strands are larger, they require active human
intervetion. So, the rise of the Semi-supervised model has been proximate in this field.
o Text document classifier- As we know, it would be very unfeasible to find a large amount of
labeled text data, so semi-supervised learning is an ideal model to overcome this.

10
What is Deep Learning?
"Deep learning is defined as the subset of machine learning and artificial intelligence that is based on
artificial neural networks". In deep learning, the deep word refers to the number of layers in a neural
network.

Deep Learning is a set of algorithms inspired by the structure and function of the human brain. It uses
a huge amount of structured as well as unstructured data to teach computers and predicts accurate
results. The main difference between machine learning and deep learning technologies is of presentation
of data. Machine learning uses structured/unstructured data for learning, while deep learning uses neural
networks for learning models.

In machine learning, if a model predicts inaccurate results, then we need to fix it manually. Further, in
deep learning techniques, these problems get fixed automatically, and we do not need to do anything
explicitly. A self-driving vehicle is one of the best examples to understand deep learning.

Deep learning can be useful to solve many complex problems with more accurate predictions such
as image recognition, voice recognition, product recommendations systems, natural language
processing (NLP), etc.

The basic structure of deep learning

Deep learning includes various neural networks that possess different layers, such as input layers,
hidden layers, and output layers. The input layer accepts input data; hidden layers are used to find any
hidden pattern and feature from the data, and output layers show the expected results.

How does deep learning work?

There are a few simple steps that deep learning follows.

1. Calculate the weighted sum

2. Use this weighted sum in step1 as input for the activation function.
3. The activation function adds bias and decides whether the neuron should be triggered or not.
4. Predict output at the output layer.

11
 5. Compare predicted output and actual output and accordingly use the backpropagation method
for improving the performance of the model. In this step, the cost function plays a vital role in
reducing the error rate.

Types of deep neural networks

There are some different types of deep learning networks available. These are as follows:

o Feedforward neural network

o Radial basis function neural networks
o Multi-layer perceptron
o Convolution neural network (CNN)
o Recurrent neural network
o Modular neural network
o Sequence to sequence models

Applications of deep learning

o Deep learning can be applied in various industries such as:
o Self-driving vehicles
o Fraud detection
o Natural language processing
o Virtual personal assistance
o Text, speech, and image recognition
o Healthcare, infrastructure, banking & finance, marketing
o Entertainment
o Education
o Automatic game playing
o Auto handwriting generation
o Automatic language translation
o Pixel restoration and photo description & tagging
o Demographic and election predictions, etc.

12
Deep learning vs. Machine learning vs. Artificial
Intelligence
Deep Learning, Machine Learning, and Artificial Intelligence are the most used terms on the
internet for IT folks. However, all these three technologies are connected with each other.
Artificial Intelligence (AI) can be understood as an umbrella that consists of both Machine
learning and deep learning. Or We can say deep learning and machine learning bothare subsets
of artificial intelligence.

As these technologies look similar, most of the persons have misconceptions about 'Deep
Learning, Machine learning, and Artificial Intelligence' that all three are similar to each other.
But in reality, although all these technologies are used to build intelligent machines or
applications that behave like a human, still, they differ by their functionalities and scope.

It means these three terms are often used interchangeably, but they do not quite refer to the
same things. Let's understand the fundamental difference between deep learning, machine
learning, and Artificial Intelligence with the below image.

With the above image, you can understand Artificial Intelligence is a branch of computer
science that helps us to create smart, intelligent machines. Further, ML is a subfield of AI that
helps to teach machines and build AI-driven applications. On the other hand, Deep learning is
the sub-branch of ML that helps to train ML models with a huge amount of input and complex
algorithms and mainly works with neural networks.

13