0 ratings0% found this document useful (0 votes) 27 views6 pagesRad Assignment (KNN)
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content,
claim it here.
Available Formats
Download as PDF or read online on Scribd
KNN Algorithm - Finding Nearest Neighbours
Introduction
K-Nearest Neighbour is one of the simplest Machine Learning algorithms
based on Supervised Learning technique.
K.NN algorithm assumes the similarity between the new case/data and
available eases and put the new case into the category that is most similar to
the available categories,
K-NN algorithm stores all the available data and classifies a new data point
based on the similarity. This means when new data appears then it can be
classified into a well suite category by using K- NN algorithm.
K-NN algorithm can be used for Regression as well as for Classification but
mostly it is used for the Classification problems.
+ Lazy learning algorithm — KNN is a lazy learning algorithm because it
does nothave a specialized training phase and uses all the data for training
while classification
+ Non-parametric learning algorithm - KNN is also a non-parametric
learning algorithm because it doesn’t assume anything about the underlying
data.
Why do we need a K-NN Algorithm?
Suppose there are two categories, i.e., Category A and Categary B, and we have a
‘new data paint x1, so this data paint will Lie in which of these categaries. Te solve
this type of problem, we need a K-NN algorithm. With the help of K-NN, we can
easily identify the category ar class of a particular dataset. Consider the below
diagram:
Scanned with CamScannerWorking of KNN Algorithm
K-nearest neighbors (KNN) algorithm uses ‘feature similarity’ to predict the
values of new datapoints which further means that the new data point will be
assigned a value based on how closely it matches the points in the training set. We
can understand its working with the help of following steps ~
Step 1 - For implementing any algorithm, we need dataset. So during the first
step of KINN, we must load the training as well as test data.
Step 2 - Next, we need to choose the value of
be any integer.
the nearest data points. K can
Step 3 - For each point in the test data do the following ~
+ 2.1 ~ Calculate the distance between test data and each row of training data
with the help of any of the method namely: Euclidean, Manhattan or
Hamming distance. The mast commonly used method to calculate distance
is Euclidean,
+ 3.2 -Now, based on the distance value, sart them in ascending order.
+ 3.3—Next, it will choose the top K rows fromthe sorted array.
+ 34 - Now, it will assign a class to the test paint based on most frequent
class of these rows
Step 4~ End
As we know K-nearest neighbors (KNN) algorithm can be used for both
classificationas well as regression. The follawing are the recipes in Python ta use.
Scanned with CamScannerKWNN as classifier as well as regressor —
Scanned with CamScannerKUN as Regressor
‘Scanned with CamScannerPros and Cons of KNN
Pros
+ itis very simple algorithen 10 understand and interpret
+ Itis very useful for nonlinear data because there is no assumption about data in
this algarithin,
+ itis a versatile algorithm as we can use it for classification as well as regression
+ It has relatively high accuracy but there are much better supervised learning
models than KNN
‘Cons
+ itis computationally a bit expensive algorithm because it stores all the training
data.
+ High memory storage required as compared to other supervised teaming
algorithms
= Prediction is slow in case ot big N.
+ ILS Very Sensitive to the Seale of data as well as irrelevant features.
Applications of KNN
The following are some of the areas in which KNN can be applied succes fully ~
© Banking System
KNN can be used in banking system to predict weather an individual is fil for loan
‘approval? Does thal individual have the charactenstics similar to the defauiters one?
* Calculating Credit Ratings
KNN algorithms can be used fo find an individual's credit rating by comparing with the
persons having similar traits.
* Politics
With the help ef KNN algorithms. we can classify a potential voter into various classes
ke “Viill Vole", “Vuill nat Vote", “Will Vole to Party ‘Congress’, “Will Vole to Party ‘BJP’.
Other areas in which KNN algorithm can be used are Spaech Recagnition, Handwriting
Detection, Image Recognition and Video Recognition
Scanned with CamScannerScanned with CamScanner