08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python for
Machine/Deep Learning Models
Kamal Pandey
Geoweb Services, IT & Distance Learning Department
Indian Institute of Remote Sensing (IIRS), ISRO Dehradun
[email protected]
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Content
Brief about ML/DL
Why Python for ML/DL
Feature of python for ML/DL
Libraries in python for ML/DL
Working environment of python for ML/DL
Example and use cases
Linear Regression: Wheat Crop yield estimation
Splitting the dataset to Train and Test
1
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning and Deep Learning
Machine Learning is a set of
algorithms that parse data, learn
from them, apply what they have
learnt to make intelligent
decision,
Deep learning is gaining much
popularity due to it’s supremacy
in terms of accuracy when
trained with huge amount of
data
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
? Python for Machine Learning and Deep Learning
• Free and open-source nature
• Community friendly and guarantees improvements
in the long run
• Exhaustive libraries
• Solution for every existing problem
• Smooth implementation and integration
• Accessible for people with the varying skill level to
adapt it
2
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
? Python for Machine Learning and Deep Learning
• Increased productivity by reducing the time to
code and debug
e.g. mymodel.LinearRegression()
• Can be used for Soft Computing, Natural
Language Processing as well
• Works seamlessly with C and C++ code
modules
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Features of Python w.r.t. ML/DL
• Easy/Simple Learning Curve
• General Purpose Language
• Ready made packages and libraries for machine learning
• Matplotlib
• Numpy
• Scipy
• scikit-learn
• tensorflow Excellent support for Deep Learning Models
• Interactive data analysis and modelling using iPython Notebooks
• Industry standard language for AI and machine Learning
3
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries : Machine Learning and Deep Learning
Numpy and
Panda MXNet
Matplotlib
Tensor
Theano Keras
Flow
Spark Mllib Scikit-lean Pytorch
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries : Deep Learning
• Handling deep neural networks
• Natural Language Processing
• Partial Differential Equation
TensorFlow • Abstraction capabilities
• Image, Text, and Speech
recognition
• Effortless collaboration of ideas
and code
Offered by Google, fast, flexible, and scalable open-source
4
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries : Deep Learning
• Neural layers
• Activation and cost functions
• Objectives
Keras
• Batch normalization
• Dropout
• Pooling
Support for the convolutional and recurrent neural network also exists
along with standard neural networks.
Keras is the high-level API of TensorFlow 2
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries : Deep Learning
• Tensor computing with the ability via
Graphics Processing Units
• Easy to learn, use and integrate with Python
ecosystem
PyTorch • Tensors — torch.Tensor
• Optimizers —
• Neural Networks —
• Autograd – Automatic gradiant calculation
5
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Machine Learning
• Classification
• Regression
• Clustering
Scikit-learn
• Dimensionality Reduction
• Model Selection
• Preprocessing
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Data preparation/processing
• Dataset reshaping and pivoting
• Merging and joining of datasets
• Handling of missing data and data
Pandas alignment
• Various indexing options such as
Hierarchical axis indexing, Fancy indexing
• Data filtration options
Pandas make use of DataFrames (2 dimensional data structure)
6
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Machine Learning
• Regression
• Clustering
• Optimization
Spark Mllib • Dimensional Reduction
• Classification
• Basic Statistics
• Feature Extraction
Spark Mllib is a machine learning library that enables easy scaling of your computations
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Machine Learning
• Support for GPUs
• Strong integration support with NumPy
Theano
• Ability to create custom C code for your
mathematical operations
Theano a robust library for carrying out scientific calculations on a large-scale
7
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Deep Learning
• Highly scalable and supports quick model training
MXNeT • Used to train and deploy deep neural networks
• Excellent Portability and Scalability
•Amazon’s AWS prefers MXNet as its choice of preferred deep learning framework.
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python Libraries: Data preparation/processing
• Shape manipulation
• Sorting and Selecting capabilities
Numpy & • Discrete Fourier transformations
Matplotlib • Basic linear algebra and statistical operations
• Random simulations
• Support for n-dimensional arrays
The NumPy library for Python concentrates on handling extensive multi-
dimensional data and the intricate mathematical functions operating on the data
8
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Python IDE for Machine Learning
• Jupyter Notebook in Anaconda
• Google Colab
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
In Machine Learning, predicting
the future is very important.
Python Code Examples
• Linear Regression using Least Square Estimation
9
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Linear Regression
• The term regression is used when you try to find
the relationship between variables.
• In Machine Learning, and in statistical modeling,
that relationship is used to predict the outcome of
future events.
• Linear regression uses the relationship between the
data-points to draw a straight line through all them.
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Linear Regression
• This line can be used to predict future values.
10
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Least Square Estimation
A common problem in experimental work is to obtain a
mathematical relationship y=f(x) between two variables
x and y by fitting a curve to the points in the plane
corresponding to various experimentally determined
values x and y, say:
(x1,y1) , (x2,y2),….,(xn,yn)
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Polynomials
On the basis of theory, or simply by the pattern of
points one decide on the general form of the curve
y=f(x) to be fitted.
Some possibilities are:
• Straight line: y=a+bx
• Quadratic polynomial: y=a+bx+cx2
• Cubic polynomial: y=a+bx+cx2+ dx3
11
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Straight Line
Suppose we want to fit a straight line:
y=a+bx
to the experimentally determined y1 =a+bx1
points: y2 =a+bx2
…..
(x1,y1) , (x2,y2),….,(xn,yn)
yn =a+bxn
If these points are on the straight line
then the following equalities hold for
these points:
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Straight Line
We can write this in a matrix form as :
𝑦1 1 𝑥1
𝑦2 1 𝑥2 𝑎
… =
… … 𝑏
𝑦𝑛 1 𝑥𝑛
or, more compactly, as ; 𝑦ത = 𝑀𝑣ҧ
𝑦1 1 𝑥1
𝑦2 1 𝑥2 𝑎
where: 𝑦=
ത … 𝑀= 𝑣=
ҧ
… … 𝑏
𝑦𝑛 1 𝑥𝑛
12
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Least Square Fitting
If the data points are not collinear,
then it’s impossible to find coefficient
a and b that satisfy 𝑦ത = 𝑀𝑣ҧ exactly.
Minimizing the error vector is done
using a technique that is called ഥ=(𝑴𝑻 𝑴)−𝟏 𝑴𝑻 𝒚
𝒗 ഥ
least square fitting.
A solution to the least square fit can
be found using the following formula:
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Example
Suppose we have the following 4 observations:
(0,1),(1,3),(2,4),(3,4)
1 1 0
3 1 1
From the data: 𝑦=
ത 𝑀=
4 1 2
4 1 3
𝑇 1.5
Solution: 𝑣=(𝑀
ҧ 𝑀)−1 𝑀𝑇 𝑦ത =
1
And so the desired best fitting line is: y=1.5+x
13
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
In Python
from numpy import *
x=matrix([[0],[1],[2],[3]])
y=matrix([[1],[3],[4],[4]])
M=matrix([1,0],[1,1],[1,2],[1,3])
v=(M.T*M).I*M.T*y [[1.5]
[1. ]]
print v
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Plotting the result
from pylab import *
plot(x,y, 'ro')
x=arange(-1,5)
y=1.5+x
plot(x,y,'b-')
14
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Quadratic Curve
The same mathematical trick can be applied to
the higher order polynomials.
Fit a quadratic curve:
𝑠 = 𝑎0 + 𝑎1 𝑡 + 𝑎2 𝑡 2
to five data points (t,s):
(0.1,-0.18),
(0.2,0.31),(0.3,1.03),(0.4,2.48),(0.5,3.73)
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Matrix Form
Again, the equations: can be written in a matrix form:
𝑠1 = 𝑎0 + 𝑎1 𝑡1 + 𝑎2 𝑡12 𝑦ത = 𝑀𝑣ҧ
𝑠2 = 𝑎0 + 𝑎1 𝑡2 + 𝑎2 𝑡22
where:
𝑠3 = 𝑎0 + 𝑎1 𝑡3 + 𝑎2 𝑡32
𝑠1 1 𝑡1 𝑡12
𝑠4 = 𝑎0 + 𝑎1 𝑡4 + 𝑎2 𝑡42 1 𝑡2 𝑡22
𝑠2
𝑠5 = 𝑎0 + 𝑎1 𝑡5 + 𝑎2 𝑡52 𝑦=
ത 𝑠3 𝑀= 1 𝑡3 𝑡32 𝑣=
ҧ
𝑠4 1 𝑡4 𝑡42
𝑠5
1 𝑡5 𝑡52
𝑎0
𝑎1
𝑎2
15
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
In Python
from numpy import *
x=matrix([[0.1],[0.2],[0.3],[0.4],[0.5]])
y=matrix([[-0.18],[0.31],[1.03],[2.48],[3.73]])
M=matrix(ones((5,3)))
M[:,1]=x
M[:,2]=multiply(x,x)
[[-0.398 ]
[0.34714286]
v=(M.T*M).I*M.T*y [16.07142857]]
print (v)
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Plotting the results
𝑠 = −0.40 + 0.35𝑡 + 16.1𝑡 2
from pylab import *
plot(x,y, 'ro')
x=arange(0.0,0.52,0.01)
y=v[0,0]+v[1,0]*x+v[2,0]*multiply(x,x)
plot(x,y,'b-')
16
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Multiple Regression
The mathematical “trick” of finding a matrix
form for the equations can be extended to
multiple dimensions.
In that case we call it multiple regression
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Linear Regression
import numpy as np
from sklearn.linear_model import LinearRegression
X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
# y = 1 * x_0 + 2 * x_1 + 3
y = np.dot(X, np.array([1, 2])) + 3+np.random.rand()
reg = LinearRegression()
reg.fit(X, y)
reg.score(X, y)
reg.coef_
reg.intercept_
reg.predict(np.array([[3, 5]]))
𝑦 = 𝑥0 + 2𝑥1 + 3
17
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Linear Regression
Crop Yield Estimation
import pandas
from sklearn import linear_model
df = pandas.read_excel("/content/drive/MyDrive/MLData/CropYeildPredictionSampleData.xlsx")
df=df.dropna(how='any')
df= df.drop(columns=['District','Year','Crop', 'Production(Tonnes)'])
y=df.pop('Yield(Tonnes/Ha)')
x =df.values
y=y.to_numpy()
y= y.reshape(-1,1)
model_yield = linear_model.LinearRegression(normalize=True)
model_yield.fit(x, y)
model_yield.predict(x[0].reshape(1,-1))
pred_y=regr.predict(x)
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Linear Regression
Data Splitting: Train and Test
import pandas
from sklearn.model_selection import train_test_split
from sklearn import linear_model
df = pandas.read_excel("/content/drive/MyDrive/MLData/CropYeildPredictionSampleData.xlsx")
df=df.dropna(how='any')
df= df.drop(columns=['District','Year','Crop', 'Production(Tonnes)'])
y=df.pop('Yield(Tonnes/Ha)')
x =df.values
y=y.to_numpy()
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.33, random_state=42)
model_yield = linear_model.LinearRegression(normalize=True)
model_yield.fit(X_train, y_train)
y_predicted= regr.predict(X_test)
pylab.scatter(y_predicted,y_test)
pylab.xlabel('Predicted')
pylab.ylabel('Actual')
18
08-07-2021
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
Machine Learning : Clustering
from sklearn.cluster import KMeans
import numpy as np
X = np.array([[1, 2], [1, 4], [1, 0],[10, 2], [10, 4], [10, 0]])
kmeans = KMeans(n_clusters=2,random_state=0).fit(X)
kmeans.labels_
kmeans.predict([[0, 0], [12, 3]])
kmeans.cluster_centers_
I N D I A N I N S T I T U T E O F R E M O T E S E N S I N G, D E H R A D U N
THANK YOU
19
