FR
RÉSIDENCES FABRIKAM
Deep Learning
Ms Mounira Zouaghi
Content
1 Introduction to deeplearning
Fundamentals of deeplearning
2
Deeplearning frameworks:PyTorch,
3 TensorFlow, Keras…
4 Types of Neural Network
5 Use cases of Neural Network
2
Chapter2: Fundamentals of Deep Learning
What is Deep Learning
a subset of machine learning that's based
on artificial neural networks.
The learning process is deep because the
structure of artificial neural networks
consists of multiple input, output, and
hidden layers.
Each layer contains units that transform the
input data into information that the next
layer can use for a certain predictive task.
Thanks to this structure, a machine can
learn through its own data processing.
Artificial Neural Netwoks
An artificial neural network (ANN) is a
Input layer hidden layer
computational model to perform tasks like
prediction, classification, decision making, etc.
wi Artificial neurons are a copy of human brain
output layer neurons.
It consists of artificial neurons. T
Neurons in the brain pass the signals to perform
the actions.
Similarly, artificial neurons connect in a neural
network to perform tasks.
The connection between the artificial neurons is
called synapsis ,
neuron synapsis
Every synapsis having a weight.
How does it work in simple way?
https://www.youtube.com/watch?v=ER2It2mIagI
What is Neuron?
Biological Neuron Artificial Neuron
Dendrite Inputs
Cell nucleus or Soma Nodes
Synapses Weights
Axon Output
How does Neuron work?
Activation function(1)
• Is the first mathematical operation done by
neuron
step1
y=
bi: bias
step2
z=Activation(y)=f(y)
Activation function (2)
What?
Activation function is a mathematical function that we use to get the output of Node called also
Transfer Function,
Why?
Used to determine the output of neural network like yes or no (Neuron is Activated or Not). Output is generally
between 0 and 1
Examples ?
How to train NN with BackProbagation
𝜕𝑙𝑜𝑠𝑠
𝑤 1𝑛𝑒𝑤=𝑤 1 𝑜𝑙𝑑 −𝜇
𝜕𝑤 1
y: correct result
x1
w1 𝝁 : 𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈 𝑹𝒂𝒕𝒆
w4 o4
x2 w2 ŷ The same process will be
done some number of
w3 times (epochs) until loss
𝑙𝑜𝑠𝑠=( 𝑦 − ŷ ) 2
function will be reduced
x3
to 0
How does it work?
https://www.youtube.com/watch?v=bfmFfD2RIcg
https://www.youtube.com/watch?v=ER2It2mIagI
Train Multilayer NNT: Gradient Descent
Hidden Hidden
input Layer
Layer1 Layer2
w11
x1
f11 O11
w21 𝝁 : 𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈 𝑹𝒂𝒕𝒆
w31 O/p Layer
x2 w41
O12 f21 O21
f 12
f21
ŷ
O22
y: eventuel result
x3
O13
f22 Loss=(y-ŷ) 2
ŷ : calculated result
f 13
x4
Gradient Descent Optimizer
Initialization of NN Parameters:Weights and
Repeat to obtain bayes
the minimum cost
Compute Gradient: calculate derivated with
respect of each parameter
Update parameters
Types of Gradient Descent
SGD: Stochastic Gradient Descent: In SGD, a random mini-batch of training examples is used to estimate the
gradient in each iteration. This introduces randomness and can lead to faster convergence and escape local
minima
Batch Gradient Descent: In this variant, the entire training dataset is used to compute the gradient in each
iteration. It can be computationally expensive for large datasets but is more stable than SGD
Mini-Batch Gradient Descent:This approach combines the benefits of full-batch gradient descent and SGD by
using a small random subset (mini-batch) of the training data in each iteration.
Chain Rule in Backpropagation
Hidden Hidden
input Layer
Layer1 Layer2
w111 𝝁 : 𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈 𝑹𝒂𝒕𝒆
x1 w 11 2
f11
w121 O11 𝝏 𝒍𝒐𝒔𝒔 𝝏 𝒍𝒐𝒔𝒔 𝝏O31
= ∗
O/p Layer 𝝏 w 3 11 𝝏O 3 1 𝝏 w 311
w212
x2
w131
f21 O21
f 12
O12 w311
f21
ŷ O31
w141 O22
x3
f22 w321
()
f 13
O13
x4
y: eventuel result
Loss=(y-ŷ) 2
ŷ : calculated result
Types of NN
Perceptron
Feed Forward Neural Network
Multilayer Perceptron
Convolutional Neural Network
Radial Basis Functional Neural Network
Recurrent Neural Network
LSTM – Long Short-Term Memory
Sequence to Sequence Models
Modular Neural Network
Perceptron (1)
Perceptron model, proposed by Minsky-Papert is one of the
simplest and oldest models of Neuron.
It is the smallest unit of neural network that does certain
computations to detect features or business intelligence in
the input data.
It accepts weighted inputs, and apply the activation function
to obtain the output as the final result. Perceptron is also
known as TLU(threshold logic unit)
Perceptron is a supervised learning algorithm that classifies
the data into two categories, thus it is a binary classifier.
Perceptron (2)
Avantages Disadvantages
Perceptrons can implement Logic Gates l Perceptrons can only learn linearly separable
like AND, OR, or NAND. problems such as boolean AND problem.
For non-linear problems such as the boolean XOR
problem, it does not work.
Bibliography
Types of NN accessible on https://www.mygreatlearning.com/blog/types-of-neural-networks/ last access
2023 October
