Subset of Machine learning, Algorithms
inspired by the structure and function of
Human Brain
Definition
Structure
Structure
Neuron
Neuron takes input and do some
function to give the output
Function going to be the
Mathematical function
Basic Neural Network Structure
Structure
Layer is Made up of Nodes of
Neuron & Weights
Input Layer
Structure Hidden Layer
Output Layer
Layer
Fully connected layers connect
1 every neuron in one layer to
every neuron in the next layer
Fully Connected Layer
Extracting features in images,
Types which uses a filter to scan an
2 image, a few pixels at a time, and
outputs a feature map that
classifies each feature.
Convolution Layer
Pooling layers are used to reduce
the dimensions of the feature
maps. Thus, it reduces the
3
number of parameters to learn
and the amount of computation
Pooling Layer
performed in the network.
Numeric Values from Neural
Networks, machine learns by
itself. They self-adjust depending
Weights
on the difference between
predicted outputs vs training
inputs.
It is a mathematical formula
which helps the neuron to switch
ON/OFF
Elements
Graph
If value of X is greater than or
Step Function equal to 0, then output is 1, If
value of X is less than 0, then
output is 0
Since step Function is non
differentiable to zero, it can’t do
the gradient descent method, so
Activation Function it can’t update weights.
Graph
If value of X is infinity, then
output is 1, If value of X is
negative infinity, then output is 0
Sigmoid Function
It captures non-linearity in the
data
Types
It can use Gradient descent &
Back propagation method to
calculate weights.
Output range [0,1]
Formula
Graph
Softmax Function
It is used to map the non-normalized output
of a network to a probability distribution
over predicted output class.
Deep Learning
That is it converts output of last layer into a
essential probability distribution.
Tanh Function
ReLU Function
variables which determines the
network structure, how the
network is trained(Eg: Learning
Rate, No. of Units).
Dropout
Hyper Parameter
Activation function
Learning Rate
Types
Number of epochs
Batch size
Number of neurons
It is the simplest form of a neural
1 Perceptron
network
Structure
MLP's decision function is a step
2 Multilayer Perceptron
function & Output is Binary
Structure
Simplest form of neural networks
where input data travels in one
3 Feed Forward Neural Networks
direction only, passing through
artificial neural nodes
Structure
3 It contains a three-dimensional
arrangement of neuron
4 Convolutional Neural Network
Types Each neuron processes the
Structure information from a small part of
the visual field
Example
In Single Perceptron / Multi-layer AND, OR functions are linearly-
Perceptron(MLP), we only have separable & XOR function is not
linear separability linearly separable.
Radial Basis Function Neural It transforms the input signal into
5 another form, which can be then
Networks
feed into the network to get
linear separability
Structure It has exactly one hidden layer
It will remember the things from
6 Recurrent Neural Networks
Previous input to generate output
Structure
Numerical computation using data flow
graphs | Backend for Keras | Distributed
1 computing on multiple GPU
Tensor Flow
Framework for Deep learning | Same code
for CPU & GPU | Uses Theano/TF as
2 Backend | CNN | not support multi GPU
Keras
Old ML & DL library | Supports CUDA -
Compute Unified Device Architecture for
3 parallel computation | Supervised image
problem with CNN
4 Torch
Caffe is a deep learning framework made
Libraries with expression, speed, and modularity in
4 mind.Very high performance | Tuning Hyper
parameters | CNN & RNN(Facebook
extends) | | CPU & GPU
Caffe
Apache MXNet is an open-source deep
learning software framework, used to train,
5 and deploy deep neural networks|More
number of Language Bindings | Distributed
Mxnet
computing
Basic operation such as Sorting, Reshaping,
indexing | Scientific computing | Most new
6 feature belong in SciPy rather than Numpy
Numpy & Scipy
1 Fruit & vegetable classification
Segregator
2 AI in Cooking
Molecular Structure Analysis
5 3 AI in Autonomous Vehicle
Applications How Autonomous Car See
4 AI Doctor
Hospital Receptionist with Basic Doctor
Functions
Prediction of Disease from
5
Medical Image
Medical Image Analysis
