Supervised Learning
of Neural Networks
P R E P A R E D B Y
M R S . D . T E J A S W I , I T
A N I T S S
Contents
• Introduction
• Perceptrons
• Adaline Back Propagation Multilayer Perceptrons
• Back Propagation Learning Rules
• Methods of Speeding
• Radial Basis Function Networks
• Functional Expansion Networks
Introduction
• As the name suggests, supervised learning takes place under the
supervision of a teacher.
• This learning process is dependent.
• During the training of ANN under supervised learning, the input
vector is presented to the network, which will produce an output
vector.
• This output vector is compared with the desired/target output vector.
• An error signal is generated if there is a diff erence between the
actual output and the desired/target output vector.
• On the basis of this error signal, the weights would be adjusted until
the actual output is matched with the desired output.
Can a single neuron learn a task?
• In 1958, Frank Rosenblatt introduced a training
algorithm that provided the first procedure for
training a simple ANN : a Perceptron.
• The perceptron is the simplest form of a neural
n/w.
• It consists of a single neuron with adjustable
synaptic weights and a hard limiter
The Main aim of perceptron is classify inputs, x1,x2,x3……….xn, into
one of two classes , say A1 and A2
Components:
1. Input Layer
2. Weights
3.Bias
4. Activation Function
5. Output
6. Training Algorithm
Types of Perceptrons
1. Single Layer Perceptron model: The main
objective of the single-layer perceptron model
is to analyze the linearly separable objects with
binary outcomes. A Single-layer perceptron can
learn only linearly separable patterns.
2. Multi-Layered Perceptron model: It is mainly
similar to a single-layer perceptron model but
has more hidden layers.
Single Layer Perceptron
model
Single Layer Perceptron model
Algorithm(For each Epoch)
Step 1: The perceptron model begins with
multiplying all input values and their
weights, then adds these values to create
the weighted sum
Step 2: Further, this weighted sum is applied
∑wi*xi = x1*w1 + x2*w2 + x3*w3+……..x4*w4
to the activation function ‘f’ to obtain the
desired output.
Y=f(∑wi*xi + b)
Step -3
Step 4: Update the weights of the Perceptrons
Step 5 :Alter the weights and bias using the Perceptron learning run
the show.
Example:
Implement a single-layer perceptron to learn the AND logical
operation. You are given two binary inputs, X1and X2, and the
initial weights and bias are set to zero (w1=0, w2=0, b=0) and
learning rate α=1
X1 X2 Target
(t)
1 1 1
1 -1 -1
-1 1 -1
-1 -1 -1
Solution
• Epotch-1
Inputs Target Y(in) Expect ∆w1 ∆w ∆b1 Updated
Output ed O/P 2 Weights
(W1=0,
W2=0,b
=0)
X1 X2 t Y WW b
12
1 1 1 0 0 1 1 1 11 1
1 -1 -1 1 1 -1 1 -1 02 0
-1 1 -1 2 1 1 -1 -1 11 -1
-1 -1 -1 -3 -1 0 0 0 11 -1
Continue
• Epotch-2
Inputs Target Yn) Expecte Updated
Output d O/P Weights
(W1=1,W2=1,b
=-1)
X1 X2 t Y W1 W2 b
1 1 1
1 -1 -1
-1 1 -1
-1 -1 -1
ADVANTAGES DRAWBACKS
• Simplicity and • Linear
Ease of Separability
Understanding
Ex: XOR Problem
• Low
• Limited to Binary
Computational
Classifi cation
Cost
• Lack of Hidden
• Fast Training
Layers
• Binary
• Over fi tting
Classifi cation
• Generalization
ADALINE Multi perceptron
• ADALINE is an acronym for ADAptive LINear Element (or
ADAptive LInear NEuron). It was developed by Bernard
Widrow and Marcian Hoff (1960).
• The Adaline learning rule (also known as the least-
mean-squares rule, the delta rule, and the Widrow-Hoff
rule) is a training rule that minimizes the output error
using (approximate) gradient descent.
• After each training pattern I p is presented, the
correction to apply to the weights is proportional to the
error.
• The correction is calculated before the thresholding
step, using err ij (p)=T p -W ij I p :
ADALINE Multi perceptron
ADALINE Multi perceptron
PEDAGOGY
Example- Design a OR Gate using
Adaline
Assume W1,W2,b=0.1
Leaning rate α=0.1
Error Rate Er=2
Solution
• Epotch-1
Inputs Target Y(in)= Expecte Updated Error
Output b+∑xi d O/P Weights
wi (W1=0.1,W2=0.
1, b=0.1)
X1 X2 t (t-yin) W1 W2 b (t-
yin)2
1 1 1
1 -1 1
-1 1 1
-1 -1 -1
Solution
• Epotch-2
Inputs Target Y(in)= Expecte Updated Error
Output b+∑xi d O/P Weights
wi (W1=0.26,W2=
0.27, b=0.27)
X1 X2 t (t-yin) W1 W2 b (t-
yin)2
1 1 1
1 -1 1
-1 1 1
-1 -1 -1
EXAMPLE -2
Continue the same process
Continue the same
process untill you satify
with error rate
Radial Bias Function in Neural network
RBFNN-Introduction
• Data can be either linearly/ non-linearly
separable.
• Single layer perceptron can be used for
classifying linearly separable data.
• Multi layer perceptron can be used for
classifying non-linearly separable data.
RBFNN-Introduction
• Radian Bias Function is a type of multi layer
perceptron which has one input layer, one
output layer and strictly one hidden layer.
•The hidden layer uses a non-
linear radial bias function as
the activation function, Which
converts the i/p parameter into
high dimension space which is
then fed into the n/w to
linearly separate the problem.
• Typical RBF’s are
1. Gaussian RBF
2. Multiquadaric RBF
Example: Design XOR Function Gaussian RBF
Thank you
