UNIVERSITY INSTITUTE OF COMPUTING

MASTER OF COMPUTER APPLICATIONS

DEEP LEARNING
23CAH-721

UNIT-1 DISCOVER . LEARN . EMPOWER

1
 • Introduction:
Neural Networks Characteristics, Challenges
motivating deep learning, artificial neural net
terminology, model of neuron, application
areas, back propagation
DEEP LEARNING .

Course Outcome
CO Title Level
Number
CO1 Identify the deep learning algorithms which Remember
are more appropriate for various types of
tasks

CO2 Categorize various types of learning tasks in Understand

various domains.
Summarize learning processes: supervised
and unsupervised, deterministic and
CO3 statistical knowledge of deep learners, and Understand
ensemble learning

2
 OUTLINE
CO2: Categorize various types of learning tasks in various domains.

TOPIC TO BE COVERED:
 model of neuron

3
 Introduction

In machine learning, particularly in the field of artificial neural networks (ANNs), neurons
are the fundamental units that process and transmit information. The concept of artificial
neurons is inspired by the biological neurons in the human brain. This document will delve
into the model of a neuron in machine learning, provide an example, and include diagrams
to illustrate the concepts.

4
 Artificial Neuron Model
• Components of an Artificial Neuron
• Inputs (x1,x2,…,xnx_1, x_2, \ldots, x_nx1​,x2​,…,xn​): Inputs are the signals or data that the neuron
receives. These could be features from a dataset in a machine learning context.
• Weights (w1,w2,…,wnw_1, w_2, \ldots, w_nw1​,w2​,…,wn​): Each input is associated with a weight,
which determines the importance of the input. Weights are learned during the training process.
• Bias (bbb): The bias is an additional parameter in the neuron that helps the model to fit the data better. It
allows the activation function to be shifted to the left or right, which can be critical for learning complex
patterns.
• Summation (zzz): The neuron computes a weighted sum of the inputs and adds the bias:
• z=∑i=1nwixi+bz = \sum_{i=1}^n w_i x_i + bz=i=1∑n​wi​xi​+b
• Activation Function (ϕ\phiϕ): The activation function determines whether the neuron should be
activated or not. It introduces non-linearity into the model, which allows the network to learn complex
patterns. Common activation functions include the sigmoid function, hyperbolic tangent (tanh), and the
rectified linear unit (ReLU).
• Output (yyy): The output of the neuron is the result of the activation function applied to the summation:
• y=ϕ(z)y = \phi(z)y=ϕ(z)

5
 Example of an Artificial Neuron
• Let's consider a simple example with two inputs and a sigmoid activation function.
• Step-by-Step Calculation
• Inputs: x1=0.5x_1 = 0.5x1​=0.5, x2=−0.2x_2 = -0.2x2​=−0.2
• Weights: w1=0.3w_1 = 0.3w1​=0.3, w2=0.8w_2 = 0.8w2​=0.8
• Bias: b=−0.1b = -0.1b=−0.1
• Step 1: Compute the Weighted Sum
• z=(0.5×0.3)+(−0.2×0.8)+(−0.1)=0.15−0.16−0.1=−0.11z = (0.5 \times 0.3) + (-0.2 \times
0.8) + (-0.1) = 0.15 - 0.16 - 0.1 = -0.11z=(0.5×0.3)+(−0.2×0.8)+
(−0.1)=0.15−0.16−0.1=−0.11Step 2: Apply the Activation Function
• Using the sigmoid activation function:
• ϕ(z)=11+e−z=11+e0.11≈0.472\phi(z) = \frac{1}{1 + e^{-z}} = \frac{1}{1 + e^{0.11}} \
approx 0.472ϕ(z)=1+e−z1​=1+e0.111​≈0.472Output
• The output of the neuron is approximately 0.472.
6
 Activation Functions
• Sigmoid Function
• The sigmoid function maps the input to a value between 0 and 1, making it useful for
binary classification problems.

7
 Hyperbolic Tangent (tanh)

The tanh function maps the input to a value between -1 and 1. It is similar to the sigmoid
function but is zero-centered.

8
 Rectified Linear Unit (ReLU)
• ReLU is the most commonly used activation function in deep learning. It is defined as:
• ϕ(z)=max⁡(0,z)\phi(z) = \max(0, z)ϕ(z)=max(0,z)

9
 Training an Artificial Neuron
• Loss Function
• The loss function measures the difference between the actual output and the predicted
output. Common loss functions include Mean Squared Error (MSE) for regression
problems and Cross-Entropy Loss for classification problems.
• Optimization
• Optimization algorithms adjust the weights and bias to minimize the loss function.
Gradient Descent is a popular optimization algorithm that updates the parameters in the
opposite direction of the gradient of the loss function.
• Backpropagation
• Backpropagation is the process of calculating the gradient of the loss function with
respect to each weight by the chain rule, and it propagates the error backward through the
network.

10
 Applications of Artificial Neurons
• Single-Layer Perceptron
• A single-layer perceptron can solve linearly separable problems. It consists of a single layer of
neurons, each connected to all the inputs.
• Multi-Layer Perceptron (MLP)
• An MLP, also known as a deep neural network, consists of multiple layers of neurons, allowing it
to solve complex, non-linearly separable problems. Each layer's output serves as the input to the
next layer.
• Convolutional Neural Networks (CNNs)
• CNNs are designed for processing structured grid data like images. They use convolutional layers
that apply filters to detect patterns and features in the data.
• Recurrent Neural Networks (RNNs)
• RNNs are suitable for sequential data, such as time series or natural language. They have
connections that form directed cycles, allowing them to maintain a memory of previous inputs.

11
 References
• Textbooks / Reference Books
• TEXT BOOKS
• T1. Goodfellow, I., Bengio,Y., and Courville, A., Deep Learning, MIT Press, 2016..
• T2. Bishop, C. ,M., Pattern Recognition and Machine Learning, Springer, 2006.
• T3. Deng & Yu, Deep Learning: Methods and Applications, Now Publishers, 2013. 3. Ian
Goodfellow,
• T4. YoshuaBengio, Aaron Courville, Deep Learning, MIT Press, 2016.
• REFERENCE BOOKS
• R1. Yegnanarayana, B., Artificial Neural Networks PHI Learning Pvt. Ltd, 2009.
• R2. Golub, G.,H., and Van Loan,C.,F., Matrix Computations, JHU Press,2013.
• R3. Satish Kumar, Neural Networks: A Classroom Approach, Tata McGraw-Hill Education, 2004.

12
THANK YOU

13