Digital : An Ingression into Deep Learning_FP
In ____________, information must flow from input to output only in one direction.
Feed-Forward artificial
Hiden
H2O.ai offers built-in integration tools for which platform?
All
tool supports the use of GPU.
turi
Machine Learning is inspired by the structure of the brain.
false
Which of the following is a type of Recurrent Neural Network?
All
Autoencoders are trained without supervision.
true
As the amount of data increases, the performance of Machine Learning algorithm
deIncrease
The following are tools offered by deepnet platforms, except
H2o
Every neuron in the Input Layer represents a/an _______________ variable that influences the
output.
Indepent
There is no exact formula for calculating the number of hidden layers, as well as the number of
neurons in each hidden layer.
True
introduces the concept of memory cell which can process data within time lags.
Autoend
____________ is a technique which helps machines to mimic human behavior.
Deep learning
Machine Learning is a subset o
Deep
Prelude
Have you ever wondered
How google translates an entire web page to a different language in a matter of seconds? Or,
how your phone gallery groups images into different folders based on their locations?
All these are innovations of Deep Learning! But, what exactly is Deep Learning? Explore to
know more!
Deep Learning
The Big Picture!
Deep Learning is a subset of Machine Learning, which in turn, is a subset of Artificial
Intelligence.
1. Artificial Intelligence
Artificial Intelligence (AI) is a technique that helps machines to mimic human behavior.
2. Machine Learning
Machine Learning is an application of AI that allows the system to learn and improve from
experience automatically.
3. Deep Learning
Deep Learning is a type of Machine Learning that is inspired by the structure of the brain. It
is also known as Artificial Neural Network (ANN). It uses complex algorithms and deep
neural networks to train models.
What is Deep Learning?
Definition
Deep Learning involves networks which are capable of learning from data and functions
similar to the human brain.
Why Deep Learning?
Let us explore the reasons which make Deep Learning shine in the industry.
Processes massive amount of data
Deep Learning can process an enormous amount of both Structured and Unstructured
data.
Performs Complex Operations
Deep Learning algorithms are capable enough to perform complex operations when
compared to the Machine Learning algorithms.
Why Deep Learning?
Achieves Best Performance
o As the amount of data increases, the performance of Machine Learning
algorithms decreases.
o On the other hand, Deep Learning maintains the performance of the model.
Feature Extraction
o Machine Learning algorithms extract patterns from labeled sample data,
while Deep Learning algorithms take large volumes of data as input, analyze
them to extract the features on its own.
Deep Learning vs. Machine Learning
Say, for instance, we develop a machine that differentiates between cherries and tomatoes.
Machine Learning
If done through Machine Learning, we need to specify the features based on which the two
can be differentiated like size and stem, in this case.
Deep Learning
In Deep Learning, the features are picked by the Neural Network without any human
intervention. But, that kind of independence can be achieved by a higher volume of data in
training the machine.
A Curtain Raiser
Watch this video to know more about Deep Learning
The human brain contains billions of cells called Neurons. The structure of a neuron
is depicted in the above image.
Neural Networks is a set of algorithms designed to learn the way our brain works.
The biological neurons inspire the structure and the functions of the neural networks.
Biological and Artificial Neurons - Terminologies
Biological Neuron Artificial Neuron
Dendrites Inputs
Nucleus Nodes
Synapse Weights
Axon Output
A Node is also called a Neuron or Perceptron.
Structure of an Artificial Neural Network
The basic structure of an Artificial Neural Network (ANN) consists of artificial neuron that
are grouped into 3 different layers namely:
There are three different layers in a neural network, namely:
1. Input Layer
2. Hidden Layer
3. Output Layer
Input Layer
The input layer communicates with the external environment and presents a
pattern to the neural network.
Every neuron in the input layer represents an independent variable that influences
the output.
Structure of an Artificial Neural Network
Hidden Layer
A Neural Network consists of several hidden layers, each consisting of a collection
of neurons.
The hidden layer is an intermediate layer found between the input layer and the
output layer.
This layer is responsible forextracting the features required from the input.
There is no exact formula for calculating the number of the hidden layers as well as the
number of neurons in each hidden layer.
Output Layer
The output layer of the neural network collects and transmits information in the
desired format.
Structure of an Artificial Neural Network
Every single layer is made up of one or more number of neurons. A simple model of an
Artificial neuron is shown in the figure.
Parameters represented are:
1. X1,X2,...,XmX_1,X_2,...,X_mX1,X2,...,Xm are the input signals.
2. W1,W2,...WmW_1,W_2,...W_mW1,W2,...Wm are the weights.
3. ∑\sum∑ is the Summation function
4. σ\sigmaσ is the Activation function.
5. bkb_kbk represents the bias value
Artificial Neural Network - Process Flow
The process flow depicted in the image is explained as follows:
A node is where the computations happen
A node combines the Input data X1,X2,...,XmX_1,X_2,...,X_mX1,X2,...,Xm with a set of
Weights W1,W2,...,WmW_1,W_2,...,W_mW1,W2,...,Wm which in turn aids in amplifying
or diminishing the input with respect to the polarity attached to it.
On multiplying the inputs with weights, a specific significance is assigned to the inputs with
regard to the task the algorithm is learning. It helps in answering the question Which
input is the most helpful in learning data without error?
The input-weight products are summed up using the Summing function, and the output
yielded is passed to the Activation function to decide whether and to what extent the
signal should progress to have an effect in the final output.
Summation and Activation Functions
Summation Function
Various inputs are multiplied with their respective connection weights and summed up
together with the bias value.
∑j=1NXjWj+bk\sum_{j=1}^{N}X_jW_j+b_k∑j=1NXjWj+bk
where:
XjX_jXj represents the inputs
WjW_jWj represents the weights
Summation and Activation Functions
Activation Function
Activation Function aids in deriving the output.
It is also known as the Transfer Function.
It maps the resulting values between 0 to 1 or -1 to 1.
There are two types of Activation function, namely:
1. Linear Activation Function
2. Non-Linear Activation Function
Non-Linear Activation Function is the most commonly used Activation function
in Neural Networks.
Non-Linear Activation Functions
Watch this video to know some of the most commonly used Activation Functions in Neural
Networks.
The default Activation Function is ReLU.
Working of a Neural Network
Watch this video to know the working of a Neural Network.
In January 2018, Google's CEO Sundar Pichai claimed that Artificial intelligence (AI) would be more
transformative to humanity than electricity.
Prelude
Curious about how weight and bias values are decided? This section will help you in
understanding the concepts of the learning process in Neural Network.
Learning Process of a Neural Network
The learning process of a Neural Network includes updating the network architecture and
connecting the weights for the network to perform efficiently.
Designing a Learning Process involves the following:
1. Learning Paradigm
Having a model from the environment in which Neural Network works.
2. Learning Rules
Figuring out the rules that aid in updating the weights.
3. Learning Algorithms
Identifying the procedure to update the weights according to the learning rules.
Learning Paradigm
The following are the various Learning Paradigms in Neural Networks:
Supervised
Unsupervised
Reinforcement
Learning Rules
The four basic types of Learning Rules in Neural Network are:
Error Correction Rules
Hebbian
Boltzmann
Competitive Learning
Learning Algorithms
Following are a few Deep Learning algorithms:
Adaptive Resonance Theory
Kochen Self Organization Map
ADALINE
Perceptron
Backpropagation
DRC-Hubo
View the following video on DRC-Hubo developed by Team KAIST from South Korea which
impresses us by remarkably precise and deliberate motion.
Principles
Principles
A Neural Network works based on two principles, namely:
1. Forward Propagation
2. Backward Propagation
Forward Propagation in Neural Network
Explore this video to know about the Forward Propagation.
Backward Propagation in Neural Network
Explore this video to know about the Backward Propagation.
Prelude
In this section, let us discover the Network Architecture that prevails in Neural Network.
Neural Network Architecture
The architecture of a Neural Network can be broadly classified into two, namely:
1. Feed Forward Artificial Neural Network
2. Recurrent Neural Network
Feed Forward Artificial Neural Network
The information must flow from input to output only in one direction.
No Feedback loops must be present.
A few Feed Forward Artificial Neural Networks are:
Single-layer Feed Forward Network
Multi-layer Feed Forward Network
Recurrent Neural Network
Information can be transmitted in both directions.
Feedback loops are allowed.
A few Recurrent Neural Networks are:
Hopfield Network
Elman Network
Jordan Network
Explore More!
The following video will help you know more about the Feed Forward Networks and
Recurrent Neural Networks in mathematical terms.
MELTANT-α\alphaα is an avatar robot with the world’s most powerful and flexible robotic hand
which directly imitates human hand movements.
Prelude
Neural Networks are employed in various fields. Following are a few types of Neural
Networks that we will explore in this section:
Radial Basis Function
Long/Short Term Memory
Gated Recurrent Unit
Autoencoder
Convolutional Neural Networks
Radial Basis Function
Radial Basis Function is similar to Feed Forward Network but uses Logistic function as the
Activation Function.
Check out this video to know more!
Long/Short Term Memory
Long/Short Term Memory (LSTM) networks are used to classify, process, and make
predictions based on time series data.
LSTM networks can predict the action in a specific video frame by keeping in mind the
action that occurred in the earlier frames.
The applications of LSTM include writing, speech recognition, and so on.
Gated Recurrent Unit
Gated Recurrent Unit (GRU) is LSTM with a forget gate.
It is used in sound, speech synthesis, and so on.
More on LSTM and GRU
Watch the following video to know more about LSTM and GRU Neural Networks.
Autoencoder
Autoencoder (AE) is used for classification, clustering, and feature compression.
It is trained without supervision.
Convolutional Neural Networks
The following video will help you understand the basic concepts of Convolutional Neural
Networks (CNN).
Applications
Image Classification
Image classification is one of the common applications of deep learning.
A convolutional neural network can be used to recognize images and label them
automatically.
Object Recognition
Deepnets (an optimized version of Deep Neural Networks) can be trained to recognize different
objects within the same image.
Video Recognition
Deepnets can also be trained for real-time object recognition with respect to time
and space.
Some of the famous use cases are driverless cars, robots, and theft detection.
Sentiment Analysis
Using sentiment analysis, the underlying intent of the text can be extracted.
With social media channels, it is possible to automate and measure the feelings of the
public on a given news story, topic, brand, or product.
Positive sentiment can be identified, thereby allowing the marketing of a product, or
understanding which elements of a business strategy are working.
Medical Applications
Deepnets can be trained to detect cancerous cells, benign and malignant tumors from
MRI and CT scans.
They are also applied in drug discovery by training nets with molecular structure and
chemical compositions.
Tools
Deepnet Platform
A deepnet platform is a service that allows you to incorporate deepnet in your
applications without building one from scratch.
This platform provides a set of tools and interfaces to create a custom deepnet.
Deepnet platforms are of two types, namely:
Software platform: This platform is available as downloadable packages that need to be
deployed on your hardware.
Full platform: It is available as an online interactive UI to build and deploy models
without any coding experience.
More on Deepnet Platform
The following video explains what exactly is a deepnet platform.
Deepnet Platforms - Tools
The following are the tools offered by the deepnet platforms:
Deepnet capability
Data Munging
UI/Model Management
Infrastructure
H2O.ai
H2O.ai is an open-source machine learning platform.
Along with many machine learning algorithms, this platform currently provides a
deepnet capability known as Multilayer Perceptron (MLP).
Features
H2O.aiH_2O.aiH2O.ai supports MLP for a deep learning model, which uses the L-
BFGS algorithm for backpropagation.
H2O.aiH_2O.aiH2O.ai offers built-in integration tools for platforms like HDFS,
Amazon S3, SQL, and NoSQL.
Its intuitive UI can be accessed by programming environments like R, Python, and
JSON.
You can model and analyze data with Tableau, Microsoft Excel, and RStudio.
Since the package needs to be downloaded and deployed in the hardware,
H2O.aiH_2O.aiH2O.ai comes with an in-memory map-reduce, and columnar
compression to accelerate the training of models.
Turi - A Graph-based Tool
Turi is a software platform that provides two deepnet capabilities (convolutional network
and MLP) along with machine learning and graph algorithms.
Features
Turi provides built-in support for integration of Amazon S3, SQL DB, HDFS, Spark RDD, and
Pandas data frames.
It offers a UI interface for model management and also includes a visualization tool called
GraphLab Canvas for visualizing the model results.
To work with large data, Turi comes with built-in storage support like SFrame, SArray, and
SGraph.
Unlike H2O.aiH_2O.aiH2O.ai, Turi supports the use of GPUs.
Turi
Watch this video to know more about the Turi platform.
Course Summary
While we've seen great strides in robotic capability recently, there's still a vast gap between
human and robot motor skills. Even at simple motor skills like gripping, machine still have a
very long way to go to suit human ability.
By linking learning with continuous feedback and control, however, we may begin to bridge
that gap, enabling robots to manage the complexities of the real world intelligently and
reliably.
Replicating the human brain at entirety will not just be a Science fiction episode anymore.
The future indeed will be full of surprises. Keep learning!
