Deep Learning –An Introduction
Dr. Anand Kumar M
Assistant Professor-I,
Department of Information Technology
National Institute of Technology Karnataka
Surathkal
Outline & Content
• AI/ML/DL
• Machine Leanring
• Deep learning?
• Why Deep Learning
• Applications
• Conclusion
ML vs DL
Deep Learning at Google
Related Fields
data
mining control theory
statistics
decision theory
information theory machine
learning
cognitive science
databases
psychological models
evolutionary neuroscience
models
Machine learning is primarily concerned with the accuracy
and effectiveness of the computer system.
What is Machine Learning?
• It is very hard to write programs that solve problems like
recognizing a face.
– We don’t know what program to write because we don’t know
how our brain does it.
– Even if we had a good idea about how to do it, the program
might be awfully complicated.
• Instead of writing a program by hand, we collect lots of examples
that specify the correct output for a given input.
• A machine learning algorithm then takes these examples and
produces a program that does the job.
– The program produced by the learning algorithm may look
very different from a typical hand-written program. It may
contain millions of numbers.
– If we do it right, the program works for new cases as well as
the ones we trained it on.
Why now?
• Flood of available data (especially with the
advent of the Internet)
• Increasing computational power
• Growing progress in available algorithms and
theory developed by researchers
• Increasing support from industries
Traditional Programming
Data
Computer Output
Program
Machine Learning
Data
Computer Program
Output
Three components of machine learning
Data
• Want to detect spam? Get samples of spam
messages. Want to forecast stocks? Find the
price history. Want to find out user
preferences?
Machine learning structure
• Supervised learning
Google trends
2020/5/25 Dr.Anand Kumar M (NITK)
2020/5/25 Dr.Anand Kumar M (NITK)
So, 1. what exactly is deep learning ?
And, 2. why is it generally better than other methods on
image, speech and certain other types of data?
2020/5/25 Dr.Anand Kumar M (NITK)
So, 1. what exactly is deep learning ?
And, 2. why is it generally better than other methods on
image, speech and certain other types of data?
The short answers
1. ‘Deep Learning’ means using a neural network
with several layers of nodes between input and output
2. the series of layers between input & output do
feature identification and processing in a series of stages,
just as our brains seem to.
2020/5/25 Dr.Anand Kumar M (NITK)
hmmm… OK, but:
3. multilayer neural networks have been around for
25 years. What’s actually new?
2020/5/25 Dr.Anand Kumar M (NITK)
hmmm… OK, but:
3. multilayer neural networks have been around for
25 years. What’s actually new?
we have always had good algorithms for learning the
weights in networks with 1 hidden layer
but these algorithms are not good at learning the weights for
networks with more hidden layers
what’s new is: algorithms for training many-later networks
2020/5/25 Dr.Anand Kumar M (NITK)
Topologies of Neural Networks
completely
connected recurrent
feedforward
(directed, a-cyclic) (feedback connections)
Handwriting Digit Recognition
Input Output
x1 y1
0.1 is 1
x2 y2
0.7 is 2
The image
is “2”
……
……
……
x256 y10
0.2 is 0
16 x 16 = 256
Ink → 1 Each dimension represents
No ink → 0 the confidence of a digit.
Example Application
• Handwriting Digit Recognition
x1 y1
x2
y2
Machine “2
……
……
”
x256 256 10 y10
𝑓 :𝑅 →𝑅
In deep learning, the function is
represented by neural network
Element of Neural Network
Neuron 𝐾
𝑓 :𝑅 →𝑅
a1 w1 z a1w1 a2 w2 aK wK b
a2 w2
z z
a
wK
…
aK Activation
weights function
b
bias
Neural Network
neuron
Input Layer 1 Layer 2 Layer L Output
x1 …… y1
x2 …… y2
……
……
……
……
……
xN …… yM
Input Output
Layer Hidden Layers Layer
Deep means many hidden layers
Sample Applications
• Web search
• Computational biology
• Finance
• E-commerce
• Space exploration
• Robotics
• Information extraction
• Social networks
• Debugging
• [Your favorite area]
Applications (conti..)
• Spam Email Detection
• Machine Translation (Language Translation)
• Image Search (Similarity)
• Clustering (KMeans) : Amazon
• Recommendations
• Classification : Google News
• Text Summarization - Google News
• Rating a Review/Comment: Yelp
• Fraud detection : Credit card Providers
• Decision Making : e.g. Bank/Insurance sector
• Sentiment Analysis
• Speech Understanding – iPhone with Siri
• Face Detection – Facebook’s Photo tagging
Image Translation
Applications
• Deep Learning AI is revolutionizing the filmmaking
process as cameras learn to study human body
language to imbibe in virtual characters.
• A deep learning model tends to associate the
video frames with a database of pre-recorded
sounds to select appropriate sounds for the scene
• https://youtu.be/0FW99AQmMc8
• http://news.mit.edu/2016/artificial-intelligence-p
roduces-realistic-sounds-0613
Bio-NLP
• Open Problems
https://towardsdatascience.com/summarising-the-latest-research-on-
coronavirus-with-nlp-and-topic-modelling-28b867ad9860
Covid NLP datasets
Allen Institute for AI Open Research Dataset (CORD-19), over 47,000
scholarly articles, including over 36,000 with full text, about COVID-19 and
the coronavirus family of viruses for use by the global research community.
• http://www.r2d3.us/visual-intro-to-machine-l
earning-part-1/
• https://www.mygreatlearning.com/blog/
deep-learning-applications/
• http://www.r2d3.us/visual-intro-to-machine-l
earning-part-2/
• http://www.hpc.lsu.edu/training/weekly-
materials/2016-Fall/
machine_learning_qb2_fall_2016.pdf
• https://vas3k.com/blog/machine_learning/
