Deep Learning with Keras and TensorFlow

Introduction to Deep Learning

 Learning Objectives

By the end of this lesson, you will be able to:

Explore the factors contributing to the achievements of Deep

Learning in the past decade

Understand the applications and challenges of Deep Learning

Evaluate various frameworks that facilitate the development of

deep learning models, including their features and uses

Engage in the end-to-end process of a deep learning project, from

planning and data collection to training, evaluation, deployment,
and monitoring
 Business Scenario

A retail company wants to enhance its customer experience by implementing

Deep Learning techniques. They plan to analyze customer data, including
purchase history, browsing behavior, and demographic information, to gain
insights and make personalized product recommendations. By using Deep
Learning algorithms, they aim to identify complex patterns and correlations in
the data to understand customer preferences better.

Approach:
To accomplish this, the company collects and preprocesses a large customer
dataset, utilizes significant computational resources for deep neural network
training, and deploys the trained model to provide real-time recommendations.
This leads to increased customer satisfaction, higher sales, and improved
marketing effectiveness.
Brief History of AI
 Artificial Intelligence

Artificial Intelligence (AI) is the field of study focused on creating machines capable of performing tasks
that would typically require human intelligence

Artificial intelligence

Machine learning

Deep learning

Deep Learning is a subset of Machine Learning, which in itself is a subset of Artificial Intelligence, each
representing a narrowing specialization within the field.
 Evolution of AI

1940
to
1950

Warren McCulloch and Walter Pits proposed a

model of artificial neurons in 1943.

Alan Turing proposed a test, the Turing Test, to

check a machine's ability to exhibit intelligent
behavior equivalent to, or indistinguishable
from that of a human.
 Evolution of AI

1940
to 1950s
1950

The word Artificial Intelligence was coined.

Perceptron was invented in 1957.

High-level programming languages like COBOL,

FORTRAN, and LISP were invented.
 Evolution of AI

1940
to 1950s 1960s
1950

Researchers solved mathematical

problems using statistical methods.

Joseph Weizenbaum invented the first chatbot,

ELIZA, in 1966.
 Evolution of AI

1940 1970
to 1950s 1960s to
1950 1980

Interest in AI research dropped significantly.

Funding for AI projects from the government

was insufficient.
 Evolution of AI

1940 1970 1980

to 1950s 1960s to to
1950 1980 1987

AI was revived as expert systems replicating human

decision-making abilities.

Stanford University hosted the inaugural national

conference of the American Association of Artificial
Intelligence (AAAI) in 1980.

There was a shortage in funding for AI research and

development after 1987.
 Evolution of AI

1940 1970 1980 1997

to 1950s 1960s to to to
1950 1980 1987 2006

1997: IBM Deep Blue beat world chess

champion Gary Kasparov in 1997.

Companies like Facebook, Twitter, and Netflix

also started using AI from 2006 onwards.
 Evolution of AI

1940 1970 1980 1997 2012

to 1950s 1960s to to to onward
1950 1980 1987 2006 s

Considered to be a golden period, Deep

Learning emerged as a giant in 2012.

Computer vision and natural language

processing benefitted the most.
Motivation for Deep Learning
 Deep Learning

The following are some of the key developments in the field of Deep Learning:

Perceptron

01

Recurrent
neural 05
networks
02 Back
propagation

Convolutional
04
neural networks
03
Multi-layer
perceptron
 Deep Learning

DL helps solve real-world problems that were once considered impossible, such as:

Natural language understanding (NLU):

Teaches computers to comprehend and
process human language

Image recognition:
Speech recognition:
Trains machines to accurately 3
1 Enables computers to transcribe and
identify and classify objects within
understand spoken words
images
 Deep Learning: Progress

The publication of the AlexNet paper in 2012 piqued global interest in Deep Learning, in which the
proposed neural network achieved state-of-the-art performance in the ImageNet challenge.

ImageNet Challenge: Classification task

Images classified into 1000 categories

• Training data: 1.2 million images

• Testing data: 1.5 million images

The runner-up used hand-crafted features and the best available classification methods available at
that time; however, the difference between the winner and the runner-up was more than 10%.
 Deep Learning: Progress

The progress in Deep Learning encountered challenges primarily due to:

Lack of funding Inadequate hardware Expensive data storage

Deep Learning
 Deep Learning

Deep Learning is a subset of machine learning, focusing on the use of deep

neural networks to process and learn complex patterns from data.

Deep Learning focuses on training models to learn and make intelligent

decisions from large datasets.
 Deep Learning

DL can effectively utilize structured and unstructured data from diverse

domains, encompassing images, text, audio, and video, to discover patterns
and make accurate predictions or classifications.

DL surpasses traditional machine learning by leveraging deep neural networks to

extract patterns from unstructured data, resulting in superior performance in domains
like computer vision, natural language processing, and speech recognition.

This technique enables Deep Learning models to extract complex features and achieve
highly accurate predictions or classifications.
Deep Learning vs. Machine Learning
 Deep Learning vs. Machine Learning

Deep Learning Machine learning

• Subset of machine learning that • A broad field of training algorithms

focuses on training deep neural to make predictions or decisions
networks based on data

• Excels in handling unstructured data • Works with structured and

such as images, audio, text, and video unstructured data

• Eliminates the need for manual • Performance depends on the

feature engineering quality and relevance of engineered
features
 Deep Learning vs. Machine Learning

Deep Learning Machine learning

• Excels in tasks like image recognition, • Utilizes techniques like decision

natural language processing, and trees, support vector machines,
speech synthesis and random forests

• Requires substantial computational • Can be effective in a wide range of

resources and large labeled datasets applications

• Utilizes deep neural networks with • Utilizes neural networks with

multiple layers and requires limited layers and requires fewer
substantial computational resources computational resources
 Deep Learning vs. Machine Learning

Deep Learning Machine learning

• To train DL models effectively, high- • Most problems like Data

quality GPUs with ample RAM are preprocessing, running simple ML
crucial. models can be executed using a
single powerful CPU.

In conclusion, Deep Learning is designed for handling large datasets and performing extensive
computations and is generally considered more expensive than machine learning.
Deep Learning: Successes in the Last Decade
 Deep Learning: Success in the Last Decade

In the past decade, Deep Learning has experienced exponential growth in the following areas:

Reinforcement learning Computer vision

Audio processing
 AlexNet (2012)

AlexNet emerged as the champion in the renowned annual image classification

competition known as ImageNet.

Since then, any groundbreaking event in Deep Learning has been referred to as an ImageNet moment.
 Word Embeddings (2013)

Word embeddings are dense representations of text data.

It was introduced by Tomas Mikolov as an

efficient way to train word embeddings.

word2vec is the model used for word

embedding.

This model has been widely utilized for text data processing purposes.

Source: https://deepai.org/profile/tomas-mikolov
 Sequence-to-Sequence Models (2014)

It comprises two components, both constructed using neural networks,

making it a novel architecture in the field.

Please Come Here

Y1 Y2 Y3

Decoder Decoder Decoder

RNN RNN RNN
Encoder: Encodes an input
C
sequence into a vector
<SOS>

H0 H1 H2
Encoder Encoder Encoder Decoder: Decodes the vector
RNN RNN RNN
into a different sequence
X1 X2 X3
Komm Bitte Her

This framework made complex tasks like machine translation and language generation easier and
achieved top-notch performance.
 Sequence-to-Sequence Models (2014)

It comprises two components, both constructed using neural networks,

making it a novel architecture in the field.

Encoder: Encodes an input

sequence into a vector

Decoder: Decodes the vector

into a different sequence

This framework made complex tasks like machine translation and language generation easier and
achieved top-notch performance.
 Generative Adversarial Neural Network (2014)

GANs are Deep Learning models with a generator and discriminator. They generate realistic synthetic
data through adversarial training.

These GANs possess the ability to generate life-like

images, introducing a new dimension to computer
vision research.

It led to the creation of new breeds of GANs such as

Cycle GAN, Style GAN, and Pix2pix.

They can be used in fashion, art, and science.

 AlphaGo Beats Human (2016)

AlphaGo, developed by DeepMind, is the first computer program to have defeated a professional
human Go player.

Go is a more complex board game than chess.

AlphaGo, a reinforcement learning model, has achieved mastery in the game of Go.
 Transformers (2017)

They employ self-attention mechanisms to capture long-range dependencies in sequences, resulting in

exceptional performance in tasks like machine translation, text generation, sentiment analysis, and
question-answering.

The paper Attention Is All You Need

introduces the transformer model and is
recommended for anyone interested in NLP.
 Deep Learning: Successes in the Last Decade

DL frameworks have supported the research community in their exploration with:

Efficient Algorithms Ease of implementation

Pretrained models Scalability

Extensive libraries Experimentation

Key Reasons to Learn Deep Learning
 Key Reasons to Learn Deep Learning

Ability to solve various problems across several industries

Widely used across different domains at scale such as:

Healthcare

Space exploration

Autonomous vehicles

Robotics

Finance
 Key Reasons to Learn Deep Learning

Access to powerful graphics processing units (GPUs) at an affordable price

In 2007, NVIDIA launched the CUDA framework, which provides an API for GPU computing.

It enabled practitioners to easily use GPUs and accelerated

the training of Deep Learning models.

Personal GPUs for Deep Learning are widely available, and

renting GPUs from cloud providers like AWS or Azure is
common.
 Key Reasons to Learn Deep Learning

GPUs play a crucial role in accelerating Deep Learning computations due to their
parallel processing capabilities.

Powerful graphics processing units

(GPUs) are accessible at an affordable
price.

A GPU performs parallel mathematical

operations.

The operations are executed by multiple

cores, which are comparatively less
powerful but function in parallel.
 Key Reasons to Learn Deep Learning

Low barrier to entry

Getting started with Deep Learning is easier with prior knowledge of:

Basic Python programming Data science and statistics Basic linear algebra

Affordable hardware, cheap cloud services, and user-friendly frameworks like TensorFlow and PyTorch
make training Deep Learning models more accessible.
 Key Reasons to Learn Deep Learning

Requirement of trained Deep Learning experts and practitioners

Companies across industries have recognized

the applicability of Deep Learning in areas
such as technology, healthcare, finance, retail,
manufacturing, and transportation.

A shortage of trained professionals in the industry has created a golden

opportunity to get into this field.
Applications of Deep Learning
 Applications of Deep Learning

DL has diverse applications in various fields.

AI leverages deep learning within Autonomous self-driving cars use

Natural Language Processing (NLP) to computer vision for object detection
enhance human language interaction and classification

Deep Learning has made its mark in

Automatic colorization of black-and- audio processing. Transforming speech
white images. to text with high accuracy
Limitations of Deep Learning
 Limitations of Deep Learning

To learn a particular task, a significant amount of data is required.

Huge data sets need to be collected, prepared,

and labeled according to the intended task.
 Limitations of Deep Learning

To learn a particular task, a significant amount of data is required.

Example
To train such a model:

Collect many images of cats and dogs

Consider using a deep Label the images appropriately

learning model to
differentiate cats from
dogs Feed the images to the model

Executing this task requires a lot of effort and time.

 Limitations of Deep Learning

Expensive hardware is necessary to support the computational

requirements.

High-speed
GPUs
RAM

Deep Learning models are hardware-intensive.

 Limitations of Deep Learning

DL models are not immune to overfitting and can be particularly susceptible

to it

Unavailability of sufficient training data results in overfitting.

The model aligns with a minimal set of data

points and fails to perform well on unseen
datasets.
 Limitations of Deep Learning

DL is more difficult to explain.

Inputs Black box Outputs

The models function like black boxes, with known inputs and outputs.

Deep Learning is not ideal for rigorous verification due to limited interpretability and reliability.
Deep Learning Frameworks
 Deep Learning Frameworks

DL frameworks provide interfaces, libraries, and tools to facilitate the development of Deep Learning
models. Additionally, they offer the flexibility to access and modify the underlying algorithms if needed.

The frameworks offer tried and tested foundations for:

Designing

Training

Debugging

Deploying

The frameworks provide simple ways to define models using ready-made and optimized functions.
 Deep Learning Frameworks

The three popular Deep Learning frameworks are:

 Keras

Keras is an open-source Python framework for building and training deep neural networks with a user-
friendly and modular interface.

It provides a Python interface for developing

artificial neural networks.

It acts as an interface for the TensorFlow library.

It is qualified as an easy-to-use, simplistic interface.

 TensorFlow

It is an end-to-end, open-source platform for ML and DL.

It was developed by Google and

coded using Python.

It is specifically optimized for the training and inference of neural networks.

 PyTorch

Meta AI developed and maintains PyTorch, an open-source Deep Learning framework based on Torch
library. It is a capable deep-learning Python library that offers tough competition to TensorFlow.

Two main features of PyTorch are:

Tensor computing libraries like NumPy offer

powerful GPU acceleration capabilities.

A tape-based automatic differentiation system

computes the numerical derivative of a function
defined by a computer program.
Lifecycle of a Deep Learning Project
 Deep Learning Lifecycle

It is a cyclical process consisting of the following four main phases:

 Planning Phase

A DL project starts with the planning phase.

During this phase:

Viability of the project is determined

Goals are set

Resources for the projects are planned

 Data Collection and Labeling Phase

After the planning phase, the data collection and labeling phase gets initiated.

This phase involves:

Deciding how to label the collected

Setting up data capturing devices
data

If collecting or labeling data proves to be challenging, the project reverts to the planning phase to
devise a more effective approach.
 Training Phase

Once the collected data gets labeled, the project then moves to the training phase.

Forward Loss
propagation calculation Backpropagation Iterative updates

During Deep Learning training, the model learns from data by iteratively adjusting its parameters
through forward and backward propagation. This minimizes errors and enhances predictions.
 Training Phase

The chosen models are implemented and debugged.

The model undergoes

The model's performance continuous improvement
gets tested until all the requirements
are fulfilled.
 Training Phase

Projects may revisit the data collection phase for the following reasons:

Insufficient data leading to overfitting

Improper labeling causing inaccurate or unreliable

outcomes

Once the model achieves satisfactory results, it is deployed for production.

 Testing Phase

In this phase, evaluate the trained model on unseen data, and analyze the results for improvement.

Tests are conducted on the model to compare the results with the planned success metrics and
decide on whether to deploy the model or not.

In this phase, every process is recorded and versioned to maintain quality and reproducibility to
ensure that its performance can be easily fixed if it degrades during production

The goal is to make the necessary adjustments or fixes to restore the model's performance to the
desired level.
 Deploying Phase

In this phase, the model is deployed into production for real-time use.

The end goal of this phase is the proper functionality of the model after deployment.
 Model Monitoring and Maintenance

After deploying the model to production, continuously check the model’s performance as it
interacts with real-world data

Ensure ongoing model efficacy by being ready to retrain with new datasets or adjust the
model parameters should there be a decline in performance
 Key Takeaways

Deep Learning (DL) is a specialized form of ML that uses artificial

neural networks to solve complex problems.

DL is well-suited for handling unstructured data due to its ability to

extract complex features and handle greater complexity compared
to machine learning.

Deep Learning models require large amounts of data to learn

different patterns and are more computationally intensive.

A DL project life cycle consists of four main phases: planning, data

collection and labeling, training, and testing and deploying.
Knowledge Check
Knowledge
Check
What is an artificial neural network?
1

A. It is a computing system made up of several simple, highly interconnected processing

elements called neurons.

B. It is a system that uses machine learning to solve complex problems.

C. It is a programming language used to write Deep Learning algorithms.

D. It is a system that uses artificial intelligence to solve complex problems.

 Knowledge
Check
What is an artificial neural network?
1

A. It is a computing system made up of several simple, highly interconnected processing

elements called neurons.

B. It is a system that uses machine learning to solve complex problems.

C. It is a programming language used to write Deep Learning algorithms.

D. It is a system that uses artificial intelligence to solve complex problems.

The correct answer is A

It is a computing system made up of several simple, highly interconnected processing elements called
neurons.
Knowledge
Check
What kind of hardware is required to train Deep Learning models?
2

A. Low-end CPU with minimal RAM

B. High-end CPU with sufficient RAM

C. Good quality GPU with sufficient RAM

D. Specialized hardware like ASICs or TPUs

 Knowledge
Check
What kind of hardware is required to train Deep Learning models?
2

A. Low-end CPU with minimal RAM

B. High-end CPU with sufficient RAM

C. Good quality GPU with sufficient RAM

D. Specialized hardware like ASICs or TPUs

The correct answer is C

A good quality GPU with sufficient RAM is required to train Deep Learning models as it is
computationally intensive and works on a large amount of data to learn different patterns.
Knowledge
Check
What are some popular Deep Learning frameworks?
3

A. TensorFlow, Keras, and PyTorch

B. MATLAB, R, and Octave

C. Spark, Hive, and Hadoop

D. None of the above

 Knowledge
Check
What are some popular Deep Learning frameworks?
3

A. TensorFlow, Keras, and PyTorch

B. MATLAB, R, and Octave

C. Spark, Hive, and Hadoop

D. None of the above

The correct answer is A

TensorFlow, Keras, and PyTorch are the popular Deep Learning framework.
Thank You