Computer vision is a field of computer science that focuses on

enabling computers to identify and understand objects and people

in images and videos. Like other types of AI, computer vision
seeks to perform and automate tasks that replicate human
capabilities.

Giving capabilities like humans to machines to see and visualize

images.
Computer Vision, abbreviated as CV, is a domain of AI that depicts the capability of a machine to
get and analyse visual information and afterwards predict some decisions about it. The entire
process involves image acquiring, screening, analysing, identifying and extracting information. This
extensive processing helps computers to understand any visual content and act on it acc
 Look for information on the
basis of image procesing Face Mapping

Autonomous Vehicles

Filters in
Snapchat
Medical Imaging In shopping malls
Pixelwise classification
A digital image is made up of
multiple arrays of pixel
A matrix of pixel values
The number of pixels in an image is
Resolution
 E AND B
unique features of object
Identify Image features
https://www.w3schools.com/colors/colors_rgb.asp

https://setosa.io/ev/image-kernels/
Evaluation
 Why do we need Evaluation?
Once a model has been created and
trained, it must undergo appropriate
testing in order to determine the model’s
effectiveness and performance.
Thus, the model is evaluated using Testing
Data (which was taken from the dataset
generated at the Data Acquisition stage).
 What is evaluation?
Evaluation is the process of understanding
the reliability of any AI model, based on
outputs by feeding test dataset into the
model and comparing with actual
answers.
There can be different Evaluation
techniques, depending of the type and
purpose of the model.
 Model Evaluation
Consider developing an AI-based prediction
model and deploying it in a forest that is
prone to forest fires. The model’s current goal
is to make predictions about whether or not a
forest fire has started. We need to consider
the two circumstances of prediction and
reality. The reality is the actual situation in the
forest at the time of the prediction, while the
prediction is the machine’s output.
Here, we can see
in the picture that
a forest fire has
broken out in the
forest. The model
predicts a Yes
which means
there is a forest
fire. The
Prediction
matches with the
Reality. Hence,
this condition is
termed as

True Positive.
Here there is
no fire in the
forest hence
the reality is
No. In this
case, the
machine too
has predicted
it correctly as
a No.
Therefore, this
condition is
termed as
True
Negative.
Here the reality is
that there is no
forest fire. But the
machine has
incorrectly
predicted that
there is a forest
fire. This case is
termed as
False Positive.
Here, a forest fire
has broken out in
the forest
because of
which the Reality
is Yes but the
machine has
incorrectly
predicted it as a
No which means
the machine
predicts that
there is no Forest
Fire.
Therefore, this
case becomes
False Negative.
Confusion matrix
The result of comparison between the
prediction and reality can be
recorded in what we call the
confusion matrix. The confusion matrix
allows us to understand the prediction
results. Note that it is not an
evaluation metric but a record which
can help in evaluation.
The four conditions that we went through
in the Forest Fire example:
Prediction and Reality can be easily mapped
together with the help of this confusion matrix.
Evaluation Methods
Accuracy, precision, and
recall are the three primary
measures used to assess the
success of a classification
algorithm.
Parameters determining the the
effectiveness of the model
Accuracy
Accuracy allows you to count the
total number of accurate predictions
made by a model. The accuracy
calculation is as follows: How many of
the model predictions were accurate
will be determined by accuracy. True
Positives and True Negatives are what
accuracy considers.
 Accuracy

The total observations cover all the possible cases of prediction that can
be True Positive (TP), True Negative (TN), False Positive (FP) and False
Negative (FN).
 Precision
Precision is defined as the percentage
of true positive cases versus all the
cases where the prediction is true. That
is, it takes into account the True
Positives and False Positives.
 Recall
It can be described as the percentage of positively
detected cases that are positive. The scenarios where a
fire actually existed in reality but was either correctly or
incorrectly recognised by the machine are heavily
considered. That is, it takes into account both False
Negatives (there was a forest fire but the model didn’t
predict it) and True Positives (there was a forest fire in
reality and the model anticipated a forest fire).
 Which Metric is Important?
Depending on the situation the model has been
deployed in, choosing between Precision and
Recall is necessary. A False Negative can cost us a
lot of money and put us in danger in a situation like
a forest fire. Imagine there is no need for a warning,
even in the case of a forest fire. The entire forest
might catch fire.
Viral Outbreak is another situation in which a False
Negative might be harmful. Consider a scenario in
which a fatal virus has begun to spread but is not
being detected by the model used to forecast viral
outbreaks. The virus may infect numerous people
and spread widely.
 Which Metric is Important?
Consider a model that can determine
whether a mail is spam or not. People
would not read the letter if the model
consistently predicted that it was
spam, which could lead to the
eventual loss of crucial information.
The cost of a False Positive condition in
this case (predicting that a message is
spam when it is not) would be high.
 F1 Score
F1 score can be
defined as the
measure of balance
between precision
and recall.
An ideal situation would be when we have a value of 1
(that is 100%) for both Precision and Recall. In that case,
the F1 score would also be an ideal 1 (100%). It is known
as the perfect value for F1 Score. As the values of both
Precision and Recall ranges from 0 to 1, the F1 score also
ranges from 0 to 1.
The variations we can have in the F1
Score:
ARTIFICIAL INTELLIGENCE
NATURAL LANGUAGE PROCESSING (NLP)

It is a sub field of AI that is focused on

enabling computers to understand and
process human language. It is concerned
with the interactions between computers
and human (natural language)
Applications of
NLP
CHATBOT……Most common application of NLP

1. Mitsuku Bot
https://www.pandorabots.com/mitsuku/
2. CleverBot 5. Haptik
https://haptik.ai/contact-us
https://www.cleverbot.com/
6. Ochatbot
3. Jabberwacky https://www.ometrics.com/blog/list-of-fun-chatbots/
http://www.jabberwacky.com/
4. Rose
http://ec2-54-215-197-164.us-west-1.compute.amazonaws.com/speech.php
 TYPES OF CHAT BOTS
Script Bot Smart Bot

Easy Flexible and powerful

Works around a programmed script Work on bigger databases

Mostly free and easy to integrate to a Learn with more data

messaging platform
It possesses No or very little language It has the language processing skills
processing skills
Limited functionality Wide functionality

Ex. Bots at customer care sections of a Ex: Google Assistant, Alexa, Siri etc.
Possible difficulties machine would face
while processing human language!!!

I. Multiple Meanings of a Word:

Check out the meaning of the word “RED” in the following
sectences…………..
❖My father turned red after seeing my result card.
❖I was very excited to see our new red car.
❖His face turned red after consuming the medicine
Before we go further we need to understand
two terms: Syntax & Semantics
Syntax: refers to the grammatical rules to be followed while writing
a sentence
Like….Rules of noun, verb, adverb, articles, conjunctions etc…
WHEREAS
Semantics: refers to the meaning conveyed by the sentence
Like….. I play with the wall.
This sentence is syntactically correct but semantically wrong.
That is, though it is grammatically correct but does not convey any
meaning.
 Possible difficulties machine would face
while processing human language!!!

II. Understanding Syntax and Semantics

III. Identifying sentences with perfect syntax but
no meaning
IV. Moreover Computer understand numbers, so
its very important to convert our language into
numbers so that the machine can process it.
 TEXT NORMALIZATION
It is the process of transforming a text into a canonical (standard)
form. When we normalize a natural language resource, we attemp
to reduce the randomness in it.
Few examples which highlights what we intend to do in normalization
of text.
✓ Removal of duplicate whitespaces and punctuation.
✓ Capital letter removal
✓ Remove hashtags
✓ Substitution of contractions e.g.: ‘I’m’→‘I am’
✓ Acronym normalization (e.g.: ‘US’→‘United States’/‘U.S.A’) and abbreviation
normalization (e.g.: ‘btw’→‘by the way’).
Text Normalization STEPS:

1. Sentence Segmentation
2. Tokenization
3. Removing stopwords, special characters and numbers
4. Converting Text to Common Case
5. Stemming or Lemmatization
 1. Sentence Segmentation
 In this case the entire corpus is divided into sentences. Corpus is nothing but the collection of
written text. For example consider the text given below converted to the output of sentence
segmentation:
 2. Tokenization
 Under tokenisation, every word, number and special character is considered separately
and thus divided into tokens. Considering the previous segmented text, we get the
following output of tokenization.
 3. Removing stopwords, special characters
and numbers
 Stopwords occur the most in any given corpus but talk very little or nothing about the
context or the meaning of it. Following are examples of the stop words..

 We check and remove the special characters

 We also check for the numbers. Depending upon what kind of system we want to develop, we see
for the relevance of the numbers .
 4. Converting Text to Common Case

Now, we convert the whole text into a similar case, preferably lower case.
 6. Stemming or Lemmatization

Stemming: Here, by removing the affixes we extract their root words. It does not take into account
if the stemmed word is meaningful or not. It just removes the affixes hence it is faster.

Here we see that,

after removing
the affix, the word
we got was not a
meaningful one.
 6. Stemming or Lemmatization

Lemmatization: Here, by removing the affixes we extract their root words. It converts
the word into a meaningful one hence is slower.

Here we see that,

after removing
the affix, the word
we got was a
meaningful one.
Now it is time to convert the tokens into numbers. For
this, we would use the Bag of Words algorithm.

Bag of Words is a Natural

Language Processing model
which helps in extracting
features out of the text which
can be helpful in machine
learning algorithms. In bag of
words, we get:
→the vocabulary of words
→ occurrences of each word
 Step-by-step approach to implement
Bag of Words algorithm:

1. Text Normalization: Collect data and pre-process it

2. Create Dictionary: Make a list of all the unique words occurring in
the corpus. (Vocabulary)
3. Create document vectors: For each document in the corpus, find
out how many times the word from the unique list of words has
occurred.
 Let us go through all the steps with an example:

Step 1(a): Collecting data & pre-processing it.

Doc 1: Aman and Anil are stressed
Doc 2: Aman went to a therapist
Step 1(b): Text normalization makes the
Doc 3: Anil went to download a health
text look like:
chatbot Doc 1: [aman, and, anil, are, stressed]
Doc 2: [aman, went, to, a, therapist]
Doc 3: [anil, went, to, download, a,
health, chatbot]
 Let us go through all the steps with an example:

Step 2: Create Dictionary

 Step 3: Create Document Vector Table
The vocabulary is written in the top row. For each word, if it matches with the vocabulary,
put a 1 under it. If the same word appears again, increment the previous value by 1. And if
the word does not occur in that document, put a 0 under it.
 TFIDF: Term Frequency and Inverse Document Frequency

Term Frequency
It is the frequency of a word in one document. Term frequency can easily be found from the
document vector table.
Document Frequency
It is the number of documents in which the word occurs irrespective of how many times it has
occurred in those documents.
Inverse Document Frequency
Here we need to put the document frequency in the denominator while the total number of
documents is the numerator
Lets create the TFIDF for our example:

Document
Vector
Table

Document
Frequencies

Inverse
Document
Frequencies
TFIDF for any word W becomes:
TFIDF(W) = TF(W) * log( IDF(W) )

Finally, the
words have
been
converted
to numbers
Artificial Intelligence

Natural Language Processing

Natural Language Processing Class 10 AI Shelly Batra

LEARNING OBJECTIVES
Introduction
Applications
Revisiting AI Project Cycle
Chatbots
Human Language vs Computer Language
Data Processing
Bag of Words
TF-IDF
Natural Language Processing Class 10 AI Shelly Batra
3
 Sentiment Analysis

Automatic Summarization

Text classification:

Natural Language Processing Class 10 AI Shelly Batra

Virtual Assistants: 4
Natural Language Processing Class 10 AI Shelly Batra
5
AI Project Cycle

Problem Data
Evaluation
Scoping Exploration

Data
Modelling
Acquisition

Natural Language Processing Class 10 AI Shelly Batra

Natural Language Processing Class 10 AI Shelly Batra
Natural Language Processing Class 10 AI Shelly Batra
8
4Ws Problem Canvas

WHO ? WHAT ?

WHERE ? WHY ?

Natural Language Processing Class 10 AI Shelly Batra

4Ws Problem Canvas
Who is having the problem?
1. Who are the stakeholders?
1. People who suffer from stress and are at the onset of
depression.
2. What do you know about them?
1. People who are going through stress are reluctant to consult a
psychiatrist.
Natural Language Processing Class 10 AI Shelly Batra
4Ws Problem Canvas
What is the nature of the problem?

1. What is the problem?

1. People who need help are reluctant to consult a psychiatrist
and hence live miserably.
2. How do you know it is a problem?
1. Studies around mental stress and depression available on
various authentic sources.
Natural Language Processing Class 10 AI Shelly Batra
4Ws Problem Canvas
Where does the problem arise?

1. What is the context/situation in which the stakeholders experience

the problem?
1. When they are going through a stressful period of time

2. Due to some unpleasant experiences

Natural Language Processing Class 10 AI Shelly Batra

4Ws Problem Canvas
Why do you believe it is a problem worth solving?
1. What would be of key value to the stakeholders?
1. People get a platform where they can talk and vent out their feelings
anonymously
2. People get a medium that can interact with and which applies primitive
CBT on them and can suggest help whenever needed
2. How would it improve their situation?
1. People would be able to vent out their stress
2. They would consider going to a psychiatrist whenever required
Natural Language Processing Class 10 AI Shelly Batra
 Problem Statement
Our People undergoing stress Who

has a problem that Not being able to share their feelings What

when / while They need help in venting out their emotions Where

An ideal solution Provide them a platform to share their Why

would thoughts anonymously and suggest help
whenever required

Natural Language Processing Class 10 AI Shelly Batra

This leads us to the goal of our project which
is:
“To create a chatbot which can interact with
people, help them to vent out their feelings and
take them through primitive CBT.”
Natural Language Processing Class 10 AI Shelly Batra
Data can be collected from various means:
1. Surveys
2. Observing the therapist’s sessions
3. Databases available on the internet
4. Interviews, etc.

Natural Language Processing Class 10 AI Shelly Batra

16
Once the textual data has been collected, it needs
to be processed and cleaned so that an easier
version can be sent to the machine. Thus, the text
is normalised through various steps and is
lowered to minimum vocabulary since the
machine does not require grammatically correct
statements but the essence of it.

Natural Language Processing Class 10 AI Shelly Batra

17
Once the text has been normalised, it is then fed to
an NLP based AI model. Note that in NLP, modelling
requires data pre-processing only after which the data
is fed to the machine. Depending upon the type of
chatbot we try to make, there are a lot of AI models
available which help us build the foundation of our
project.

Natural Language Processing Class 10 AI Shelly Batra

18
The model trained is then evaluated and the
accuracy for the same is generated on the basis
of the relevance of the answers which the
machine gives to the user’s responses. To
understand the efficiency of the model, the
suggested answers by the chatbot are
compared to the actual answers.
Natural Language Processing Class 10 AI Shelly Batra
19
Natural Language Processing Class 10 AI Shelly Batra
Model Evaluation
 The blue line talks about the model’s output while the green one is the
actual output along with the data samples.
 Figure1:-The model’s output does not match the true function at all. Hence
the model is said to be underfitting and its accuracy is lower.
 Figure 2:-the model’s performance matches well with the true function
which states that the model has optimum accuracy and the model is called
a perfect fit.
 Figure3:-model performance is trying to cover all the data samples even if
they are out of alignment to the true function. This model is said to be
overfitting and this too has a lower accuracy.

Natural Language Processing Class 10 AI Shelly Batra

Chatbots
Cleverbot Mitsuku Bot

Jabberwacky Haptik

Rose
Ochatbot

Natural Language Processing Class 10 AI Shelly Batra

Natural Language Processing Class 10 AI Shelly Batra
23
 Script Bot Smart Bot
Script bots are easy to make Smart bots are flexible and
powerful

Script bots work around a script Smart bots work on large

which is programmed around databases and other resources
them directly

Most of them are free and easy Smart bots learn with more data
to integrate with a messaging
platform

None or little language Coding is required

processing skills

Limited functionality Wide functionality

Humans communicate through language which we process all the
time. Our brain keeps on processing the sounds that it hears around
itself and tries to make sense out of them all the time. Even in the
classroom, as the teacher delivers the session, our brain is
continuously processing everything and storing it in some place.
Also, while this is happening, when your friend whispers something,
the focus of your brain automatically shifts from the teacher’s speech
to your friend’s conversation. So now, the brain is processing both
the sounds but is prioritising the one on which our interest lies.
Natural Language Processing Class 10 AI Shelly Batra
25
Syntax: There are rules in human language. There are
nouns, verbs, adverbs, adjectives. A word can be a noun at
one time and an adjective some other time. There are rules
to provide structure to a language. Syntax refers to the
grammatical structure of a sentence.

Semantics: What does it mean?

Semantics is the study of the meaning of linguistic
expressions. Semantics is the study of meaning. There
are two types of meaning: conceptual meaning and
associative meaning. Meaning in natural languages is
mainly studied by linguists.
Natural Language Processing Class 10 AI Shelly Batra
 Syntax: What is grammatical?

Semantics: What does it mean?

Analogy with programming language:

Different syntax, same semantics (5): 2+3 = 3+2
Same syntax, different semantics (1 and 1.5)
3 / 2 (Python 2.7) =/= 3/2 (Python 3)
Natural Language Processing Class 10 AI Shelly Batra
 Multiple Meanings of a word

His face turned red after he found out that

he had taken the wrong bag

The red car zoomed past his nose

His face turns red after consuming the

medicine

Natural Language Processing Class 10 AI Shelly Batra

Natural Language Processing Class 10 AI Shelly Batra
29
During Text Normalisation, collection of written text
undergoes several steps to normalise the text to a lower
level.
The text on which we work upon is from multiple
documents and the term used for the whole textual data
from all the documents altogether is known as corpus.

Natural Language Processing Class 10 AI Shelly Batra

Sentence Segmentation:- Each sentence is taken as a
different data so now the whole corpus gets reduced to sentences.

Natural Language Processing Class 10 AI Shelly Batra

Tokenisation:- After segmenting the sentences, each sentence is then
further divided into tokens. Tokens is a term used for any word or number or special
character occurring in a sentence. Under tokenisation, every word, number and special
character is considered separately and each of them is now a separate token.

Natural Language Processing Class 10 AI Shelly Batra

Remove Stop words, Numbers and Special
Characters
Stop words are the words which occur very frequently in the corpus
but do not add any value to it.

Natural Language Processing Class 10 AI Shelly Batra

Convert text to common case
After the stop words removal, we convert the whole text into a similar case, preferably lower case.
This ensures that the case-sensitivity of the machine does not consider same words as different
just because of different cases.
Example, the all the 6 forms of hello would be converted to lower case and hence would be
treated as the same word by the machine.

Natural Language Processing Class 10 AI Shelly Batra

Stemming:- stemming is the process in which the affixes of words
are removed and the words are converted to their base form or root word.
Word Affixes Stem

healed -ed heal

healing -ing heal

healer -er heal

studies -es studi

studying -ing study

Natural Language Processing Class 10 AI Shelly Batra
Lemmatisation:- the word we get after affix removal (also
known as lemma) is a meaningful one. Lemmatization makes sure that
lemma is a word with meaning and hence it takes a longer time to execute
than stemming.
Word Affixes Stem
healed -ed heal
healing -ing heal
healer -er heal
studies -es study
studying -ing study
Natural Language Processing Class 10 AI Shelly Batra
Difference between stemming and lemmatization can be
summarized by this example:

Natural Language Processing Class 10 AI Shelly Batra

Natural Language Processing Class 10 AI Shelly Batra
Bag of words

Natural Language Processing Class 10 AI Shelly Batra

 Corpus
• Document 1: Aman and Anil are stressed
• Document 2: Aman went to a therapist
• Document 3: Anil went to download a health chatbot

Natural Language Processing Class 10 AI Shelly Batra

Here is the step-by-step approach to implement bag of words
algorithm:
1. Text Normalisation: Collect data and pre-process it
2. Create Dictionary: Make a list of all the unique words occurring
in the corpus. (Vocabulary)
3. Create document vectors: For each document in the corpus,
find out how many times the word from the unique list of words
has occurred.
4. Create document vectors for all the documents.

Natural Language Processing Class 10 AI Shelly Batra

After text normalisation, the text becomes:
Document 1: [aman, and, anil, are, stressed]
Document 2: [aman, went, to, a, therapist]
Document 3: [anil, went, to, download, a,
health, chatbot]

Natural Language Processing Class 10 AI Shelly Batra

Create Dictionary

Natural Language Processing Class 10 AI Shelly Batra

Create Document Vectors

Natural Language Processing Class 10 AI Shelly Batra

Repeat for all documents

Natural Language Processing Class 10 AI Shelly Batra

46
Natural Language Processing Class 10 AI Shelly Batra
Term Frequency

Natural Language Processing Class 10 AI Shelly Batra

Document Frequency

Natural Language Processing Class 10 AI Shelly Batra

Inverse Document Frequency

Natural Language Processing Class 10 AI Shelly Batra

Term Frequency – Inverse Document Frequency
TFIDF(W) = TF(W) * log( IDF(W) )

Natural Language Processing Class 10 AI Shelly Batra

Term Frequency – Inverse Document Frequency

Natural Language Processing Class 10 AI Shelly Batra

Natural Language Processing Class 10 AI Shelly Batra
53
Natural Language Processing Class 10 AI Shelly Batra
Factors contributing to wastage of food / Data Acquisition on the basis of following factors
System Map for the problem
understanding