Chapter 6

Natural Language Processing

Definition:
Natural Language Processing (NLP)
• Is defined as the branch of Artificial Intelligence that provides computers with the
capability of understanding text and spoken words in the same way a human being
can.
• It incorporates machine learning models, statistics, and deep learning models in
to computational linguistics i.e.,
• Rule-based modeling of human language to allow computers to understand text,
spoken words and understands human language, intent, and sentiment
• Natural Language Processing or NLP refers to the branch of Artificial Intelligence
that gives the machines the ability to read, understand and derive meaning from
human languages.
• NLP combines the field of linguistics and computer science to decipher language
structure and guidelines and to make models which can comprehend, break down
and separate significant details from text and speech.
Foundations of NLP
Phases of NLP
The different phases of Natural Language Processing (NLP).
1. Lexical Analysis:
This phase involves breaking the sentences and paragraphs into tokens, which are the smallest units
of text. It scans the source text and identifies meaningful lexemes.
For example, the sentence
"He goes to college." would be divided into tokens like
"He," "goes," "to," "college," and ".

2. Syntactic Analysis/Parsing:
In this phase, the sentence structure and grammar are analyzed to determine if it is well-formed.
The word arrangement is studied, and syntactic relationships between words are identified.
Incorrect word arrangements or grammar may result in rejection by the syntactic parser
 Cont’d
 The Semantic analysis phase:
• focuses on understanding the literal meaning of each word and how they
combine to form coherent sentences. Sentences that do not make sense or have
conflicting meanings may be rejected by the semantic analyzer.
 Discourse Integration:
• This phase considers the impact of prior sentences on the current sentence and
the effect of the current sentence on upcoming sentences. The context and
coherence of the discourse are determined to ensure a meaningful flow of
information.
 Pragmatic Analysis:
• The final phase, pragmatic analysis, deals with the actual effect and
interpretation of the text. It applies rules that characterize cooperative dialogues
to understand the intentions behind the language. For example, a sentence like
"Close the window?" would be interpreted as a request rather than an order.
Phases of NLP
NLP Implementation
• Natural Language Processing (NLP) methods and outlines the steps
involved in performing NLP.
• NLP utilizes machine learning and statistical inference to process text
data.
• The steps in NLP include segmentation, tokenizing, removing stop
words, stemming, lemmatization, part of speech tagging, named entity
tagging, and applying machine learning algorithms.
• These steps help in breaking down text, extracting meaningful
information, and training models for various NLP applications.
Popular methods used for Natural Learning Process:

• Machine learning:
• Natural Learning Process procedures used during machine learning.
• It automatically focuses on the most common cases. Errors will be minimized.
• Statistical inference:
• NLP can make use of statistical inference algorithms. It helps you to produce
models that are robust. e.g., containing words or structures which are known
to every one
Performing NLP?
1. Segmentation
Break the entire document down into its constituent sentences. You can do
this by segmenting the article along with its punctuation like full stops and
commas.
2. Tokenizing

• For the algorithm to understand these sentences, you need to get the words in a
sentence and explain them individually to our algorithm. So, you break down your
sentence into its constituent words and store them.
• This is called tokenizing, and each world is called a token.
3. Removing Stop Words

• You can make the learning process faster by getting rid of non-essential
words, which add little meaning to our statement and are just there to make
our statement sound more cohesive.
• Words such as was, in, is, and, the, are called stop words and can be
removed.
4. Stemming

• It is the process of obtaining the Word Stem of a word. Word Stem gives new words upon
adding affixes to them

5. Lemmatization
• The process of obtaining the Root Stem of a word. Root Stem gives the new base form
of a word that is present in the dictionary and from which the word is derived.
• You can also identify the base words for different words based on the tense, mood,
gender, etc.
Part of Speech Tagging

• Now, you must explain the concept of nouns, verbs, articles, and other parts of
speech to the machine by adding these tags to our words. This is called part of
Speech Tagging.
Named Entity Tagging

• It introduce the machine to pop culture references and everyday names by flagging names of
movies, important personalities or locations, etc. that may occur in the document.
• This can be done by classifying the words into subcategories helpful for finding any
keywords in a sentence.
• The subcategories are person, location, monetary value, quantity, organization, movie.

• After performing the preprocessing steps, then it gives resultant data to a machine learning
algorithm like Naive Bayes, etc., to create NLP application.
Applications of NLP
• Natural Language Processing (NLP) has humanized machines and reduced the need for

manual labor.

• It enables automation of speech-related tasks and human interaction.

• Applications of NLP include translation tools, chatbots, virtual assistants, targeted advertising,

autocorrect, information retrieval, grammar correction, question answering, text summarization,

and machine translation.

• The future of NLP holds the potential for computers to learn from online information and apply

it in the real world.

• NLP systems allow users to ask questions and receive direct responses in natural language,

providing accurate and relevant information.

Application areas of NLP
Application areas :
 Translation Tools: Tools such as Google Translate, Amazon Translate, etc. translate sentences from one language to

another using NLP.

 Chatbots: Chatbots can be found on most websites and are a way for companies to deal with common queries quickly.
 Virtual Assistants: Virtual Assistants like Siri, Cortana, Google Home, Alexa, etc can not only talk to you but understand
commands given to them.
 Targeted Advertising: Have you ever talked about a product or service or just googled something and then started seeing
ads for it? This is called targeted advertising, and it helps generate tons of revenue for sellers as
they can reach niche audiences at the right time.
 Autocorrect: Autocorrect will automatically correct any spelling mistakes you make, apart from this grammar checkers
also come into the picture which helps you write flawlessly.
 Information retrieval & Web Search: Google, Yahoo, Bing, and other search engines base their machine translation
technology on NLP deep learning models. It allows algorithms to read text on a webpage, interpret its
meaning and translate it to another language.
 Grammar Correction: NLP technique is widely used by word processor software like MS- word for spelling correction &
grammar check.
 Question Answering፡ Type in keywords to ask Questions in Natural Language.
 Text Summarization፡ The process of summarizing important information from a source to produce a shortened version
Future of NLP

 Human readable natural language processing is the biggest Al- problem.

 It is all most same as solving the central artificial intelligence problem and
making computers as intelligent as people.
 Future computers or machines with the help of NLP will able to learn
from the information online and apply that in the real world, however, lots
of work need to on this regard.
 Natural language toolkit or nltk become more effective; combined with
natural language generation, computers will become more capable of
receiving and giving useful and resourceful information or data.
Natural Language vs. Computer Language

Parameter Natural Language Computer Language

Ambiguous They are ambiguous in nature They are design to unambiguous

Natural Languages employ lots of Formal languages are less redundant

Redundancy redundancy

Literalness Natural Languages are made of idiom Formal language mean exactly what
and metaphor they want to say
 Advantages of NLP

• Users can ask questions about any subject and get a direct response within seconds.
NLP system provides answers to the questions in natural language
• NLP system offers exact answers to the questions, no unnecessary or unwanted
information
• The accuracy of the answers increases with the amount of relevant information
provided in the question.
• NLP process helps computers communicate with humans in their language and scales
other language-related tasks
• Allows you to perform more language-based data compares to a human being without
fatigue and in an unbiased and consistent way.
• Structuring a highly unstructured data source
Disadvantages of NLP

• Complex Query Language- the system may not be able to provide the correct
answer it the question that is poorly worded or ambiguous.
• The system is built for a single and specific task only; it is unable to adapt to new
domains and problems because of limited functions.
• NLP system doesn‘t have a user interface which lacks features that allow users to
further interact with the system
Computer vision and Image
processing
• It is the subfields of artificial intelligence.
• Computer vision involves acquiring, processing, analyzing, and interpreting visual
data such as images and videos.
• It enables computer systems to "see" and understand the world around them.
• Computer vision tasks include object classification, object
identification/detection, object verification, object landmark detection, image
segmentation, and object recognition.
• Computer vision finds applications in various industries such as facial
recognition, healthcare, self-driving vehicles, retail, and surveillance.
 Cont’d
• Image processing, on the other hand, involves manipulating and enhancing digital
images using computer algorithms.
• It serves purposes like visualization, image sharpening and restoration, image
retrieval, object measurement, and pattern recognition.
• Image processing includes stages such as image acquisition, image enhancement,
image restoration, color image processing, and image
compression/decompression.
• Both computer vision and image processing rely on artificial intelligence
techniques and play significant roles in various fields, improving efficiency,
accuracy, and automation.
 Cont’d
 Computer vision is a subfield of artificial intelligence that deals with acquiring, processing, analyzing, and making

sense of visual data such as digital images and videos.

 Computer vision is one of the most important fields of artificial intelligence (AI) and computer science engineering that

makes computer systems capable of extracting meaningful information from visual data like videos and images.

 It is one of the most compelling types of artificial intelligence that we regularly implement in our daily routines.

Computer vision helps to understand the complexity of the human vision system and trains computer systems to

interpret and gain a high-level understanding of digital images or videos

 It also helps to take appropriate actions and make recommendations based on the extracted information.

 In the early days, developing a machine system having human-like intelligence was just a dream, but with the

advancement of artificial intelligence and machine learning, it also became possible. Similarly, such intelligent systems

have been developed that can "see" and interpret the world around them, similar to human eyes.

 The fiction of yesterday has become the fact of today.

 How does Computer Vision Work?

• Computer vision is a technique that extracts information from visual data, such as
images and videos.
• Although computer vision works similarly to human eyes with brain work, this is
probably one of the biggest open questions for IT professionals:
• How does the human brain operate and solve visual object recognition?
Task Associated with Computer Vision

• Although computer vision has been utilized in so many fields, there are a
few common tasks for computer vision systems. These tasks are given
below:
Applications of computer vision

 Facial recognition

 Healthcare and Medicine

 Self-driving vehicles
 Optical character recognition (OCR)

 Machine inspection

 Retail (e.g., automated checkouts)

 3D model building

 Medical imaging

 Automotive safety

 Surveillance
 Fingerprint recognition and biometrics:
Image Processing
 Image processing involves manipulating images to enhance them or extract valuable
information. There are two methods:
 analog image processing for physical photographs and
 digital image processing using computer algorithms.

 The main purposes of image processing include visualization, image sharpening

and restoration, image retrieval, object measurement, and pattern recognition.
 Image acquisition is the process of capturing an image and converting it into a
digital file.
 Image enhancement improves image quality to extract hidden information.
 Image restoration removes corruptions like blur, noise, and missing pixels to
obtain a cleaner version. color
Main purposes of image processing:

• Visualization:
Represent processed data in an understandable way, giving visual form to
objects that aren‘t visible, for instance:
• Image sharpening and restoration:
Improve the quality of processed images Image retrieval: Help with image
search
• Object measurement:
Measure objects in an image
• Pattern recognition:
Distinguish and classify objects in an image, identify their positions, and
understand the scene
Key Phases of Digital Image
Processing
Thank You !