1.

Introduction
 History of NLP

Generic NLP system

Levels of NLP

1. Knowledge in language processing

Introduction Ambiguity in Natural language

Stages in NLP

Challenges of NLP

Applications of NLP
 •Mode of
Language
Communication

NLP
•Artificial
Two Language
types
•Natural language
 Natural language

•Two types
Natural •Spoken form
Language •Written form

Why processing??
NLP
•Natural Language Processing
• The process of computer analysis of input provided in a human
language ( any natural language), and conversion of this input into
a useful form of representation.
HISTORY OF NLP (1906-1911)
• In the early 1900s, a Swiss linguistics professor named Ferdinand de
Saussure
• From 1906 to 1911, Professor Saussure offered three courses at the
University of Geneva, where he developed an approach describing
languages as “systems.
• Within the language, a sound represents a concept – a concept that
shifts meaning as the context changes.
• He argued that meaning is created inside language, in the relations
and differences between its parts
HISTORY OF NLP (1913)
• Saussure died in 1913, but two of his colleagues, Albert Sechehaye
and Charles Bally, recognized the importance of his concepts.
• Based on the concepts of Prof Sausre, they wrote a book which laid
the foundation for what has come to be called the structuralist
approach, starting with linguistics, and later expanding to other fields,
including computers.
History of NLP (1950…)
• In 1950, Alan Turing wrote a paper describing a test for a “thinking”
machine.
• He stated that if a machine could be part of a conversation through
the use of a teleprinter, and it imitated a human, then the machine
could be considered capable of thinking.
• in 1952, the Hodgkin-Huxley model showed how the brain uses
neurons in forming an electrical network.
• These events helped inspire the idea of Artificial Intelligence
(AI), Natural Language Processing (NLP), and the evolution of
computers.
History of NLP (1957…)
• Noam Chomsky published his book, Syntactic Structures, in 1957.
• In it, he revolutionized previous linguistic concepts, concluding that
for a computer to understand a language, the sentence structure
would have to be changed.
• With this as his goal, Chomsky created a style of grammar called
Phase-Structure Grammar, which methodically translated natural
language sentences into a format that is usable by computers.
• he overall goal was to create a computer capable of imitating the
human brain, in terms of in thinking and communicating, or AI.
HISTORY OF NLP (1960’S)
• In 1958, the programming language LISP (Locator/Identifier
Separation Protocol), a computer language still in use today, was
released by John McCarthy.
• LISP was designed for research on AI
• In 1964, ELIZA, a “typewritten” comment and response process,
designed to imitate a psychiatrist using reflection techniques, was
developed.
• ELIZA is a natural language conversation program described by Joseph
Weizenbaum in January 1966. It features the dialog between a
human user and a computer program representing a mock Rogerian
psychotherapist.
Eliza
HISTORY OF NLP
• In 1964, the U.S. National Research Council (NRC) created the Automatic
Language Processing Advisory Committee, or ALPAC
• This committee was tasked with evaluating the progress of Natural
Language Processing research.
• In 1966, the NRC and ALPAC initiated the first AI and NLP stoppage, by
halting the funding of research on Natural Language Processing and
machine translation.
• After twelve years of research, and $20 million dollars, machine
translations were still more expensive than manual human translations,
and there were still no computers that came anywhere near being able to
carry on a basic conversation.
• In 1966, Artificial Intelligence and Natural Language Processing (NLP)
research was considered a dead end by many.
History of NLP (1980…)
• AI stoppage had initiated a new phase of fresh ideas, with earlier
concepts of machine translation being abandoned, and new ideas
promoting new research, including expert systems.
• The mixing of linguistics and statistics, which had been popular in
early NLP research, was replaced with a theme of pure statistics.
• The 1980s initiated a fundamental reorientation, with simple
approximations replacing deep analysis, and the evaluation process
becoming more rigorous.
History of NLP(1990….)
• Until the 1980s, the majority of NLP systems used complex,
“handwritten” rules.
• steady increase of computational power, and the shift to Machine
Learning algorithms, research has increasingly focused on statistical
models.
• 1990’s saw statistical models and N grams model used for NLP
research
• In 1997, LSTM recurrent neural net (RNN) models were introduced,
and found their niche in 2007 for voice and text processing.
History of NLP (2000….)
• In 2001, Yoshio Bengio and his team proposed the first neural “language”
model, using a feed-forward neural network
• In this type of network, the data moves only in one direction, from input
nodes, through any hidden nodes, and then on to the output nodes.
• In the year 2011, Apple’s Siri became known as one of the world’s first
successful NLP/AI assistants to be used by general consumers.
• Within Siri, the Automated Speech Recognition module translates the
owner’s words into digitally interpreted concepts.
• The Voice-Command system then matches those concepts to predefined
commands, initiating specific actions.
History of NLP
• 1950- Attempts to automate translation between Russian and
English
• 1960- The work of Chomsky and others on formal language theory
and generative syntax
• 1990- Probabilistic and data-driven models had become quite
standard
• 2000- A Large amount of spoken and textual data become available
Generic NLP systems
 Levels of NLP

Phonology Morphology Syntax

Semantics Reasoning
Levels of NLP
• Phonology
• Speech processing
Phonology
• Morphology
• The cats are fighting

Morphology
Levels of NLP
• Syntax/Syntactic
Syntax
processing
• It is a study of formal
relationships
between words.

• The cat chased the

mouse
• Parts of speech
• The--- determinant
• Cat- noun
• Chase- verb
• Mouse - noun
Levels of NLP
• Semantics
• Determining the meaning of the words
Semantics
• Bat is flying in the sky
Levels of NLP- reasoning
• To produce an answer to a question which is not explicitly stored in a
database;
• Natural Language Interface to Database (NLIDB) carries out reasoning
based on data stored in the database.
• For example, consider the database that holds the academic
information about student,
• and user posed a query such as:
Reasoning
• ‘Which student is likely to fail in Maths subject?’.
• To answer the query, NLIDB needs a domain expert to narrow down
the reasoning process.
 Lexical • “The crane is loaded.”
Semantic • “The beak of the crane is very
Ambiguity big.”

Lexical • “She received three silver vessels.

Ambiguity in Ambiguity • “Reena gave a silver talk”

Natural
Language Syntactic
Ambiguity
• “He saw a man with binoculars.”

Anaphoric • “Cat went up the hill. It was

Ambiguity slipery. It got angry”
Ambiguity in Natural Language
Pronoun Translation Parallel Corpus Ambiguity

• “Monkey ate the • “Monkey ate the • “Bank”

banana as it was banana as it was • “उ र ”
hungry.” hungry”
• “Monkey ate the • “माकडा ने केळ खा ल
banana as it was ripe” कारण तो भुकेला होता.”
• “बंदर ने केला खाया
यो॑ क उसे भूख लगी थी
।”
Stages in NLP
Preprocessing in NLP
 Bat is flying in the sky

[bat, flying, in, sky] [is, the]

Knowledge in Flying ---- fly. (Morphology)
language
[batNN, flyVB, inPP, skyNN]
processing
Bat fly, sky. (dependency / syntactic)

Bat – mammal ( semantics)

Stages in NLP
• Lexical Analysis
• Refer the dictionary and obtain the properties of the word

• Eg: Dog
• Noun
• Take ‘s’ in plural
• Animate
• 4 legged
• carnivore
Stages in NLP
Challenges in Lexical analysis
Stages in
NLP
Syntactic Parsing strategy
Stages of NLP
• Challenges in Syntactic Processing
Stages in
NLP
Stages in NLP- semantic analysis
• John gave book to Mary
• Agent is john
• Object is book
• Mary is recipient

• Challenges:
• Bat/crane/ tank
• Uttar/peru in marathi
Stages in NLP- Disclosure Integration
• Disclosure Integration
• The meaning of any sentence depends upon the
meaning of the sentence just before it. In addition,
it also brings about the meaning of immediately
succeeding sentence.

• Example

• Manoj went to the bank. He said it was crowded

Stages in NLP- Disclosure Integration
Stages in NLP - Pragmatics
• During this, what was said is re-interpreted on
what it actually meant.
• It involves deriving those aspects of language
which require real world knowledge.
Challenges of NLP • POS tagging
Challenges of NLP
• Pronoun resolution

• a. The thieves stole the paintings. They were subsequently sold.

• b. The thieves stole the paintings. They were subsequently caught.
• c. The thieves stole the paintings. They were subsequently found.
 Word sense
disambiguation

Challenges of a. He went to the bank

NLP to withdraw money

b. He went near the

river bank
 Applications of NLP

Machine Information Question

translation Retrieval Answering

Sentiment Text
NER
analysis summarization
 Machine Translation
• Automated system for translation of text from any
source language to target language.

वा षक ग त
संगो ठ म आपका
वागत है
IR
QA system
Text summarization
TEXT SUMMARIZATION
NER