NLP Final Notes
Uploaded by
mankameapurvaaNLP Final Notes
Uploaded by
mankameapurvaaMODULE 1
1.Explain Ambiguity associated at each level in NLP with example
In Natural Language Processing (NLP), ambiguity refers to situations where a word, phrase, or
sentence has multiple possible interpretations. Ambiguity can arise at different levels of
language processing, each presenting unique challenges for machines to resolve. Here, we'll
explain the types of ambiguity associated with each level of NLP, along with examples.
Levels of Ambiguity in NLP
.
1 exical Ambiguity(Word Level)
L
2. Syntactic Ambiguity(Sentence Structure Level)
3. Semantic Ambiguity(Meaning Level)
4. Anaphoric Ambiguity
5. Discourse Ambiguity(Multiple Sentences)
1. Lexical Ambiguity (Word Level)
efinition: Lexical ambiguity occurs when a singleword has multiple meanings (polysemy), and
D
the correct meaning depends on the context in which the word is used.
Example:
● Word: "Bank"
○ Sense 1: Afinancial institutionwhere people keepmoney.
○ Sense 2: Theside of a river(riverbank).
● Sentence:"He went to the bank to fish."
○ Interpretation 1: The person went to afinancial institutionto engage in some
activity (unlikely here).
○ Interpretation 2: The person went to theriverbankto fish (this is the correct
meaning in the context).
esolution: The surrounding context (e.g., the word"fish") helps disambiguate the meaning of
R
"bank" as ariverbank.
2. Syntactic Ambiguity (Sentence Structure Level)
efinition: Syntactic ambiguity arises when a sentence can be parsed in more than one way
D
due to its syntactic structure, often due to word order or the lack of punctuation.
Example:
● Sentence:"I saw the man with the telescope."
This sentence has two interpretations:
Interpretation 1: The speaker sawa manwho hadatelescope.
●
● Interpretation 2: The speaker saw the man throughthe lens ofa telescope.
esolution: The ambiguity arises because the phrase"with the telescope" can be interpreted
R
as either a modifier of "man" (describing the man) or as a prepositional phrase indicating the
method of seeing.
3. Semantic Ambiguity (Meaning Level)
efinition: Semantic ambiguity occurs when a word,phrase, or sentence has multiple
D
meanings due to ambiguity in its overall meaning. This is more than just word-level ambiguity
and can be seen when different interpretations arise from the same sentence structure.
Example:
● Sentence:"She was reading a book on the beach."
The phrase "reading a book on the beach" could mean:
. Interpretation 1: She was reading a book while physicallylocatedon the beach.
1
2. Interpretation 2: She was reading a book aboutbeaches.
esolution: The meaning depends on the surroundingcontext. If the sentence is part of a
R
vacation narrative, the first interpretation is likely. If it's in a discussion about literature, the
second interpretation is more probable.
4. Anaphoric Ambiguity (Reference Ambiguity)
efinition: Anaphoric ambiguity arises when a pronounor another referring expression (like
D
"he," "she," "it," "they," etc.) refers to something that could be understood in multiple ways. The
ambiguity occurs due to a lack of clarity about what the pronoun refers to.
Example:
● Sentence:"John went to the park. He saw a bird."
● Ambiguity: The pronoun"he"could refer to:
1. John(the person mentioned earlier).
2. Another person or entity (if there is a previous sentence with a different subject).
● Resolution: In this case, the most logical resolutionis that"he"refers toJohn. The
context and proximity of "John" help resolve this ambiguity.
5. Pragmatic Ambiguity (Context/Use Level)
efinition: Pragmatic ambiguity occurs when the meaningof a sentence or phrase depends
D
heavily on the context of the conversation, the speaker's intention, or the broader situational
context.
Example:
S
● entence:"Can you pass the salt?"
● Interpretation 1: The speaker is asking if the listeneris able to pass the salt (a literal
question).
● Interpretation 2: The speaker is requesting the listenerto pass the salt (a polite
command or request).
esolution: The context of the situation resolvesthe ambiguity. In a dinner setting, the phrase
R
is typically interpreted as a polite request rather than a question about the listener’s ability.
MODULE 2
Affixes and Their Types
naffixis a morpheme (the smallest meaningful unit of language) that is attached to a word to
A
modify its meaning or grammatical function. Affixes are not words on their own and must
combine with root words, stems, or bases to form a complete word. They are fundamental to
morphology, the study of word structure.
Types of Affixes
ffixes can be categorized based on their position relative to the root or stem of a word and their
A
function. The major types include:
1. Prefixes
D
● efinition: Affixes that are added to the beginning of a root or stem.
● Function: Often change the meaning of the word but rarely its grammatical category.
● Examples:
○ "Un-" inunhappy(negation)
○ "Re-" inrewrite(repetition)
○ "Pre-" inpreview(time or order)
2. Suffixes
D
● efinition: Affixes that are attached to the end of a root or stem.
● Function: Can change the grammatical category of the word (e.g., noun to adjective) or
its tense, number, case, etc.
● Examples:
○ "-ed" inwalked(past tense)
○ "-ness" inkindness(noun formation)
○ "-ly" inquickly(adverb formation)
3. Infixes
D
● efinition: Affixes that are inserted within a rootor stem.
● Function: Rare in English but common in other languages(e.g., Tagalog, Arabic).
● Examples:
○ In Tagalog:sulat(write) →sumulat(to write).
○ In informal English:abso-bloody-lutely(infixationfor emphasis).
Output of Morphological Analysis
orphological analysis decomposes a word into its components (morphemes) and identifies its
M
base form and grammatical features. Below are examples of morphological analysis for different
word categories:
1. Regular Verb
ord:walked
W
Output:
R
● oot:walk
● Tense:past
● Verb Type:regular
2. Irregular Verb
ord:went
W
Output:
R
● oot:go
● Tense:past
● Verb Type:irregular
3. Singular Noun
ord:cat
W
Output:
R
● oot:cat
● Number:singular
● Part of Speech:noun
4. Plural Noun
ord:cats
W
Output:
R
● oot:cat
● Number:plural
● Part of Speech:noun
Role of Finite State Transducer (FST) in Morphological Parsing
Finite State Transducer (FST)is a computational model used formorphological parsing. It
A
maps input strings (surface forms) to their respectivelexical forms(base/root words and
features) by using states and transitions. FSTs are widely used because of their efficiency and
ability to represent complex linguistic rules.
How FST Works in Morphological Parsing
. Input:Surface form of a word (e.g.,walked).
1
2. Transitions:Based on rules, FST transitions throughstates that correspond to
morphemes and grammatical markers.
3. Output:The FST generates the lexical form of the word (e.g., root word and
grammatical features).
Example of FST for Morphological Parsing
Input Word:cats
FST Representation:
1. States:
○ q0q_0q0: Start state.
○ q1q_1q1: Root word state.
○ q2q_2q2: Suffix state.
2. Transitions:
○ q0→cat/q1q1q_0 \xrightarrow{cat/q_1} q_1q0cat/q1q1: Extract root "cat".
○ q1→s/+PLURALq2q_1 \xrightarrow{s/+PLURAL} q_2q1s/+PLURALq2: Identify
suffix "s" and its feature (plural).
3. Output:
○ Root:cat
○ Number:plural
Advantages of FST in Morphological Parsing
. E
1 fficiency:Processes words quickly with low computational overhead.
2. Modularity:Easily integrates with other linguisticprocessing tasks.
3. Rule-Based Flexibility:Can handle complex morphological rules (e.g., regular and
irregular inflections).
4. Reversible:Can generate surface forms from lexicalforms.
Example of a Complete FST Process
Word:running
. Input:running
1
2. Steps in Parsing:
○ Detectrunas the root word.
○ Detect-ingas a suffix indicating present participle.
3. Output:
○ Root:run
○ Grammatical Features:progressive
FST Representation:
. q
1 0→run/q1q1q_0 \xrightarrow{run/q_1} q_1q0run/q1q1: Root extraction.
2. q1→ing/+PROGRESSIVEq2q_1 \xrightarrow{ing/+PROGRESSIVE}
q_2q1ing/+PROGRESSIVEq2: Suffix analysis.
Good Turing Discounting
1. Finite State Automaton Overview
An FSA consists of:
S
● tates: Represent conditions or stages (e.g., start,intermediate, and accept states).
● Transitions: Represent the movement between statestriggered by input symbols (e.g.,
letters or words).
● Alphabet: The set of symbols the FSA operates on (e.g.,characters or words).
● Start State: The state where processing begins.
● Accept/Final States: States where the FSA concludes recognizing valid patterns.
2. FSA for Nouns
ouns in English can be singular or plural, and they often follow specific patterns. FSAs for
N
nouns can recognize these patterns.
Example 1: Recognizing Singular and Plural Nouns
A
● lphabet: {boy, girl, book, cats, dogs, ...}
● States:
○ q0q_0q0: Start state.
q
○ 1q_1q1: Intermediate state (recognizing singular nouns).
○ q2q_2q2: Intermediate state (recognizing plural nouns).
○ qfq_fqf: Accept state.
Transition Rules:
. F
1 rom q0q_0q0, if a singular noun (e.g., "boy") is read, move to q1q_1q1.
2. From q0q_0q0, if a plural noun (e.g., "cats") is read, move to q2q_2q2.
3. From q1q_1q1or q2q_2q2, move to qfq_fqf.
Example 2: Noun Phrases
Input: "The big dog"
●
● States:
○ q0q_0q0: Start state.
○ q1q_1q1: Recognize determiner (e.g., "The").
○ q2q_2q2: Recognize adjective (e.g., "big").
○ q3q_3q3: Recognize noun (e.g., "dog").
○ qfq_fqf: Accept state.
3. FSA for Verbs
erbs can occur in different tenses, forms (e.g., base, past, present participle), and follow
V
specific syntactic rules. FSAs can help identify these forms.
Example 1: Recognizing Verb Forms
A
● lphabet: {run, runs, running, ran, walk, walked,walking, walks, ...}
● States:
○ q0q_0q0: Start state.
○ q1q_1q1: Base form (e.g., "run").
○ q2q_2q2: Third person singular (e.g., "runs").
○ q3q_3q3: Past form (e.g., "ran").
○ q4q_4q4: Present participle (e.g., "running").
○ qfq_fqf: Accept state.
Transition Rules:
.
1 rom q0q_0q0, if the input is "run" or "walk," move to q1q_1q1.
F
2. From q0q_0q0, if the input is "runs" or "walks," move to q2q_2q2.
3. From q0q_0q0, if the input is "ran" or "walked," move to q3q_3q3.
4. From q0q_0q0, if the input is "running" or "walking," move to q4q_4q4.
5. Any state can transition to qfq_fqfupon recognizing a valid verb form.
4. Combined FSA for Nouns and Verbs
To create an FSA that recognizes nouns and verbs in a sentence:
U
● se separate paths in the automaton for noun and verb recognition.
● Add transitions between noun-related states and verb-related states to model sentences
like"The dog runs."
Example:
Input: "The dog runs"
●
● States:
○ q0q_0q0: Start.
○ q1q_1q1: Recognize determiner ("The").
○ q2q_2q2: Recognize noun ("dog").
○ q3q_3q3: Recognize verb ("runs").
○ qfq_fqf: Accept.
5. Advantages of Using FSAs
F
● SAs are simple and efficient for recognizing patterns like noun and verb forms.
● Useful for designing basic parsers in NLP.
● Help detect valid sequences in input, making them ideal for syntax checking.
owever, FSAs have limitations; they cannot handle nested dependencies or context-sensitive
H
structures. For more complex languages,pushdown automata(with a stack) or other models
are used.
Open-Class Words
pen-class words are content words that convey the primary meaning in a sentence. These
O
categories are flexible, and new words are frequently added to them, especially as language
evolves or new concepts arise.
Characteristics:
. S
1 emantic Role: They carry the main meaning or "content"of a sentence.
2. Word Classes:
○ Nouns: Names of people, places, things, or ideas (e.g.,dog, city, happiness).
○ Verbs: Actions, states, or processes (e.g.,run, eat,believe).
○ Adjectives: Describe or modify nouns (e.g.,blue,happy, fast).
Adverbs: Modify verbs, adjectives, or other adverbs (e.g.,quickly, very, always).
○
3. Dynamic Nature:
○ New words are easily added, such as slang, borrowed terms, or technical jargon
(e.g.,selfie, blog, emoji).
Examples:
● ouns:computer, teacher, innovation
N
● Verbs:explore, innovate, dance
● Adjectives:intelligent, red, efficient
● Adverbs:silently, joyfully, often
Closed-Class Words
losed-class words are functional words that serve grammatical or structural roles in a
C
sentence. Their categories are stable, with few or no new additions over time.
Characteristics:
1. G rammatical Role: They provide structure, connections,and relationships between
content words.
2. Word Classes:
○ Pronouns: Replace nouns (e.g.,he, she, it, they).
○ Prepositions: Indicate relationships in time, space,or direction (e.g.,in, on, at,
under).
○ Conjunctions: Connect clauses or words (e.g.,and,but, because).
○ Determiners: Specify nouns (e.g.,the, a, some, my).
○ Auxiliary Verbs: Help main verbs (e.g.,is, have,can).
○ Particles: Add grammatical meaning (e.g.,toin "togo").
3. Fixed Nature:
○ New words are rarely added to these categories, as they perform fundamental
grammatical functions.
Examples:
● ronouns:you, we, them
P
● Prepositions:with, before, between
● Conjunctions:or, while, although
● Determiners:this, that, several
MODULE 3
challenges of POS Tagging
Types of POS TAGGiNG
Maximum Entropy Model for POS Tagging
heMaximum Entropy (MaxEnt) modelis a probabilisticframework used in Natural Language
T
Processing (NLP) for tasks such asPart-of-Speech(POS) tagging. It is based on the principle
ofmaximum entropy, which states that when estimatingprobabilities, one should prefer the
distribution that maximizes entropy (i.e., is most uniform) while satisfying known constraints.
Introduction to Maximum Entropy Model
● T heMaximum Entropy modelis a general-purpose statisticalmodel that estimates
conditional probabilities.
● In the context ofPOS tagging, the task is to assignthe most likely part-of-speech tag to
each word in a sentence, given its context.
● The model considers all available features of the data and combines them to compute
the probability of a tag for a word without making any unnecessary assumptions about
feature dependencies.
Key Components
1. Feature Functions (fif_ifi):
○ Binary or real-valued functions that represent specific aspects of the data. For
example:
■ Is the current word capitalized?
■ Is the previous word a determiner?
○ Features encode information about the word and its context, which is crucial for
POS tagging.
2. Weights (wiw_iwi):
○ Each feature function has an associated weight that indicates its importance in
predicting a particular tag.
○ The model adjusts these weights during training to fit the observed data.
3. Conditional Probability:
○ The probability of a tag ttt for a word www given the context xxx is calculated as:
P(t∣x)=1Z(x)exp(∑iwifi(t,x))P(t | x) = \frac{1}{Z(x)} \exp\left(\sum_{i} w_i f_i(t,
x)\right)P(t∣x)=Z(x)1exp(i∑wifi(t,x)) Where:
■ Z(x)Z(x)Z(x): Normalization factor ensuring probabilities sum to 1.
Z(x)=∑t′exp(∑iwifi(t′,x))Z(x) = \sum_{t'} \exp\left(\sum_{i} w_i f_i(t',
x)\right)Z(x)=t′∑exp(i∑wifi(t′,x))
4. Training:
○ The model is trained on annotated data using techniques likeGeneralized
Iterative Scaling (GIS)orL-BFGSto find the optimalweights for features.
Example of POS Tagging Using MaxEnt
Sentence:“The cat sleeps.”
Context Features:
For each word, consider:
● he word itself.
T
● Previous word and tag.
● Next word.
● Capitalization, suffixes, prefixes, etc.
Training Data Example:
Word Tag Features
The DET IsDet=true, IsUppercase=false
cat NN Suffix="at", PreviousTag=DET
leep
s VBZ Suffix="s", PreviousTag=NN, IsVerb=true
s
MaxEnt Output:
or each word, compute P(t∣x)P(t | x)P(t∣x) for all possible tags and assign the tag with the
F
highest probability.
Applications of MaxEnt in POS Tagging
1. P art-of-Speech Tagging:
Assign tags like nouns, verbs, adjectives, etc., to words in a sentence.
2. Named Entity Recognition (NER):
Identifying entities such as names, locations, and organizations.
3. Chunking:
Segmenting sentences into syntactic phrases.
4. Dependency Parsing:
Building dependency relations between words in a sentence.
5. Machine Translation:
Handling ambiguities in word usage by leveraging context.
Advantages of MaxEnt for POS Tagging
1. Feature-Rich:
○ Can incorporate diverse and overlapping features.
○ Handles complex dependencies between features effectively.
2. No Independence Assumptions:
○ Unlike models like Naive Bayes, it does not assume feature independence.
3. Robustness:
○ Can model various distributions without overfitting if features are chosen
carefully.
4. Flexibility:
○ Works with various types of features, including categorical, binary, and
real-valued features.
Disadvantages
1. Computational Complexity:
○ Training can be slow due to iterative optimization and the need to compute
normalization factors.
2. Feature Engineering Dependency:
○ Performance heavily relies on the quality and relevance of the chosen features.
3. Data Requirement:
○ Requires a large annotated dataset to estimate weights accurately.
4. No Sequential Modeling:
○ Treats each instance (word) independently, ignoring the sequential nature of
POS tagging. Sequential models like HMM or CRF may perform better in such
cases.
Conditional Random Fields (CRF) in NLP example
Conditional Random Field (CRF)is a statistical modeling method used for structured
A
prediction tasks in natural language processing (NLP). CRFs are specifically designed to model
the relationships between adjacent elements in a sequence while considering the entire input
context. They are widely used for tasks such assequence labeling,POS tagging,named
entity recognition (NER), andshallow parsing.
Inference and parameter learning refer techneo
PCFG
Top-Down and Bottom-Up Parsers in NLP
arsing is the process of analyzing the grammatical structure of a sentence.Top-down
P
andbottom-upparsers are two primary approaches forsyntactic parsing. These
methods differ in how they construct the parse tree and navigate grammar rules.
Top-Down Parser
Definition
top-down parserstarts with the highest-level grammarrule (the root of the parse
A
tree, typically the start symbol SSS) and tries to rewrite it into the input sentence using
grammar rules. It works by breaking down the sentence into smaller components based
on the grammar.
Working Mechanism
1. Initialization:
○ Begin with the start symbol SSS.
2. Expansion:
○ Use grammar rules to expand SSS into its possible derivations.
○ Continue expanding non-terminal symbols (e.g., NP,VPNP, VPNP,VP) until
the leaves of the tree match the input sentence.
3. Matching:
○ Check if the generated parse tree matches the input sentence. If not,
backtrack and try another rule.
4. Output:
○ Return the parse tree if a match is found, or report failure if no valid tree
can be constructed.
Example
Grammar Rules (Context-Free Grammar - CFG):
.
1 →NP VPS \to NP\ VPS→NP VP
S
2. NP→Det NNP \to Det\ NNP→Det N
3. VP→V NPVP \to V\ NPVP→V NP
4. Det→theDet \to \text{the}Det→the
5. N→dogN \to \text{dog}N→dog
6. V→chasedV \to \text{chased}V→chased
Input Sentence:"The dog chased the cat."
Steps:
. S
1 tart with SSS.
2. Apply S→NP VPS \to NP\ VPS→NP VP.
3. Expand NP→Det NNP \to Det\ NNP→Det N and VP→V NPVP \to V\ NPVP→V
NP.
4. Further expand Det→theDet \to \text{the}Det→the, N→dogN \to
\text{dog}N→dog, and V→chasedV \to \text{chased}V→chased.
5. Match these expansions to the input sentence.
Parse Tree:
S
/ \
NP
VP
/ \
/ \
Det
N V NP
|
| | | \
the
dog chased Det N
|
|
the cat
Advantages
P
● redictive:Can predict what structure is expectednext.
● Logical Flow:Mirrors human thought processes in grammaranalysis.
Disadvantages
B
● acktracking:May need to try multiple rules if theinitial ones fail.
● Inefficiency:Can generate irrelevant or invalid intermediatestructures.
● Left Recursion:Struggles with grammars containingleft-recursive rules, e.g.,
A→AαA \to A \alphaA→Aα.
Bottom-Up Parser
Definition
bottom-up parserstarts with the input sentenceand attempts to construct the parse
A
tree by gradually combining smaller components into larger structures, eventually
reaching the start symbol SSS.
Working Mechanism
1. Initialization:
○ Begin with the words in the input sentence.
2. Reduction:
○ Identify components (e.g., Det,NDet, NDet,N) in the input and apply
grammar rules in reverse to combine them into higher-level constituents.
. Validation:
3
○ Check if combining components leads to the start symbol SSS.
4. Output:
○ Return the parse tree if the start symbol SSS is reached, or report failure.
Example
Grammar Rules (Same as Above):
.
1 →NP VPS \to NP\ VPS→NP VP
S
2. NP→Det NNP \to Det\ NNP→Det N
3. VP→V NPVP \to V\ NPVP→V NP
4. Det→theDet \to \text{the}Det→the
5. N→dogN \to \text{dog}N→dog
6. V→chasedV \to \text{chased}V→chased
Input Sentence:"The dog chased the cat."
Steps:
.
1 tart with words: ["The", "dog", "chased", "the", "cat"].
S
2. Combine "The" and "dog" using NP→Det NNP \to Det\ NNP→Det N.
3. Recognize "chased the cat" as VP→V NPVP \to V\ NPVP→V NP.
4. Combine NPNPNP and VPVPVP into S→NP VPS \to NP\ VPS→NP VP.
Parse Tree:(Same as the top-down parse tree above.)
Advantages
E
● fficiency:Avoids unnecessary derivations by onlyworking with valid structures.
● Handles Left Recursion:Not affected by left-recursivegrammar rules.
Disadvantages
N
● on-Predictive:Cannot predict the next componentto process.
● Complexity:May combine components incorrectly, leadingto invalid
intermediate structures.
Comparison: Top-Down vs Bottom-Up
Aspect Top-Down Parser Bottom-Up Parser
Starting Point egins with the start symbol Begins with the input sentence.
B
SSS.
Direction xpands grammar rules
E Combines components upward.
downward.
Focus redicts what should come
P atches what is currently
M
next. present.
Backtracking rone to backtracking for
P imited backtracking (depends
L
invalid paths. on algorithm).
andling Left
H annot handle left-recursive Handles left recursion efficiently.
C
Recursion grammars.
Example Algorithm Recursive Descent Parser. Shift-Reduce Parser.
MODULE 4
1.Semantic Analysis in NLP
emantic analysisin Natural Language Processing (NLP)refers to theprocess of
S
understanding the meaning of words, phrases, and sentences.It aims to extract meaning
from text, resolve ambiguities, identify relationships between words, and produce
machine-readable semantic representations. Unlike syntactic analysis, which deals with the
structure of sentences, semantic analysis focuses on understanding the meaning conveyed by
the language in a particular context.
Key Features of Semantic Analysis
1. Word Sense Disambiguation (WSD)
● D efinition: Identifying which sense of a word is usedin a given context, especially for
words that have multiple meanings.
● Example: The word“bank”can refer to:
○ A financial institution:I deposited money in thebank.
○ The side of a river:The boat docked at the bank ofthe river.
● Challenge: Correctly choosing the appropriate meaningof ambiguous words based on
surrounding context.
2. Named Entity Recognition (NER)
● D efinition:Identifying and categorizing entitiesin text into predefined classessuch
as person names, organizations, locations, dates, etc.
● Example:
○ Sentence:Apple Inc. was founded by Steve Jobs inCupertino in 1976.
○ Entities:Apple Inc.(Organization),Steve Jobs(Person),Cupertino(Location),
1976(Date).
● Challenge: Detecting entities accurately in unstructuredtext, especially when they are
used in less conventional ways.
4. Sentiment Analysis
● D efinition: Determining thesentiment or emotion conveyedin a piece of text
(positive, negative, neutral).
● Example:
○ Sentence:The movie was fantastic!
○ Sentiment: Positive
● C
hallenge: Analyzing subtle variations in sentiment, such as sarcasm, and
contextualizing the sentiment based on the words used.
5. Word Embeddings
● D efinition: Representing words asdense vectors ofreal numbers that capture their
semantic meaning and relationships with other words.These representations are
learned from large corpora.
● Example:
○ In Word2Vec,king - man + woman ≈ queen, illustratinghow word embeddings
capture relationships between words.
● Challenge: Capturing complex, high-dimensional relationshipsand making them
interpretable for downstream tasks.
6. Coreference Resolution
● D efinition: Identifying whichwords in a text referto the same entity, such as
pronouns referring to their antecedents.
● Example:
○ Sentence:John went to the store. He bought some apples.
○ Coreference resolution: "He" refers to "John."
● Challenge: Correctly linking pronouns and noun phrasesin longer, more complex texts.
7. Relationship Extraction
● D efinition:Identifying relationships between entitiesin a text, such as who is related
to whom or what action is taking place.
● Example:
○ Sentence:Bill Gates founded Microsoft in 1975.
○ Relationship extraction:
■ Bill Gates→found→Microsoft
■ Microsoft→founded in→1975
● Challenge: Accurately identifying and categorizingrelationships in unstructured text.
Applications of Semantic Analysis
emantic analysis has a wide range of applications across various NLP tasks. Some of the
S
most prominent applications are:
1. Information Retrieval
● A pplication: Enhances search engines by understandingthe meaning behind queries
and retrieving relevant documents based on semantic content, not just keyword
matching.
● Example: Searching for "best mobile phones" will returnresults that understand the
concept of "best" and "mobile phones" in context.
2. Question Answering Systems
● A pplication: Allows machines to understand and respondaccurately to natural
language questions by interpreting the meaning of the query and extracting the relevant
information.
● Example:Question:Who is the CEO of Tesla?
○ The system would look for the most relevant passage and provide the correct
answer, such as "Elon Musk."
3. Machine Translation
● A pplication: Translates text from one language toanother while preserving the
meaning, context, and relationships between entities.
● Example: Translating"She is going to the market"from English to Spanish ensures that
the correct meaning and grammatical structure are maintained in the target language.
4. Sentiment Analysis
● A pplication: Analyzes social media, reviews, and feedbackto determine customer
sentiments (positive, negative, neutral) towards a product, service, or brand.
● Example: Companies can use sentiment analysis to gaugepublic opinion on their
products by analyzing online reviews and social media posts.
5. Text Summarization
● A pplication: Generates concise and informative summariesof larger texts while
retaining the meaning of the original content.
● Example: A system can automatically summarize a lengthynews article into a short
paragraph that captures the main points.
6. Dialogue Systems and Chatbots
● A pplication: Semantic analysis enables chatbots andvirtual assistants to comprehend
and respond meaningfully to user queries.
● Example: Virtual assistants like Siri or Alexa usesemantic analysis to understand the
meaning behind commands such as "Set a reminder for tomorrow at 9 AM."
8. Text Classification
● A pplication: Categorizes text into predefined classesbased on its content, such as
spam detection, topic categorization, and sentiment categorization.
● Example: Classifying an email as "spam" or "not spam"based on its semantic content.
2.Diff
Aspect Syntactic Ambiguity Lexical Ambiguity
Definition ccurs when a sentence or phrase
O ccurs when a word has multiple
O
can be interpreted in multiple ways meanings or interpretations.
due to its structure or syntax.
Cause rises from sentence structure or the
A rises from words having more
A
arrangement of words. than one meaning.
Example "I saw the man with the telescope." " Bank" (financial institution vs.
riverbank).
Types mbiguity in sentence structure or
A mbiguity in the meaning of a
A
grammatical relations. single word.
Resolution equires parsing the sentence to
R equires understanding the
R
identify the correct syntactic structure. word's meaning in context (e.g.,
disambiguation).
esolution
R yntax tree parsing, dependency
S ord sense disambiguation
W
Techniques parsing. (WSD), contextual analysis.
Impact eads to multiple interpretations
L eads to confusion due to
L
based on how the sentence is multiple meanings of a word.
structured.
ommon NLP
C entence parsing, machine
S amed Entity Recognition (NER),
N
Tasks Affected translation, and question answering. text classification, and machine
translation.
xample of
E " The chicken is ready to eat."(Is the " Bark"(The sound a dog makes
Ambiguity chicken going to eat, or is the chicken vs. the outer covering of a tree).
ready to be eaten?)
3.Demonstrate lexical semantic analysis using an example
exical semanticsfocuses on the study of word meaningsand the relationships between
L
words. In natural language, many words have multiple meanings (polysemy), and lexical
semantic analysis seeks to identify the correct meaning of these words based on context. The
goal is to understand how words convey meaning, their interrelations, and how they are
interpreted in different contexts.
Example of Lexical Semantic Analysis
Sentence:
“He went to the bank to fish.”
● Word "bank": This word has multiple meanings:
1. Financial institution: A place where financial transactionsoccur.
2. Riverbank: The edge of a river or stream.
Analysis Process:
● T okenization: Break the sentence into individual words:["He", "went", "to", "the", "bank",
"to", "fish"].
● Word Sense Disambiguation (WSD): The word “bank” hastwo senses, but the
presence of the word “fish” indicates that the correct sense is theriverbank(side of a
river).
● Contextual Clue: The word “fish” suggests an actionthat typically happens at the side
of a river, not in a financial institution.
● Final Meaning: The word “bank” here refers toriverbank.
Explanation of Lexical Semantic Analysis
exical semantic analysis is the process of analyzing and understanding the meanings of words
L
in a language, particularly focusing on how they are used in context. This involves:
Identifying ambiguitiesin word meanings.
●
● Disambiguatingthese meanings based on the contextin which the word is used.
● Understanding relationshipsbetween words, such assynonyms, antonyms,
hyponyms, and hypernyms.
or example, in the sentence"I went to the bank tofish,"lexical semantics focuses on
F
determining that “bank” in this context refers to the side of a river, not a financial institution. This
involves examining the surrounding context, such as the word “fish,” which serves as a clue for
disambiguation.
Key Aspects of Lexical Semantics
a. Word Sense Disambiguation (WSD)
● D efinition: WSD is the task of identifying the correctmeaning of a word that has multiple
meanings. It is essential in disambiguating words that have more than one possible
interpretation.
● Example: In the sentence"He went to the bank to fish,"the word"bank"has multiple
meanings. WSD helps us identify that in this context,"bank"refers to theriverbank, not
a financial institution.
b. Semantic Role Labeling (SRL)
● D efinition: SRL is the process of identifying theroles that words or phrases play in a
sentence (e.g., agent, theme, goal). It helps determine "who did what to whom" in a
sentence.
● Example: In the sentence"The dog chased the ball,"SRL would identify:
○ Agent: The dog (who performed the action)
○ Theme: The ball (what is being acted upon)
c. Named Entity Recognition (NER)
● D efinition: NER involves identifying and classifyingproper nouns into predefined
categories, such as persons, organizations, locations, dates, etc.
● Example: In the sentence"Apple Inc. was founded bySteve Jobs in Cupertino,"NER
would identify:
○ Apple Inc.(Organization)
○ Steve Jobs(Person)
○ Cupertino(Location)
d. Sentiment Analysis
● D efinition: Sentiment analysis involves determiningthe emotional tone of a piece of
text, such as whether it is positive, negative, or neutral.
● Example: In the sentence"I love this phone!", sentimentanalysis would classify the
sentiment aspositive.
Techniques Used in Lexical Semantics:
a. Distributional Semantics
● D efinition: Distributional semantics is based on theidea that words with similar
meanings tend to appear in similar contexts. It uses word co-occurrence patterns from
large text corpora to understand the meaning of words.
● Example: By examining how often words like "bank"appear in contexts involving "river,"
"fishing," and "water," the model can determine that "bank" likely refers to ariverbankin
the sentence"He went to the bank to fish."
b. Lexical Resources
● D
efinition: Lexical resources are structured databasesthat store the meanings of
words, their relationships, and various semantic properties. Examples include WordNet,
a lexical database that categorizes words and their senses (synonyms, antonyms,
hypernyms).
● E
xample: WordNet might link"bank"(as in a financial institution) with terms like
"money"and"finance", while"bank"(as in a riverbank) might be linked to"river"and
"shore."
c. Deep Learning
● D efinition: Deep learning techniques, particularlyneural networks, are used to model
complex semantic representations of words. Models likeWord2VecorBERTcapture
semantic relationships by learning vector embeddings of words based on large amounts
of text data.
● Example: A deep learning model likeWord2Vecwouldlearn to represent the word
"bank" as a vector that captures both senses of the word: one related tofinanceand the
other related toriverbanks, based on the contextin which the word is used.
d. Semantic Role Labeling Models
● D efinition: These models are used to assign semanticroles (like Agent, Theme, Goal,
etc.) to different parts of a sentence. SRL models typically rely on machine learning
algorithms and large annotated corpora.
● Example: In the sentence"The chef cooked dinner forhis family,"an SRL model would
label:
○ Agent: The chef
○ Action: Cooked
○ Theme: Dinner
○ Goal: For his family
4. Explain in detail Hyponymy,homonymy,polysemy,Synonymy,Antonymy
1. Hyponymy
efinition:
D
Hyponymy refers to a relationship between words where the meaning of one word (the
hyponym) is more specific than the other (the hypernym or superclass). Essentially, a hyponym
is a "type of" the hypernym.
Example:
H
● ypernym:Animal
● Hyponyms:Dog, Cat, Elephant, Bird
Here, "Dog" is a type of "Animal."
Features:
H
● ierarchical: Hyponymy represents a hierarchy (e.g.,animal → dog → poodle).
● Asymmetrical: If A is a hyponym of B, then B is nota hyponym of A.
Applications in NLP:
O
● ntology creation (e.g., WordNet)
● Taxonomic classification
● Question answering systems
2. Homonymy
omonymy refers to words that have the same spelling or pronunciationbut have different,
H
unrelated meanings. Homonyms can be further dividedinto:
● H omophones: Words that sound the same but may differin spelling and meaning.
Example:flour(used in baking) vs.flower(a partof a plant).
● Homographs: Words that are spelled the same but maydiffer in pronunciation and
meaning. Example:lead(to guide) vs.lead(a metal).
Example:
● Bat: An object used in sports or a flying mammal.
Features:
M
● eanings are unrelated.
● Can cause lexical ambiguity.
Applications in NLP:
Disambiguation in speech and text (e.g., Word Sense Disambiguation).
●
● Improving search engines and information retrieval.
3. Polysemy
efinition:
D
Polysemy occurs when asingle word has multiple relatedmeanings. Unlike homonyms, the
meanings of polysemous words are connected through a shared origin or context.
Example:
● Bank:
1. A financial institution.
2. The side of a river.
ere, the meanings are related: the financial institution might have originated from "riverbanks"
H
as early trading places.
Features:
M
● eanings are related and share a conceptual base.
● A word can have multiple senses within different contexts.
Applications in NLP:
W
● ord Sense Disambiguation
● Semantic analysis and knowledge graphs
● Language translation
4. Synonymy
efinition:
D
Synonymy is the relationship betweenwords that havethe same or nearly the same
meaning in a particular context.Synonyms are context-sensitive;they are not always
interchangeable.
Example:
B
● igandLarge
● HappyandJoyful
Features:
● S ynonyms are not always perfect substitutes.
Example:BigandLargeare synonyms, but we say "Bigmistake," not "Large mistake."
● Synonymy is important for paraphrasing and natural language understanding.
Applications in NLP:
T
● hesaurus generation (e.g., WordNet)
● Text summarization
● Paraphrase detection and generation
5. Antonymy
efinition:
D
Antonymy refers to words that have opposite meanings.Antonyms can be:
● G radable Antonyms: Words that represent opposite endsof a spectrum (e.g.,hot↔
cold).
● Complementary Antonyms: Words where the presence ofone implies the absence of
the other (e.g.,alive↔dead).
● Relational Antonyms: Words that describe opposite relationships (e.g.,buy↔sell).
Example:
G
● radable:Tall↔Short
● Complementary:True↔False
● Relational:Teacher↔Student
Features:
A
● ntonyms highlight contrasts.
● The relationship is context-dependent.
Applications in NLP:
S
● entiment analysis (e.g.,goodvs.badsentiment).
● Opposite meaning detection in semantic tasks.
4.WSD
5.Yarowsky
Advantages of the Yarowsky Algorithm
1. Semi-Supervised:
○ Requires only a small amount of labeled data, making it efficient for large-scale
tasks with minimal annotation.
2. Bootstrapping:
○ Automatically expands the training set by leveraging unlabeled data, reducing
manual annotation effort.
3. Effective Assumptions:
○ The "one sense per collocation" and "one sense per discourse" principles often
hold true in natural language, improving accuracy.
4. Adaptability:
○ Can be applied to various WSD tasks and other semi-supervised learning
problems.
Limitations of the Yarowsky Algorithm
1. Dependency on Initial Seeds:
○ The quality of the initial labeled examples significantly impacts the algorithm's
performance.
2. Error Propagation:
○ Incorrect high-confidence predictions in early iterations can propagate and
degrade performance.
3. Limited Scope:
○ Works best for words with clear collocational patterns; may struggle with
ambiguous words that lack strong contextual clues.
4. Binary Focus:
○ Originally designed for binary sense distinctions. Handling multiple senses
requires additional complexity.
Applications
W
● ord Sense Disambiguation: Resolving ambiguity inword meanings.
● Information Retrieval: Improving search results by understanding word senses.
● Machine Translation: Disambiguating words to choose the correct translation.
MODULE 5
MODULE 6
1.What is Machine Translation (MT)?
achine Translation (MT) is the process of using computational techniques to automatically
M
translate text or speech from one language to another. It aims to bridge language barriers by
enabling effective communication across languages. MT systems attempt to preserve the
meaning, structure, and fluency of the original text in the target language.
Approaches to Machine Translation
here are several approaches to machine translation, each with its methodologies, strengths,
T
and limitations. These include:
1. Rule-Based Machine Translation (RBMT)
● Overview:
○ RBMT relies on linguistic rules and dictionaries to translate text.
○ It uses grammatical, syntactic, and semantic rules to map text from the source
language to the target language.
○ Requires extensive linguistic knowledge of both languages.
● Components:
○ Lexicon: Dictionary of words and their translations.
○ Syntactic Rules: Define sentence structures in bothlanguages.
○ Semantic Rules: Ensure the meaning of the sentenceis preserved.
● Strengths:
○ High interpretability of translations.
○ Works well for languages with rich linguistic resources.
○ Effective for domain-specific translation with carefully designed rules.
● Limitations:
○ Requires significant manual effort to build rules.
○ Limited scalability for new languages or domains.
○ Struggles with idiomatic expressions and complex sentence structures.
2. Statistical Machine Translation (SMT)
● Overview:
○ SMT is a data-driven approach that uses statistical models to generate
translations based on probabilities.
○ It relies on bilingual parallel corpora to learn mappings between source and
target languages.
● Key Concepts:
○ T ranslation Model: Captures probabilities of translating words or phrases from
the source language to the target language.
○ Language Model: Ensures fluency by modeling the likelihoodof word
sequences in the target language.
○ Decoder: Combines the translation and language models to generate the most
probable target sentence.
● Training Process:
○ Use a parallel corpus to extract word alignments and translation probabilities.
○ Train an n-gram-based language model on target language text.
○ Optimize model parameters to improve translation accuracy.
Strengths:
●
○ Requires no explicit linguistic rules.
○ Adaptable to many languages with sufficient training data.
○ Transparent in terms of probabilities and alignments.
● Limitations:
○ Highly dependent on the availability and quality of bilingual corpora.
○ Struggles with rare words, idiomatic expressions, and long-range dependencies.
○ Outputs may lack fluency and coherence.
3. Example-Based Machine Translation (EBMT)
● Overview:
○ EBMT translates by reusing examples of translations stored in a database.
○ It relies on the principle that similar input sentences often translate similarly.
● Process:
○ Retrieve similar examples from a translation database.
○ Adapt the examples to create a translation for the input text.
○ Combine fragments of examples to handle complex inputs.
● Strengths:
○ Does not require linguistic rules or probabilistic models.
○ Useful for specific domains with repetitive patterns.
● Limitations:
○ Dependent on the quality and coverage of the example database.
○ May struggle with novel or diverse inputs.
4. Neural Machine Translation (NMT)
● Overview:
○ NMT uses deep learning models to perform end-to-end translation.
○ It learns to map sentences from the source language to the target language
using large datasets.
● Key Features:
○ Encoder-Decoder Architecture:
■ The encoder processes the source sentence into a fixed-size vector
representation.
■ The decoder generates the target sentence from this representation.
○ Attention Mechanisms: Allow the model to focus on relevant parts of the source
sentence during translation.
○ Transformers: State-of-the-art architecture in NMTthat uses self-attention
mechanisms for better context understanding.
● Training:
○ Requires large amounts of parallel corpora for supervised training.
○ Optimization techniques like stochastic gradient descent (SGD) are used to
minimize translation errors.
Strengths:
●
○ Produces more fluent and coherent translations than SMT.
○ Handles long-range dependencies and context better.
○ Can generalize well to unseen phrases with sufficient training data.
● Limitations:
○ Requires substantial computational resources.
○ Dependent on the quality and quantity of training data.
○ May struggle with low-resource languages.
5. Hybrid Machine Translation
● Overview:
○ Combines elements of RBMT, SMT, and NMT to leverage their respective
strengths.
○ For example, a system may use rules for grammatical structure and SMT or NMT
for lexical choice.
● Strengths:
○ Provides a balance between rule-based precision and statistical flexibility.
○ Can perform well in low-resource settings by using linguistic rules to supplement
sparse data.
● Limitations:
○ Integration of different approaches can be complex.
○ Requires careful design to avoid conflicts between components.
2.Working of Machine translation system
he working of aMachine Translation (MT)system involves a sequence of steps that process
T
input text in the source language and generate its equivalent in the target language. While
specific implementations differ depending on the MT approach (Rule-Based, Statistical, Neural,
etc.), the general workflow includes:
1. Input Preprocessing
Before translation begins, the input text is prepared for processing:
T
● okenization: Splits the input text into smaller unitslike words, subwords, or characters.
● Lowercasing and Normalization: Converts all text to lowercase and standardizes text
formats (e.g., removing diacritics, handling contractions).
● Language-Specific Processing: Includes stemming, lemmatization,or handling
language-specific grammar rules, if necessary.
● Handling Out-of-Vocabulary Words: For unknown words,subword tokenization (e.g.,
Byte Pair Encoding) or placeholders might be used.
2. Analysis of Source Language
The system analyzes the grammatical and semantic structure of the source sentence:
S
● yntax Analysis: Identifies the sentence structure(e.g., subject, verb, object).
● Morphological Analysis: Understands word forms andtheir grammatical functions
(e.g., verb tense, pluralization).
● Semantic Analysis: Ensures the meaning of the inputsentence is captured accurately.
3. Translation Process
The method used to translate depends on the MT approach:
A. Rule-Based MT (RBMT)
1. Source Language Parsing:
○ Breaks down the source text into linguistic components using rules.
2. Transfer Rules:
○ Maps the source language structure to the target language structure based on a
predefined set of rules.
3. Synthesis:
○ Constructs the target sentence using dictionaries and grammar rules of the target
language.
B. Statistical MT (SMT)
1. Word or Phrase Alignment:
○ The system determines which words or phrases in the source language
correspond to those in the target language based on probabilities learned from
bilingual corpora.
2. Translation Model:
○ Generates translations for words/phrases based on the most likely mappings.
3. Language Model:
○ Ensures the output in the target language is grammatically correct by modeling
sequences of words.
4. Decoding:
○ Combines the translation model and language model to generate the most
probable target sentence.
C. Neural MT (NMT)
1. Encoding:
○ The source sentence is converted into a continuous vector representation by the
encoder(a neural network like RNN, LSTM, or Transformer).
○ This representation captures the semantic and syntactic information of the
sentence.
2. Attention Mechanism:
○ Helps the system focus on relevant parts of the source sentence while
generating each word in the target sentence.
3. Decoding:
○ Thedecodergenerates the target sentence word byword based on the encoded
representation and attention context.
4. Post-Processing:
○ The output is detokenized, capitalized, or otherwise normalized into readable
text.
4. Reordering
Languages often differ in word order. For example:
E
● nglish: "I read a book."
● Japanese: "Watashi wa hon o yomimasu." (Literal: "I a book read.") The system must
adjust the word order in the target language to preserve meaning and grammatical
correctness.
5. Output Postprocessing
he generated translation is processed to ensure it is well-formed and meets the output
T
requirements:
D
● etokenization: Combines tokens into complete wordsand sentences.
● Grammatical Adjustment: Corrects minor errors in verbtense, agreement, or
punctuation.
● Handling Unknown Words: Attempts to infer or leave placeholders for untranslated
terms.
6. Quality Evaluation
The translated text is evaluated using metrics like:
● B LEU (Bilingual Evaluation Understudy): Measures n-gramoverlap between the
machine-generated translation and reference translations.
● TER (Translation Error Rate): Measures the numberof edits required to make the
translation match the reference.
● Human Evaluation: Judges fluency, adequacy, and grammaticalcorrectness.
Example: Translation Workflow
Input Sentence:
Source Language: "The weather is nice today."
Output (English to French using Neural MT):
1. Preprocessing:
○ Tokenization: ["The", "weather", "is", "nice", "today"]
○ Normalization: ["the", "weather", "is", "nice", "today"]
2. Encoding:
○ Sentence encoded as a numerical vector: [0.1, 0.5, ..., 0.9]
3. Attention Mechanism:
○ Focuses on relevant words (e.g., "weather" aligns with "météo").
4. Decoding:
○ Word-by-word generation: "Le", "temps", "est", "agréable", "aujourd'hui."
5. Postprocessing:
○ Detokenization: "Le temps est agréable aujourd'hui."
Challenges in MT Systems
1. A mbiguity: Words or phrases with multiple meanings(e.g., "bank" as a financial
institution vs. riverbank).
2. Idiomatic Expressions: Phrases like "kick the bucket"cannot be translated literally.
3. Low-Resource Languages: Insufficient data for training models in some languages.
4. Context Dependency: Translation may require understandingthe broader document
context, not just isolated sentences.
3.Information Retrieval (IR) vs. Information Extraction (IE)
Information Retrieval (IR)andInformation Extraction(IE)are two distinct processes in
Natural Language Processing (NLP), both aimed at processingand leveraging information.
However, their purposes, techniques, and outputs differ significantly. Here is a detailed
comparison:
1. Information Retrieval (IR)
efinition:
D
Information Retrieval focuses on finding and retrieving relevant documents or pieces of
information from large collections of unstructured or semi-structured data (like text, images, or
videos). It does not alter the content but ranks and presents the most relevant items to a user
query.
Key Characteristics:
O
● perates at the document level.
● Concerned with findingrelevantdata, not understandingor extracting specific details
from it.
● Outputs a ranked list of documents or passages related to the user’s query.
Typical Workflow:
1. Indexing:
○ Convert the raw text into a searchable structure (e.g., inverted index).
○ Tokenize text and store term-document relationships for fast lookup.
2. Query Processing:
○ Analyze the user’s query to understand the search intent.
○ Tokenize and normalize the query (e.g., stemming, stopword removal).
3. Matching:
○ C ompare the query against the indexed documents using various similarity
measures (e.g., cosine similarity, TF-IDF).
4. Ranking:
○ Rank documents based on relevance to the query using scoring algorithms (e.g.,
BM25).
. Result Presentation:
5
○ Return the most relevant results to the user, often with snippets.
Examples:
S
● earch engines like Google or Bing.
● Library catalogs or document repositories.
● Question-answering systems returning relevant documents.
Key Techniques:
● B oolean Retrieval: Uses Boolean logic (AND, OR, NOT)for matching queries to
documents.
● Vector Space Models: Represents documents and queriesas vectors and calculates
their similarity.
● Probabilistic Models: Ranks documents based on theprobability of relevance to the
query (e.g., BM25).
● Latent Semantic Analysis (LSA): Captures relationshipsbetween terms and concepts
for better matching.
● Deep Learning in IR: Neural approaches like BERT-basedmodels improve query
understanding and ranking.
2. Information Extraction (IE)
efinition:
D
Information Extraction is the process of identifying and extracting structured, meaningful data
from unstructured text. It focuses on understanding the content and deriving specific entities,
relationships, and facts.
Key Characteristics:
● O perates at a finer granularity, often extracting specific pieces of data (e.g., names,
dates, or relationships).
● Aims to transform unstructured data into structured formats (e.g., database entries,
knowledge graphs).
● Outputs structured information like tables, JSON objects, or semantic triples.
Typical Workflow:
1. Preprocessing:
○ Clean and normalize the text (e.g., tokenization, POS tagging).
○ Remove noise like irrelevant characters or stopwords.
2. Entity Recognition:
○ Identify named entities (e.g., people, locations, organizations) using Named
Entity Recognition (NER).
3. Relation Extraction:
○ Identify relationships between entities (e.g., "John works at Microsoft").
4. Event Extraction:
○ Identify events and their attributes (e.g., "A meeting was held on [date] between
[person1] and [person2]").
5. Template Filling:
○ Populate pre-defined templates or databases with extracted information.
6. Output:
○ Present the extracted data in structured formats like CSV, JSON, or RDF.
Examples:
E
● xtracting contact details (email, phone) from resumes.
● Summarizing and extracting financial data from annual reports.
● Populating knowledge graphs with facts from Wikipedia.
Key Techniques:
● amed Entity Recognition (NER): Identifies entitieslike names, dates, and locations.
N
● Dependency Parsing: Analyzes sentence structure toextract relationships.
● Rule-Based Systems: Uses regular expressions and templatesfor extraction.
● Machine Learning-Based IE:
○ Supervised models trained on labeled data.
○ Techniques like Conditional Random Fields (CRFs) for sequential data
extraction.
Deep Learning for IE:
●
○ Transformer-based models like BERT fine-tuned for entity and relation extraction
tasks.
Key Differences Between IR and IE
Aspect Information Retrieval (IR) Information Extraction (IE)
Goal etrieve relevant documents
R xtract specific structured information
E
based on a query. from unstructured text.
Granularity perates at the document or
O perates at the word, sentence, or
O
passage level. phrase level.
Input arge text corpora, documents, or
L Individual documents or smaller datasets.
data repositories.
Output anked list of relevant documents
R tructured data (e.g., tables, knowledge
S
or snippets. graphs).
Focus inding and ranking relevant
F Understanding and structuring content.
content.
Techniques T
F-IDF, BM25, BERT for query ER, relation extraction, dependency
N
ranking. parsing.
Examples earch engines,
S esume parsers, knowledge graph
R
question-answering systems. creation, data extraction.
Applications of IR and IE
Applications of Information Retrieval:
.
1 eb Search: Search engines like Google retrieve relevantweb pages.
W
2. Document Search: Finding relevant research papers,articles, or legal documents.
3. Enterprise Search: Searching internal organizationalrepositories.
4. Question-Answering Systems: Retrieving relevant documentsfor answering user
queries.
Applications of Information Extraction:
1. K nowledge Graph Construction: Extracting facts topopulate knowledge bases like
Google Knowledge Graph.
2. Summarization: Extracting key entities and eventsfor text summarization.
3. Business Intelligence: Extracting trends, financialfigures, or insights from reports.
4. Social Media Analysis: Identifying sentiments, trends,or events from social media
posts.
You might also like
- Ambiguity in Natural Language ProcessingNo ratings yetAmbiguity in Natural Language Processing9 pages
- ACFrOgBKMtkrKQXYgwzYfGAQxQ0GJjQ4MloahBs6vi5pwqo xRZUN6IRgh8lAAyR2U7sguAn6becvxh174Y RYo84nZ3K9mm OlN3Q JrDvd18FxMzMkCBuxruzd1tH0C6XqndKXsCSXuwHIWVT7olg5FKOstIhFYq-Kh6hMBgNo ratings yetACFrOgBKMtkrKQXYgwzYfGAQxQ0GJjQ4MloahBs6vi5pwqo xRZUN6IRgh8lAAyR2U7sguAn6becvxh174Y RYo84nZ3K9mm OlN3Q JrDvd18FxMzMkCBuxruzd1tH0C6XqndKXsCSXuwHIWVT7olg5FKOstIhFYq-Kh6hMBg32 pages
- Unit V Intelligence and Applications: Morphological Analysis/Lexical AnalysisNo ratings yetUnit V Intelligence and Applications: Morphological Analysis/Lexical Analysis30 pages
- Linguistics Essentials: Instructor: Rada Mihalcea Taught by J. Hajic at Johns Hopkins UniversityNo ratings yetLinguistics Essentials: Instructor: Rada Mihalcea Taught by J. Hajic at Johns Hopkins University46 pages
- NLP Exam: WordNet, Ambiguity, MorphemesNo ratings yetNLP Exam: WordNet, Ambiguity, Morphemes25 pages
- Semantic Concepts and Linguistic TheoriesNo ratings yetSemantic Concepts and Linguistic Theories4 pages
- Chapter 7 - Communication Perceving and ActingNo ratings yetChapter 7 - Communication Perceving and Acting21 pages
- The Future of Automotive Manufacturing - Integrating AI... For Next-Gen Automatic CarsNo ratings yetThe Future of Automotive Manufacturing - Integrating AI... For Next-Gen Automatic Cars9 pages
- Design and Manufacturing of Carbon Fiber Composite Drive Shaft As An Alternative To Conventional Steel Drive ShaftNo ratings yetDesign and Manufacturing of Carbon Fiber Composite Drive Shaft As An Alternative To Conventional Steel Drive Shaft10 pages
- Lance Design For Argon Bubbling in Molten MetalNo ratings yetLance Design For Argon Bubbling in Molten Metal12 pages
- Falke Talk - The Falke 80 - 90 Serial No Database - 03No ratings yetFalke Talk - The Falke 80 - 90 Serial No Database - 035 pages
- The Yellow World How Fighting For My Life Taught Me How To Live Espinosa Albert DownloadNo ratings yetThe Yellow World How Fighting For My Life Taught Me How To Live Espinosa Albert Download35 pages
- Technical Vocational Education: Quarter 1-Week4-Module 4No ratings yetTechnical Vocational Education: Quarter 1-Week4-Module 420 pages
- Hull For: Aerodynamic Design HASPA LTA OptimizationNo ratings yetHull For: Aerodynamic Design HASPA LTA Optimization5 pages
- Colour Dilution Alopecia in Doberman Pinschers With Blue or Fawn Coat Colours - A Study On The Incidence and Histopathology of This DiNo ratings yetColour Dilution Alopecia in Doberman Pinschers With Blue or Fawn Coat Colours - A Study On The Incidence and Histopathology of This Di10 pages
- An Islamic Perspective On Public AdminisNo ratings yetAn Islamic Perspective On Public Adminis4 pages