Semantics: Lexical Semantics
Pawan Goyal
CSE, IIT Kharagpur
October 17, 2014
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
1 / 29
Semantics
What is Semantics?
The study of meaning: Relation between symbols and their denotata.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
2 / 29
Computational Semantics
Computational Semantics
The study of how to automate the process of constructing and reasoning with
meaning representations of natural language expressions.
Methods in Computational Semantics generally fall in two categories:
Formal Semantics: Construction of precise mathematical models of the
relations between expressions in a natural language and the world.
John chases a bat x[bat(x) chase(john, x)]
Distributional Semantics: The study of statistical patterns of human
word usage to extract semantics.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
3 / 29
Lexical Semantics
Definition
Lexical semantics is concerned with the systematic meaning related
connections among lexical items, and the internal meaning-related structure of
individual lexical items.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
4 / 29
Lexical Semantics
Definition
Lexical semantics is concerned with the systematic meaning related
connections among lexical items, and the internal meaning-related structure of
individual lexical items.
To identify the semantics of lexical items, we need to focus on the notion of
lexeme, an individual entry in the lexicon.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
4 / 29
Lexical Semantics
Definition
Lexical semantics is concerned with the systematic meaning related
connections among lexical items, and the internal meaning-related structure of
individual lexical items.
To identify the semantics of lexical items, we need to focus on the notion of
lexeme, an individual entry in the lexicon.
What is a lexeme?
Lexeme should be thought of as a pairing of a particular orthographic and
phonological form with some sort of symbolic meaning representation.
Orthographic form, and phonological form refer to the appropriate form
part of a lexeme
Sense refers to a lexemes meaning counterpart.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
4 / 29
Example
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
5 / 29
Example: meaning related facts?
Definitions from the American Heritage Dictionary (Morris, 1985)
right adj. located near the right hand esp. being on the right when facing
the same direction as the observer
left adj. located near to this side of the body than the right
red n. the color of blood or a ruby
blood n. the red liquid that circulates in the heart, arteries and veins of
animals
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
6 / 29
Example: meaning related facts?
Definitions from the American Heritage Dictionary (Morris, 1985)
right adj. located near the right hand esp. being on the right when facing
the same direction as the observer
left adj. located near to this side of the body than the right
red n. the color of blood or a ruby
blood n. the red liquid that circulates in the heart, arteries and veins of
animals
The entries are description of lexemes in terms of other lexemes
Definitions make it clear that right and left are similar kind of lexemes that
stand in some kind of alternation, or opposition, to one another
We can glean that red is a color, it can be applied to both blood and
rubies, and that blood is a liquid.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
6 / 29
Relations between word meanings
Homonymy
Polysemy
Synonymy
Antonymy
Hypernymy
Hyponymy
Meronymy
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
7 / 29
Homonymy
Definition
Homonymy is defined as a relation that holds between words that have the
same form with unrelated meanings.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
8 / 29
Homonymy
Definition
Homonymy is defined as a relation that holds between words that have the
same form with unrelated meanings.
Examples
Bat (wooden stick-like thing) vs Bat (flying mammal thing)
Bank (financial institution) vs Bank (riverside)
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
8 / 29
Homonymy
Definition
Homonymy is defined as a relation that holds between words that have the
same form with unrelated meanings.
Examples
Bat (wooden stick-like thing) vs Bat (flying mammal thing)
Bank (financial institution) vs Bank (riverside)
homophones and homographs
homophones are the words with the same pronunciation but different
spellings.
write vs right
piece vs peace
homographs are the lexemes with the same orthographic form but different
meaning. Ex: bass
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
8 / 29
Problems for NLP applications
Text-to-Speech
Same orthographic form but different phonological form
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
9 / 29
Problems for NLP applications
Text-to-Speech
Same orthographic form but different phonological form
Information Retrieval
Different meaning but same orthographic form
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
9 / 29
Problems for NLP applications
Text-to-Speech
Same orthographic form but different phonological form
Information Retrieval
Different meaning but same orthographic form
Speech Recognition
to, two, too
Perfect homonyms are also problematic
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
9 / 29
Polysemy
Multiple related meanings within a single lexeme.
The bank was constructed in 1875 out of local red brick.
I withdrew the money from the bank.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
10 / 29
Polysemy
Multiple related meanings within a single lexeme.
The bank was constructed in 1875 out of local red brick.
I withdrew the money from the bank.
Are those the same sense?
Sense 1: The building belonging to a financial institution
Sense 2: A financial institution
Another example
Heavy snow caused the roof of the school to collapse.
The school hired more teachers this year than ever before.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
10 / 29
Polysemy: multiple related meanings
Often, the relationships are systematic
E.g., building vs. organization
school, university, hospital, church, supermarket
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
11 / 29
Polysemy: multiple related meanings
Often, the relationships are systematic
E.g., building vs. organization
school, university, hospital, church, supermarket
More examples:
Author (Jane Austen wrote Emma) Works of Author (I really love Jane
Austen)
Animal (The chicken was domesticated in Asia) Meat (The chicken
was overcooked)
Tree (Plums have beautiful blossoms) Fruit (I ate a preserved plum
yesterday)
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
11 / 29
Polysemy: multiple related meanings
Zeugma test
Which of these flights serve breakfast?
Does Midwest Express serve Philadelphia?
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
12 / 29
Polysemy: multiple related meanings
Zeugma test
Which of these flights serve breakfast?
Does Midwest Express serve Philadelphia?
*Does Midwest Express serve breakfast and San Jose?
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
12 / 29
Polysemy: multiple related meanings
Zeugma test
Which of these flights serve breakfast?
Does Midwest Express serve Philadelphia?
*Does Midwest Express serve breakfast and San Jose?
Since it sounds weird, we say that these are two different senses of serve.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
12 / 29
Synonymy
Words that have the same meaning in some or all contexts.
filbert / hazelnut
couch / sofa
big / large
automobile / car
vomit / throw up
water / H2 O
Two lexemes are synonyms if they can be successfully substituted for each
other in all situations.
If so they have the same propositional meaning.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
13 / 29
Synonymy: A relation between senses
Consider the words big and large.
Are they synonyms?
How big is that plane?
Would I be flying on a large or small plane?
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
14 / 29
Synonymy: A relation between senses
Consider the words big and large.
Are they synonyms?
How big is that plane?
Would I be flying on a large or small plane?
How about here?
Miss Nelson, for instance, became a kind of big sister to Benjamin.
*Miss Nelson, for instance, became a kind of large sister to Benjamin.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
14 / 29
Synonymy: A relation between senses
Consider the words big and large.
Are they synonyms?
How big is that plane?
Would I be flying on a large or small plane?
How about here?
Miss Nelson, for instance, became a kind of big sister to Benjamin.
*Miss Nelson, for instance, became a kind of large sister to Benjamin.
Why?
big has a sense that means being older, or grown up
large lacks this sense
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
14 / 29
Synonyms
Shades of meaning
What is the cheapest first class fare?
*What is the cheapest first class price?
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
15 / 29
Synonyms
Shades of meaning
What is the cheapest first class fare?
*What is the cheapest first class price?
Collocational constraints
We frustate em and frustate em, and pretty soon they make a big
mistake.
*We frustate em and frustate em, and pretty soon they make a large
mistake.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
15 / 29
Antonyms
Senses that are opposites with respect to one feature of their meaning
Otherwise, they are similar!
I
I
I
I
I
dark / light
short / long
hot / cold
up / down
in / out
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
16 / 29
Antonyms
Senses that are opposites with respect to one feature of their meaning
Otherwise, they are similar!
I
I
I
I
I
dark / light
short / long
hot / cold
up / down
in / out
More formally: antonyms can
define a binary opposition or at opposite ends of a scale (long/short,
fast/slow)
Be reversives: rise/fall
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
16 / 29
Hyponymy and Hypernymy
Hyponymy
One sense is a hyponym of another if the first sense is more specific, denoting
a subclass of the other
car is a hyponym of vehicle
dog is a hyponym of animal
mango is a hyponym of fruit
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
17 / 29
Hyponymy and Hypernymy
Hyponymy
One sense is a hyponym of another if the first sense is more specific, denoting
a subclass of the other
car is a hyponym of vehicle
dog is a hyponym of animal
mango is a hyponym of fruit
Hypernymy
Conversely
vehicle is a hypernym/superordinate of car
animal is a hypernym of dog
fruit is a hypernym of mango
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
17 / 29
Hyponymy more formally
Entailment
Sense A is a hyponym of sense B if being an A entails being a B.
Ex: dog, animal
Transitivity
A hypo B and B hypo C entails A hypo C
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
18 / 29
Meronyms and holonyms
Definition
Meronymy: an asymmetric, transitive relation between senses.
X is a meronym of Y if it denotes a part of Y .
The inverse relation is holonymy.
meronym
porch
wheel
leg
nose
Pawan Goyal (IIT Kharagpur)
holonym
house
car
chair
face
Semantics: Lexical Semantics
October 17, 2014
19 / 29
WordNet
A hierarchically organized lexical database
A machine-readable thesaurus, and aspects of a dictionary
Versions for other languages are under development
part of speech
noun
verb
adjective
adverb
Pawan Goyal (IIT Kharagpur)
no. synsets
82,115
13,767
18,156
3,621
Semantics: Lexical Semantics
October 17, 2014
20 / 29
Synsets in WordNet
A synset is a set of synonyms representing a sense
Example: chump as a noun to mean a person who is gullible and easy to
take advantage of
Each of these senses share this same gloss.
For WordNet, the meaning of this sense of chump is this list.
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
21 / 29
lemma vs. synsets
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
22 / 29
Wordnet noun and verb relations
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
23 / 29
WordNet Hierarchies
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
24 / 29
Word Similarity
Synonymy is a binary relation
I
Two words are either synonymous or not
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
25 / 29
Word Similarity
Synonymy is a binary relation
I
Two words are either synonymous or not
We want a looser metric
I
I
Word similarity or
Word distance
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
25 / 29
Word Similarity
Synonymy is a binary relation
I
Two words are either synonymous or not
We want a looser metric
I
I
Word similarity or
Word distance
Two words are more similar
I
If they share more features of meaning
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
25 / 29
Word Similarity
Synonymy is a binary relation
I
Two words are either synonymous or not
We want a looser metric
I
I
Word similarity or
Word distance
Two words are more similar
I
If they share more features of meaning
Actually these are really relations between senses:
I
I
Instead of saying bank is like fund
We say
F
F
Bank1 is similar to fund3
Bank2 is similar to slope5
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
25 / 29
Word Similarity
Synonymy is a binary relation
I
Two words are either synonymous or not
We want a looser metric
I
I
Word similarity or
Word distance
Two words are more similar
I
If they share more features of meaning
Actually these are really relations between senses:
I
I
Instead of saying bank is like fund
We say
F
F
Bank1 is similar to fund3
Bank2 is similar to slope5
We will compute similarity over both words and senses
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
25 / 29
Two classes of algorithms
Distributional algorithms
By comparing words based on their distributional context in the corpora
Thesaurus-based algorithms
Based on whether words are nearby in WordNet
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
26 / 29
Thesaurus-based Word Similarity
We could use anything in the thesaurus:
I
I
Meronymy, hyponymy, troponymy
Glosses and example sentences
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
27 / 29
Thesaurus-based Word Similarity
We could use anything in the thesaurus:
I
I
Meronymy, hyponymy, troponymy
Glosses and example sentences
In practice, thesaurus-based methods usually use:
I
I
the is-a/subsumption/hypernymy hierarchy
and sometimes the glosses too
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
27 / 29
Thesaurus-based Word Similarity
We could use anything in the thesaurus:
I
I
Meronymy, hyponymy, troponymy
Glosses and example sentences
In practice, thesaurus-based methods usually use:
I
I
the is-a/subsumption/hypernymy hierarchy
and sometimes the glosses too
Word similarity vs. word relatedness
I
I
Similar words are near-synonyms
Related words could be related any way
F
F
car, gasoline : related, but nor similar
car, bicycle: similar
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
27 / 29
Path-based similarity
Basic Idea
Two words are similar if they are nearby in the hypernym graph
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
28 / 29
Path-based similarity
Basic Idea
Two words are similar if they are nearby in the hypernym graph
pathlen(c1 , c2 ) = number of edges in shortest path (in hypernym graph)
between senses c1 and c2
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
28 / 29
Path-based similarity
Basic Idea
Two words are similar if they are nearby in the hypernym graph
pathlen(c1 , c2 ) = number of edges in shortest path (in hypernym graph)
between senses c1 and c2
simpath (c1 , c2 ) =
Pawan Goyal (IIT Kharagpur)
1
1+pathlen(c1 ,c2 )
Semantics: Lexical Semantics
October 17, 2014
28 / 29
Path-based similarity
Basic Idea
Two words are similar if they are nearby in the hypernym graph
pathlen(c1 , c2 ) = number of edges in shortest path (in hypernym graph)
between senses c1 and c2
simpath (c1 , c2 ) =
1
1+pathlen(c1 ,c2 )
sim(w1 , w2 ) = maxc1 senses(w1 ),c2 senses(w2 ) sim(c1 , c2 )
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
28 / 29
Shortest path in the hierarchy
Pawan Goyal (IIT Kharagpur)
Semantics: Lexical Semantics
October 17, 2014
29 / 29
