lexical semantics ii - umass csdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind...
TRANSCRIPT
![Page 1: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/1.jpg)
Lexical Semantics II
Introduction to Natural Language ProcessingComputer Science 585—Fall 2009
University of Massachusetts Amherst
David Smith
1
![Page 2: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/2.jpg)
Topic Models
Unsupervised Models of
Word Co-occurrences
2
![Page 3: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/3.jpg)
A Probabilistic Approach
• Define a probabilistic generative
model for documents.
• Learn the parameters of this
model by fitting them to the data
and a prior.
3
![Page 4: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/4.jpg)
Clustering words into topics with
Latent Dirichlet Allocation[Blei, Ng, Jordan 2003]
Sample a distributionover topics, !
For each document:
Sample a topic, z
For each word in doc
Sample a word
from the topic, w
Example:
70% Iraq war
30% US election
Iraq war
“bombing”
Generative
Process:
4
![Page 5: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/5.jpg)
STORY
STORIES
TELL
CHARACTER
CHARACTERS
AUTHOR
READ
TOLD
SETTING
TALES
PLOT
TELLING
SHORT
FICTION
ACTION
TRUE
EVENTS
TELLS
TALE
NOVEL
MIND
WORLD
DREAM
DREAMS
THOUGHT
IMAGINATION
MOMENT
THOUGHTS
OWN
REAL
LIFE
IMAGINE
SENSE
CONSCIOUSNESS
STRANGE
FEELING
WHOLE
BEING
MIGHT
HOPE
WATER
FISH
SEA
SWIM
SWIMMING
POOL
LIKE
SHELL
SHARK
TANK
SHELLS
SHARKS
DIVING
DOLPHINS
SWAM
LONG
SEAL
DIVE
DOLPHIN
UNDERWATER
DISEASE
BACTERIA
DISEASES
GERMS
FEVER
CAUSE
CAUSED
SPREAD
VIRUSES
INFECTION
VIRUS
MICROORGANISMS
PERSON
INFECTIOUS
COMMON
CAUSING
SMALLPOX
BODY
INFECTIONS
CERTAIN
Example topics
induced from a large collection of text
FIELD
MAGNETIC
MAGNET
WIRE
NEEDLE
CURRENT
COIL
POLES
IRON
COMPASS
LINES
CORE
ELECTRIC
DIRECTION
FORCE
MAGNETS
BE
MAGNETISM
POLE
INDUCED
SCIENCE
STUDY
SCIENTISTS
SCIENTIFIC
KNOWLEDGE
WORK
RESEARCH
CHEMISTRY
TECHNOLOGY
MANY
MATHEMATICS
BIOLOGY
FIELD
PHYSICS
LABORATORY
STUDIES
WORLD
SCIENTIST
STUDYING
SCIENCES
BALL
GAME
TEAM
FOOTBALL
BASEBALL
PLAYERS
PLAY
FIELD
PLAYER
BASKETBALL
COACH
PLAYED
PLAYING
HIT
TENNIS
TEAMS
GAMES
SPORTS
BAT
TERRY
JOB
WORK
JOBS
CAREER
EXPERIENCE
EMPLOYMENT
OPPORTUNITIES
WORKING
TRAINING
SKILLS
CAREERS
POSITIONS
FIND
POSITION
FIELD
OCCUPATIONS
REQUIRE
OPPORTUNITY
EARN
ABLE
[Tennenbaum et al]
5
![Page 6: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/6.jpg)
STORY
STORIES
TELL
CHARACTER
CHARACTERS
AUTHOR
READ
TOLD
SETTING
TALES
PLOT
TELLING
SHORT
FICTION
ACTION
TRUE
EVENTS
TELLS
TALE
NOVEL
MIND
WORLD
DREAM
DREAMS
THOUGHT
IMAGINATION
MOMENT
THOUGHTS
OWN
REAL
LIFE
IMAGINE
SENSE
CONSCIOUSNESS
STRANGE
FEELING
WHOLE
BEING
MIGHT
HOPE
WATER
FISH
SEA
SWIM
SWIMMING
POOL
LIKE
SHELL
SHARK
TANK
SHELLS
SHARKS
DIVING
DOLPHINS
SWAM
LONG
SEAL
DIVE
DOLPHIN
UNDERWATER
DISEASE
BACTERIA
DISEASES
GERMS
FEVER
CAUSE
CAUSED
SPREAD
VIRUSES
INFECTION
VIRUS
MICROORGANISMS
PERSON
INFECTIOUS
COMMON
CAUSING
SMALLPOX
BODY
INFECTIONS
CERTAIN
FIELD
MAGNETIC
MAGNET
WIRE
NEEDLE
CURRENT
COIL
POLES
IRON
COMPASS
LINES
CORE
ELECTRIC
DIRECTION
FORCE
MAGNETS
BE
MAGNETISM
POLE
INDUCED
SCIENCE
STUDY
SCIENTISTS
SCIENTIFIC
KNOWLEDGE
WORK
RESEARCH
CHEMISTRY
TECHNOLOGY
MANY
MATHEMATICS
BIOLOGY
FIELD
PHYSICS
LABORATORY
STUDIES
WORLD
SCIENTIST
STUDYING
SCIENCES
BALL
GAME
TEAM
FOOTBALL
BASEBALL
PLAYERS
PLAY
FIELD
PLAYER
BASKETBALL
COACH
PLAYED
PLAYING
HIT
TENNIS
TEAMS
GAMES
SPORTS
BAT
TERRY
JOB
WORK
JOBS
CAREER
EXPERIENCE
EMPLOYMENT
OPPORTUNITIES
WORKING
TRAINING
SKILLS
CAREERS
POSITIONS
FIND
POSITION
FIELD
OCCUPATIONS
REQUIRE
OPPORTUNITY
EARN
ABLE
Example topics
induced from a large collection of text
[Tennenbaum et al]
6
![Page 7: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/7.jpg)
Collocations
• An expression consisting of two or more
words that correspond to some conventional
way of saying things.
• Characterized by limited compositionality.
– compositional: meaning of expression can be
predicted by meaning of its parts.
– “dynamic programming”, “hidden Markov model”
– “weapons of mass destruction”
– “kick the bucket”, “hear it through the grapevine”
7
![Page 8: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/8.jpg)
Topics Modeling Phrases
• Topics based only on unigrams often
difficult to interpret
• Topic discovery itself is confused because
important meaning / distinctions carried by
phrases.
• Significant opportunity to provide improved
language models to ASR, MT, IR, etc.
8
![Page 9: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/9.jpg)
Topical N-gram Model
z1 z2 z3 z4
w1 w2 w3 w4
y1 y2 y3 y4
!
"1
T
D
. . .
. . .
. . .
#
WTW
$ %1%2& "2
[Wang, McCallum 2005]
9
![Page 10: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/10.jpg)
LDA Topic
LDA
algorithms
algorithm
genetic
problems
efficient
Topical N-grams
genetic algorithms
genetic algorithm
evolutionary computation
evolutionary algorithms
fitness function
10
![Page 11: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/11.jpg)
Topic Comparison
learning
optimal
reinforcement
state
problems
policy
dynamic
action
programming
actions
function
markov
methods
decision
rl
continuous
spaces
step
policies
planning
LDA
reinforcement learning
optimal policy
dynamic programming
optimal control
function approximator
prioritized sweeping
finite-state controller
learning system
reinforcement learning rl
function approximators
markov decision problems
markov decision processes
local search
state-action pair
markov decision process
belief states
stochastic policy
action selection
upright position
reinforcement learning methods
policy
action
states
actions
function
reward
control
agent
q-learning
optimal
goal
learning
space
step
environment
system
problem
steps
sutton
policies
Topical N-grams (2) Topical N-grams (1)
11
![Page 12: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/12.jpg)
Topic Comparison
motion
visual
field
position
figure
direction
fields
eye
location
retina
receptive
velocity
vision
moving
system
flow
edge
center
light
local
LDA
receptive field
spatial frequency
temporal frequency
visual motion
motion energy
tuning curves
horizontal cells
motion detection
preferred direction
visual processing
area mt
visual cortex
light intensity
directional selectivity
high contrast
motion detectors
spatial phase
moving stimuli
decision strategy
visual stimuli
motion
response
direction
cells
stimulus
figure
contrast
velocity
model
responses
stimuli
moving
cell
intensity
population
image
center
tuning
complex
directions
Topical N-grams (2) Topical N-grams (1)
12
![Page 13: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/13.jpg)
Topic Comparison
word
system
recognition
hmm
speech
training
performance
phoneme
words
context
systems
frame
trained
speaker
sequence
speakers
mlp
frames
segmentation
models
LDA
speech recognition
training data
neural network
error rates
neural net
hidden markov model
feature vectors
continuous speech
training procedure
continuous speech recognition
gamma filter
hidden control
speech production
neural nets
input representation
output layers
training algorithm
test set
speech frames
speaker dependent
speech
word
training
system
recognition
hmm
speaker
performance
phoneme
acoustic
words
context
systems
frame
trained
sequence
phonetic
speakers
mlp
hybrid
Topical N-grams (2) Topical N-grams (1)
13
![Page 14: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/14.jpg)
Unsupervised learning of
topic hierarchies(Blei, Griffiths, Jordan & Tenenbaum, NIPS 2003)
14
![Page 15: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/15.jpg)
Joint models of syntax and semantics (Griffiths,
Steyvers, Blei & Tenenbaum, NIPS 2004)
• Embed topics model inside an nth order
Hidden Markov Model:
Document-specific distribution over topics
15
![Page 16: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/16.jpg)
FOOD
FOODS
BODY
NUTRIENTS
DIET
FAT
SUGAR
ENERGY
MILK
EATING
FRUITS
VEGETABLES
WEIGHT
FATS
NEEDS
CARBOHYDRATES
VITAMINS
CALORIES
PROTEIN
MINERALS
MAP
NORTH
EARTH
SOUTH
POLE
MAPS
EQUATOR
WEST
LINES
EAST
AUSTRALIA
GLOBE
POLES
HEMISPHERE
LATITUDE
PLACES
LAND
WORLD
COMPASS
CONTINENTS
DOCTOR
PATIENT
HEALTH
HOSPITAL
MEDICAL
CARE
PATIENTS
NURSE
DOCTORS
MEDICINE
NURSING
TREATMENT
NURSES
PHYSICIAN
HOSPITALS
DR
SICK
ASSISTANT
EMERGENCY
PRACTICE
BOOK
BOOKS
READING
INFORMATION
LIBRARY
REPORT
PAGE
TITLE
SUBJECT
PAGES
GUIDE
WORDS
MATERIAL
ARTICLE
ARTICLES
WORD
FACTS
AUTHOR
REFERENCE
NOTE
GOLD
IRON
SILVER
COPPER
METAL
METALS
STEEL
CLAY
LEAD
ADAM
ORE
ALUMINUM
MINERAL
MINE
STONE
MINERALS
POT
MINING
MINERS
TIN
BEHAVIOR
SELF
INDIVIDUAL
PERSONALITY
RESPONSE
SOCIAL
EMOTIONAL
LEARNING
FEELINGS
PSYCHOLOGISTS
INDIVIDUALS
PSYCHOLOGICAL
EXPERIENCES
ENVIRONMENT
HUMAN
RESPONSES
BEHAVIORS
ATTITUDES
PSYCHOLOGY
PERSON
CELLS
CELL
ORGANISMS
ALGAE
BACTERIA
MICROSCOPE
MEMBRANE
ORGANISM
FOOD
LIVING
FUNGI
MOLD
MATERIALS
NUCLEUS
CELLED
STRUCTURES
MATERIAL
STRUCTURE
GREEN
MOLDS
Semantic classes
PLANTS
PLANT
LEAVES
SEEDS
SOIL
ROOTS
FLOWERS
WATER
FOOD
GREEN
SEED
STEMS
FLOWER
STEM
LEAF
ANIMALS
ROOT
POLLEN
GROWING
GROW
16
![Page 17: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/17.jpg)
GOOD
SMALL
NEW
IMPORTANT
GREAT
LITTLE
LARGE
*
BIG
LONG
HIGH
DIFFERENT
SPECIAL
OLD
STRONG
YOUNG
COMMON
WHITE
SINGLE
CERTAIN
THE
HIS
THEIR
YOUR
HER
ITS
MY
OUR
THIS
THESE
A
AN
THAT
NEW
THOSE
EACH
MR
ANY
MRS
ALL
MORE
SUCH
LESS
MUCH
KNOWN
JUST
BETTER
RATHER
GREATER
HIGHER
LARGER
LONGER
FASTER
EXACTLY
SMALLER
SOMETHING
BIGGER
FEWER
LOWER
ALMOST
ON
AT
INTO
FROM
WITH
THROUGH
OVER
AROUND
AGAINST
ACROSS
UPON
TOWARD
UNDER
ALONG
NEAR
BEHIND
OFF
ABOVE
DOWN
BEFORE
SAID
ASKED
THOUGHT
TOLD
SAYS
MEANS
CALLED
CRIED
SHOWS
ANSWERED
TELLS
REPLIED
SHOUTED
EXPLAINED
LAUGHED
MEANT
WROTE
SHOWED
BELIEVED
WHISPERED
ONE
SOME
MANY
TWO
EACH
ALL
MOST
ANY
THREE
THIS
EVERY
SEVERAL
FOUR
FIVE
BOTH
TEN
SIX
MUCH
TWENTY
EIGHT
HE
YOU
THEY
I
SHE
WE
IT
PEOPLE
EVERYONE
OTHERS
SCIENTISTS
SOMEONE
WHO
NOBODY
ONE
SOMETHING
ANYONE
EVERYBODY
SOME
THEN
Syntactic classes
BE
MAKE
GET
HAVE
GO
TAKE
DO
FIND
USE
SEE
HELP
KEEP
GIVE
LOOK
COME
WORK
MOVE
LIVE
EAT
BECOME
17
![Page 18: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/18.jpg)
Corpus-specific factorization
(NIPS)Semantics
Syntax
18
![Page 19: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/19.jpg)
REMAINED
5 8 14 25 26 30 33
IN ARE THE SUGGEST LEVELS RESULTS BEEN
FOR WERE THIS INDICATE NUMBER ANALYSIS MAY
ON WAS ITS SUGGESTING LEVEL DATA CAN
BETWEEN IS THEIR SUGGESTS RATE STUDIES COULD
DURING WHEN AN SHOWED TIME STUDY WELL
AMONG REMAIN EACH REVEALED CONCENTRATIONS FINDINGS DID
FROM REMAINS ONE SHOW VARIETY EXPERIMENTS DOES
UNDER REMAINED ANY DEMONSTRATE RANGE OBSERVATIONS DO
WITHIN PREVIOUSLY INCREASED INDICATING CONCENTRATION HYPOTHESIS MIGHT
THROUGHOUT BECOME EXOGENOUS PROVIDE DOSE ANALYSES SHOULD
THROUGH BECAME OUR SUPPORT FAMILY ASSAYS WILL
TOWARD BEING RECOMBINANT INDICATES SET POSSIBILITY WOULD
INTO BUT ENDOGENOUS PROVIDES FREQUENCY MICROSCOPY MUST
AT GIVE TOTAL INDICATED SERIES PAPER CANNOT
INVOLVING MERE PURIFIED DEMONSTRATED AMOUNTS WORK
THEY
AFTER APPEARED TILE SHOWS RATES EVIDENCE ALSO
ACROSS APPEAR FULL SO CLASS FINDING
AGAINST ALLOWED CHRONIC REVEAL VALUES MUTAGENESIS BECOME
WHEN NORMALLY ANOTHER DEMONSTRATES AMOUNT OBSERVATION MAG
ALONG EACH EXCESS SUGGESTED SITES MEASUREMENTS LIKELY
Syntactic classes in PNAS
19
![Page 20: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/20.jpg)
Semantic highlighting
Darker words are more likely to have been generated from the
topic-based “semantics” module:
20
![Page 21: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/21.jpg)
PP Attachment:A Simple Application of
Word Association
21
![Page 22: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/22.jpg)
Attachment Ambiguity
• Where to attach a phrase in the parse tree?
• “I saw the man with the telescope.”
– What does “with a telescope” modify?
– Is the problem AI complete? Yes, but…
– Proposed simple structural factors
• Right association [Kimball 1973]
‘low’ or ‘near’ attachment = ‘early closure’ of NP
• Minimal attachment [Frazier 1978]
(depends on grammar) = ‘high’ or ‘distant’ attachment
= ‘late closure’ (of NP)
22
![Page 23: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/23.jpg)
Attachment Ambiguity
• “The children ate the cake with a spoon.”
• “The children ate the cake with frosting.”
• “Joe included the package for Susan.”
• “Joe carried the package for Susan.”
• Ford, Bresnan and Kaplan (1982):“It is quite evident, then, that the closure effects inthese sentences are induced in some way by thechoice of the lexical items.”
23
![Page 24: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/24.jpg)
Lexical acquisition, semantic similarity
• Previous models give same estimate to allunseen events.
• Unrealistic - could hope to refine that basedon semantic classes of words
• Examples– “Susan ate the cake with a durian.”
– “Susan had never eaten a fresh durian before.”
– Although never seen “eating pineapple” should bemore likely than “eating holograms” becausepineapple is similar to apples, and we have seen“eating apples”.
24
![Page 25: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/25.jpg)
An application: selectional preferences
• Most verbs prefer arguments of a particular
type. Such regularities are called selectional
preferences or selectional restrictions.
• “Bill drove a…” Mustang, car, truck, jeep
• Selectional preference strength: how strongly
does a verb constrain direct objects
• “see” versus “unknotted”
25
![Page 26: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/26.jpg)
Measuring selectional preference strength
• Assume we are given a clustering of (direct object) nouns.Resnick (1993) uses WordNet.
• Selectional association between a verb and a class
Proportion that its summand contributes to preference strength.
• For nouns in multiple classes, disambiguate as most likelysense:
26
![Page 27: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/27.jpg)
Selection preference strength
(made up data)
Noun class c P(c) P(c|eat) P(c|see) P(c|find)
people 0.25 0.01 0.25 0.33
furniture 0.25 0.01 0.25 0.33
food 0.25 0.97 0.25 0.33
action 0.25 0.01 0.25 0.01
SPS S(v) 1.76 0.00 0.35
A(eat, food) = 1.08
A(find, action) = -0.13
27
![Page 28: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/28.jpg)
Selectional Preference Strength example(Resnick, Brown corpus)
28
![Page 29: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/29.jpg)
But how might we measure
word similarity for word classes?
• Vector spaces
29
![Page 30: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/30.jpg)
But how might we measure
word similarity for word classes?
• Vector spacesword-by-word matrix B
30
![Page 31: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/31.jpg)
Similarity measures for binary vectors
31
![Page 32: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/32.jpg)
Cosine measure
32
![Page 33: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/33.jpg)
Example of cosine measure on
word-by-word matrix on NYT
33
![Page 34: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/34.jpg)
Probabilistic measures
34
![Page 35: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/35.jpg)
Neighbors of word “company”[Lee]
35
![Page 36: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/36.jpg)
Learning syntactic patterns for
automatic hypernym discovery
Rion Snow, Daniel Jurafsky, and Andrew Y. Ng.
36
![Page 37: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/37.jpg)
37
![Page 38: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/38.jpg)
38
![Page 39: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/39.jpg)
39
![Page 40: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/40.jpg)
40
![Page 41: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/41.jpg)
41
![Page 42: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/42.jpg)
42
![Page 43: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/43.jpg)
43
![Page 44: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/44.jpg)
VERBOCEAN: Mining the Web for
Fine-Grained Semantic Verb Relations
Timothy Chklovski and Patrick Pantel
44
![Page 45: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/45.jpg)
45
![Page 46: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/46.jpg)
46
![Page 47: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/47.jpg)
47
![Page 48: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/48.jpg)
48
![Page 49: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/49.jpg)
49
![Page 50: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/50.jpg)
50
![Page 51: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/51.jpg)
51
![Page 52: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/52.jpg)
52
![Page 53: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/53.jpg)
53
![Page 54: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/54.jpg)
54
![Page 55: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/55.jpg)
55
![Page 56: Lexical Semantics II - UMass CSdasmith/inlp2009/lect19-cs585.pdf · lexical semantics ii ... mind world dream dreams thought imagination moment thoughts own ... map north earth south](https://reader031.vdocuments.mx/reader031/viewer/2022022004/5aaad0e87f8b9a72188ec4fb/html5/thumbnails/56.jpg)
http://semantics.isi.edu/ocean/
Demo
56