lecture 16 - university of illinois · active/passive alternation: tom broke the window with a...
TRANSCRIPT
![Page 1: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/1.jpg)
CS447: Natural Language Processinghttp://courses.engr.illinois.edu/cs447
Julia [email protected] Siebel Center
Lecture 16: More on Compositional Semantics, Verb Semantics
![Page 2: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/2.jpg)
CS447: Natural Language Processing
AdminMidterm:
Regrade requests for midterm accepted until Nov 9Points available on Compass. 22 points = 100%
Project/Literature review proposals:Due at the end of day on Monday on CompassOne page PDF (in LaTeX, not Word) is sufficientInclude your names and NetIDsInclude all references (ideally with hyperlinks)Explain what you want to do and why.Include a to-do listFor projects: describe what resources you have or need. (Use existing datasets, don’t annotate your own data)
�2
![Page 3: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/3.jpg)
CS447: Natural Language Processing
Combinatory Categorial Grammar (CCG)
�3
![Page 4: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/4.jpg)
CS447 Natural Language Processing
CCG categoriesSimple (atomic) categories: NP, S, PP
Complex categories (functions):Return a result when combined with an argument VP, intransitive verb S\NP
Transitive verb (S\NP)/NP
Adverb (S\NP)\(S\NP)
Prepositions ((S\NP)\(S\NP))/NP (NP\NP)/NPPP/NP
�4
![Page 5: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/5.jpg)
CCG categories are functionsCCG has a few atomic categories, e.g
S, NP, PPAll other CCG categories are functions:
�5
SResult
NPArgument
/Dir.
![Page 6: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/6.jpg)
Rules: Function application
S/NPFunction
NPArgument
�6
Result S
x
y
· y = x
![Page 7: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/7.jpg)
Rules: Function application
S\NPFunction
NPArgument
�7
Result S
y · x
y
= x
![Page 8: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/8.jpg)
Rules: Function application
(S\NP)/NPFunction
NPArgument
�8
ResultS\NP
x
y
· y = x
![Page 9: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/9.jpg)
CS447 Natural Language Processing
A (C)CG derivation
�9
![Page 10: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/10.jpg)
Rules: Function Composition
S/S1st Function
S\NP2nd Function
�10
S\NP
x
y
· y
z
=x
z
![Page 11: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/11.jpg)
Rules: Type-Raising
NP�11
S/(S\NP)
y =x
x
· y =x�x
y
�
![Page 12: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/12.jpg)
CS447: Natural Language Processing
Type-raising and compositionType-raising: X → T/(T\X)
Turns an argument into a function. NP → S/(S\NP) (subject)NP → (S\NP)\((S\NP)/NP) (object)
Harmonic composition: X/Y Y/Z → X/ZComposes two functions (complex categories)(S\NP)/PP PP/NP → (S\NP)/NPS/(S\NP) (S\NP)/NP → S/NP
Crossing function composition: X/Y Y\Z → X\ZComposes two functions (complex categories)(S\NP)/S S\NP → (S\NP)\NP
�12
![Page 13: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/13.jpg)
CS447: Natural Language Processing
Type-raising and composition
�13
Wh-movement (relative clause):
Right-node raising:
![Page 14: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/14.jpg)
CS447: Natural Language Processing
Using Combinatory Categorial Grammar (CCG) to map sentences to predicate logic
�14
![Page 15: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/15.jpg)
CS447: Natural Language Processing
λ-ExpressionsWe often use λ-expressions to construct complex logical formulas:
-λx.φ(..x...) is a function where x is a variable, and φ some FOL expression.
-β-reduction (called λ-reduction in textbook):Apply λx.φ(..x...) to some argument a: (λx.φ(..x...) a) ⇒ φ(..a...) Replace all occurrences of x in φ(..x...) with a
-n-ary functions contain embedded λ-expressions:λx.λy.λz.give(x,y,z)
�15
![Page 16: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/16.jpg)
CS447: Natural Language Processing
CCG semanticsEvery syntactic constituent has a semantic interpretation:
Every lexical entry maps a word to a syntactic category and a corresponding semantic type:
John=(NP, john’ ) Mary= (NP, mary’ ) loves: ((S\NP)/NP λx.λy.loves(x,y))
Every combinatory rule has a syntactic and a semantic part: Function application: X/Y:λx.f(x) Y:a → X:f(a) Function composition: X/Y:λx.f(x) Y/Z:λy.g(y) → X/Z:λz.f(λy.g(y).z)Type raising: X:a → T/(T\X) λf.f(a)
�16
![Page 17: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/17.jpg)
CS447: Natural Language Processing
An example with semantics
�17
John sees MaryNP : John (S\NP)/NP : �x.�y.sees(x,y) NP : Mary
>S\NP : �y.sees(Mary,y)
<S : sees(Mary,John)
![Page 18: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/18.jpg)
CS447: Natural Language Processing
Supplementary material: quantifier scope ambiguities in CCG
�18
![Page 19: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/19.jpg)
CS447: Natural Language Processing
Quantifier scope ambiguity“Every chef cooks a meal”
- Interpretation A:For every chef, there is a meal which he cooks.
- Interpretation B:There is some meal which every chef cooks.
�19
⇤y[meal(y)⌅⇥x[chef (x)� cooks(y,x)]]
⇥x[chef (x)�⇤y[meal(y)⌅ cooks(y,x)]]
![Page 20: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/20.jpg)
CS447: Natural Language Processing �20
Every chef cooks a meal(S/(S\NP))/N N (S\NP)/NP ((S\NP)\((S\NP)/NP))/N N
�P�Q.⇤x[Px⇥ Qx] � z.chef (z) �u.�v.cooks(u,v) �P�Q⌅y[Py⇧Qy] � z.meal(z)> >
S/(S\NP) (S\NP)\((S\NP)/NP)�Q.⇤x[� z.chef (z)x⇥ Qx] �Q⌅y[� z.meal(z)y⇧Qy]� �Q.⇤x[chef (x)⇥ Qx] � �Q�w.⌅y[meal(y)⇧Qyw]
<S\NP
�w.⌅y[meal(y)⇧�u�v.cooks(u,v)yw]� �w.⌅y[meal(y)⇧ cooks(y,w)]
>S : ⇤x[chef (x)⇥ �w.⌅y[meal(y)⇧ cooks(y,w)]x]� ⇤x[chef (x)⇥⌅y[meal(y)⇧ cooks(y,x)]]
Interpretation A
![Page 21: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/21.jpg)
CS447: Natural Language Processing �21
Every chef cooks a meal(S/(S\NP))/N N (S\NP)/NP (S\(S/NP))/N N
�P�Q.⇤x[Px⇥ Qx] � z.chef (z) �u.�v.cooks(u,v) �P�Q⌅y[Py⇧Qy] � z.meal(z)> >
S/(S\NP) S\(S/NP)�Q⇤x[� z.chef (z)x⇥ Qx] �Q⌅y[� z.meal(z)y⇧Qy]� �Q⇤x[chef (x)⇥ Qx] � �Q⌅y[meal(y)⇧Qy]
>BS/NP
�w.⇤x[chef (x)⇥ �u�v.cooks(u,v)wx]� �w.⇤x[chef (x)⇥ cooks(w,x)]
<S⌅y[meal(y)⇧�w.⇤x[chef (x)⇥ cooks(y,w)]x]� ⌅y[meal(y)⇧⇤x[chef (x)⇥ cooks(y,x)]]
Interpretation B
![Page 22: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/22.jpg)
CS447: Natural Language Processing
To summarize…
�22
![Page 23: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/23.jpg)
CS447: Natural Language Processing
Understanding sentences“Every chef cooks a meal”⇥x[chef (x)�⇤y[meal(y)⌅ cooks(y,x)]]⇤y[meal(y)⌅⇥x[chef (x)� cooks(y,x)]]
We translate sentences into (first-order) predicate logic.
Every (declarative) sentence corresponds to a proposition, which can be true or false.
�23
![Page 24: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/24.jpg)
CS447: Natural Language Processing
But…… what can we do with these representations?
Being able to translate a sentence into predicate logic is not enough, unless we also know what these predicates mean.Semantics joke (B. Partee): The meaning of life is life’Compositional formal semantics tells us how to fit together pieces of meaning, but doesn’t have much to say about the meaning of the basic pieces (i.e. lexical semantics)
… how do we put together meaning representations of multiple sentences?
We need to consider discourse (there are approaches within formal semantics, e.g. Discourse Representation Theory)
… Do we really need a complete analysis of each sentence? This is pretty brittle (it’s easy to make a parsing mistake)Can we get a more shallow analysis?
�24
![Page 25: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/25.jpg)
CS447: Natural Language Processing
Semantic Role Labeling/Verb Semantics
�25
![Page 26: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/26.jpg)
CS447: Natural Language Processing
What do verbs mean?Verbs describe events or states (‘eventualities’):
Tom broke the window with a rock. The window broke. The window was broken by Tom/by a rock.
We want to translate verbs to predicates. But: a naive translation (e.g. subject = first argument, object = second argument, etc.) does not capture the differences in meaningbreak(Tom, window, rock)break(window)break(window, Tom)break(window, rock)
�26
![Page 27: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/27.jpg)
CS447: Natural Language Processing
Semantic/Thematic rolesVerbs describe events or states (‘eventualities’):
Tom broke the window with a rock. The window broke. The window was broken by Tom/by a rock.
Thematic roles refer to participants of these events:Agent (who performed the action): TomPatient (who was the action performed on): windowTool/Instrument (what was used to perform the action): rock
Semantic/thematic roles (agent, patient) are different from grammatical roles (subject or object).
�27
![Page 28: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/28.jpg)
CS447: Natural Language Processing
The inventory of thematic rolesWe need to define an inventory of thematic roles
To create systems that can identify thematic roles automatically, we need to create labeled training data.
It is difficult to give a formal definition of thematic rolesthat generalizes across all verbs.
�28
![Page 29: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/29.jpg)
CS447: Natural Language Processing
PropBank and FrameNetProposition Bank (PropBank):Very coarse argument roles (arg0, arg1,…), used for all verbs (but interpretation depends on the specific verb)
Arg0 = proto-agentArg1 = proto-patientArg2...: specific to each verbArgM-TMP/LOC/...: temporal/locative/... modifiers
FrameNet:Verbs fall into classes that define different kinds of frames(change-position-on-a-scale frame: rise, increase,...).Each frame has its own set of “frame elements” (thematic roles)
�29
![Page 30: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/30.jpg)
CS447: Natural Language Processing
PropBankagree.01 Arg0: Agreer Arg1: Proposition Arg2: Other entity agreeing[Arg0 The group] agreed [Arg1 it wouldn’t make an offer] [Arg0 John] agrees with [Arg2 Mary]
fall.01 Arg1: patient/thing falling Arg2: extent/amount fallen Arg3: start point Arg4: end point[Arg1 Sales] fell [Arg4 to $251 million] [Arg1 Junk bonds] fell [Arg2 by 5%]
Semantic role labeling: Recover the semantic roles of verbs (nowadays typically PropBank-style)
Machine learning; trained on PropBankSyntactic parses provide useful information
�30
![Page 31: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/31.jpg)
CS447: Natural Language Processing
Diathesis AlternationsActive/passive alternation:
Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice)
Causative alternation:Tom broke the window. (‘causative’; active voice)The window broke. (‘anticausative’/‘inchoative’; active voice)
Dative alternationTom gave the gift to Mary. Tom gave Mary the gift.
Locative alternation:Jessica loaded boxes into the wagon.Jessica loaded the wagon with boxes.
�31
![Page 32: Lecture 16 - University Of Illinois · Active/passive alternation: Tom broke the window with a rock. (active voice) The window was broken by Tom/by a rock. (passive voice) Causative](https://reader034.vdocuments.mx/reader034/viewer/2022043012/5fa8196206fdcb2aef48b134/html5/thumbnails/32.jpg)
CS447: Natural Language Processing
Verb classesVerbs with similar meanings undergo the same syntactic alternations, and have the same set of thematic roles(Beth Levin, 1993)
VerbNet (verbs.colorado.edu; Kipper et al., 2008) A large database of verbs, their thematic roles and their alternations
�32