1 natural language processing lecture notes 10 chapter 14
TRANSCRIPT
1
Natural Language Processing
Lecture Notes 10Chapter 14
2
Big Transition
• First we did words (morphology)• Then we looked at syntax• Now we’re moving on to meaning.
3
Semantics
• What things mean• Approach: meaning representations
which bridge the gap from linguistic forms to knowledge of the world
• Serve the practical purposes of a program doing semantic processing
4
Semantic Processing
• Representations that allow a system to– Answer questions– Determine truth– Perform inference– …
5
Types of Meaning Representations
• First-order predicate calculus• Semantic networks• Conceptual dependency• Frame-based representations
• See lecture for examples…
6
Verifiability
• Does Spice Island serve vegetarian food?
• Serves(spiceisland,vegetarianfood)• Verifiability: the system’s ability to
compare representations to facts in memory
7
Current Focus in Class
• Conventional meanings of words• Ignore context• Literal meaning
8
Ambiguity
• I want to eat someplace that’s close to Pitt
• Mary kissed her husband and Joan did too
• I baked the cake on the table• Old men and women go to the park• Every student ate a sandwich
9
Canonical Form
• Does Spice Island have vegetarian dishes?
• Do they have vegetarian food at Spice Island?
• Are vegetarian dishes served at Spice Island?
• Does Spice Island serve vegetarian fare?• Canonical form: inputs that mean the
same thing should have the same meaning representations
10
Canonical Form
• Simplifies reasoning• Makes representations more compact
(fewer different representations)• BUT: makes semantic analysis harder
– Need to figure out that “have” and “serve” mean the same thing in the previous examples; same for the various phrases for vegetarian food
– BUT: can perform word sense disambiguation; use a single representation for all senses in a synset
11
Representational Schemes
• We’re going to make use of First Order Predicate Calculus (FOPC) as our representational framework– Not because we think it’s perfect– All the alternatives turn out to be
either too limiting or too complicated, or
– They turn out to be notational variants
12
Knowledge Based Agents
• Central component: knowledge base, or KB.
• A set of sentences in a knowledge representation language
• Generic Functions– TELL (add a fact to the knowledge base)– ASK (get next action based on info in KB)
• Both often involve inference, which is?• Deriving new sentences from old
13
Fundamental Concepts of logical representation and reasoning
• Information is represented in sentences, which must have correct syntax( 1 + 2 ) * 7 = 21 vs. 2 ) + 7 = * ( 1 21
• The semantics of a sentence defines its truth with respect to each possible world
• W is a model of S means that sentence S is true in world W
• What do the following mean?– X |= Y – X entails Y– Y logically follows from X
14
Entailment
• A |= B• In all worlds in which A is true, B
must be true as well• All models of A are models of B• Whenever A is true, B must be true
as well• A entails B• B logically follows from A
15
Inference
KB |-i A
Inference algorithm i can derive A from KB
i derives A from KBi can derive A from KBA can be inferred from KB by i
16
Propositional Logic Syntax
• Sentence -> AtomicSent | complexSentAtomicSent -> true|false| P, Q, R …ComplexSent ->sentence | ( sentence sentence ) |( sentence sentence ) |( sentence sentence ) |( sentence sentence ) |( sentence )
[no predicate or function symbols]
17
Propositional Logic Sentences
• If there is a pit at [1,1], there is a breeze at [1,0]P11 B10
• There is a breeze at [2,2], if and only if there is a pit in the neighborhoodB22 ( P21 P23 P12 P32 )
• There is no breeze at [2,2]B22
18
Semantics of Prop Logic
• In model-theoretic semantics, an interpretation assigns elements of the world to sentences, and defines the truth values of sentences
• Propositional logic: easy! Assign T or F to each proposition symbol; then assign truth values to complex sentences in the obvious way
19
Propositional Logic
• A ^ B is true if both A and B are true
• A v B is true if one or both A and B are true
• P Q equiv ~P v Q. Thus, P Q is false if P is true and Q is false. Otherwise, P Q is true.
• ~A is true if A is false
20
Proofs
• A derivation• A sequence of applications of (usually
sound) rules of inference• Reasoning by Search• Example KB = AB, BC, DE, EF, D• Forward chaining: Add A, infer B, infer C• Backward chaining:F? E? D? Yes…• Sound but not complete inference
procedures
21
Resolution
• Resolution allows a sound and complete inference mechanism (search-based) using only one rule of inference
• Resolution rule:– Given: P1 P2 P3 … Pn, and P1 Q1 … Qm
– Conclude: P2 P3 … Pn Q1 … Qm
Complementary literals P1 and P1 “cancel out”
For your information only; resolution won’t be on exam
22
Resolution
• Any complete search algorithm, applying only the resolution rule, can derive any conclusion entailed by any KB in propositional logic.
• Refutation completeness: Given A, we cannot use resolution to generate the consequence A v B. But we can answer the question, is A v B true. I.e., resolution can be used to confirm or refute a sentence
Again, for your information only; will not be on the exam
23
Unsound (but useful) Inference
• Where there is smoke, there is fire• Example KB: Fire Smoke, Smoke• Abduction: conclude Fire• Unsound:
– Example KB1: Fire Smoke, DryIce Smoke, Smoke
– DryIce rather than Fire could be true
24
Propositional Logic FOPC
B11 (P12 v P21)B23 (P32 v P 23 v P34 v P 43) …“Internal squares adjacent to pits are
breezy”:All X Y (B(X,Y) ^ (X > 1) ^ (Y > 1) ^
(Y < 4) ^ (X < 4)) (P(X-1,Y) v P(X,Y-1) v P(X+1,Y) v
(X,Y+1))
25
FOPC Worlds
• Rather than just T,F, now worlds contain:
• Objects: the gold, the wumpus, people, ideas, … “the domain”
• Predicates: holding, breezy, red, sisters
• Functions: fatherOf, colorOf, plusOntological commitment
26
FOPC Syntax
• Add variables and quantifiers to propositional logic
27
Sentence AtomicSentence | (Sentence Connective Sentence) | Quantifier Variable, … Sentence | ~SentenceAtomicSentence Predicate(Term,…) | Term = TermTerm Function(Term,…) | Constant | VariableConnective | ^ | v | Quantifier all, existsConstant john, 1, …Variable A, B, C, XPredicate breezy, sunny, redFunction fatherOf, plus
Knowledge engineering involves deciding what types of things Should be constants, predicates, and functions for your problem
28
Examples
• Everyone likes chocolate X (person(X) likes(X, chocolate))
• Someone likes chocolate X (person(X) ^ likes(X, chocolate))
• Everyone likes chocolate unless they are allergic to it X ((person(X) ^ allergic (X, chocolate)) likes(X, chocolate))
29
Quantifiers
• All X p(X) means that p holds for all elements in the domain
• Exists X p(X) means that p holds for at least one element of the domain
• In well-formed FOPC, all variables are bound by (in the scope of) a quantifier
30
Nesting of Variables
1. Everyone likes some kind of food2. There is a kind of food that
everyone likes3. Someone likes all kinds of food4. Every food has someone who likes
it
31
Answers
Everyone likes some kind of foodAll P (person(P) Exists F (food(F) and
likes(P,F)))There is a kind of food that everyone likes Exists F (food(F) and (All P (person(P)
likes(P,F))))Someone likes all kinds of food Exists P (person(P) and (All F (food(F)
likes(P,F)))) Every food has someone who likes it All F (food (F) Exists P (person(P) and
likes(P,F)))
32
Semantics of FOPC: Interpretation
• Specifies which objects, functions, and predicates are referred to by which constant symbols, function symbols, and predicate symbols.
33
Example
• 3 people: John, Sally, Bill• John is tall• Sally and Bill are short• John is Bill’s father• Sally is Bill’s sister• Interpretation 1 (others are possible):
– “John”, “Sally”, and “Bill” as you think– “person” {John, Sally,Bill}– “short” {Sally,Bill}– “tall” {John}– “sister” {<Sally,Bill>} A 2-ary predicate– “father” {<Bill,John>} A 1-ary function
34
Exampletall(father(bill)) ^ ~sister(sally,bill)
• Assign meanings to terms:– “bill” Bill; “sally” Sally; “father(bill)”
John
• Assign truth values to atomic sentences– Tall(father(bill)) is T because John is in the set
assigned to “tall”– ~sister(sally,bill) is F because <Sally,Bill> is in
the set assigned to “sister”– So, sentence is false, because T ^ F is F
35
Determining Truth Values
• Exist X tall(X) : true, because the set assigned to “tall” isn’t {}
• All X short(X) : false, because there are objects that are not in the set assigned to “short”
36
Representational Schemes
• What are the objects, predicates, and functions?
37
Choices:Functions vs Predicates
• Rep-Scheme 1: tall(fatherOf(bob)).
• Rep-Scheme 2:Exists X (fatherOf(bob,X) ^ tall(X) ^ (All Y (fatherOf(bob,Y) X = Y)))
• “fatherOf” in both cases is assigned a set of 2-tuples: {<b,bf>,<t,tf>,…}
• But {<b,bf>,<t,tf>,<b,bff>,…} is possible if it is a predicate
38
Choices: Predicates versus Constants
• Rep-Scheme 1: D = {a,b,c,d,e}. red: {a,b,c}. pink: {d,e}. Some true sentences:
red(a). red(b). pink(d). ~(All X red(X)). All X (red(X) v pink(X)).
But what if we want to say that pink is pretty?
39
Choices: Predicates versus Constants
• Rep-Scheme 2: D = {a,b,c,d,e,red,pink} colorof:
<a,red>,<b,red>,<c,red>,<d,pink>,<e,pink>}
pretty: {pink} primary: {red}
• Some true sentences: colorOf(a,red). colorOf(b,red). colorOf(d,pink).
~(All X colorOf(X,red)). All X (colorOf(X,red) v colorOf(X,pink)).
pretty(pink). primary(red).We have reified predicates pink and red: made them
into objects
40
Inference with Quantifiers• Universal Instantiation:
– Given X (person(X) likes(X, sun))– Infer person(john) likes(john,sun)
• Existential Instantiation:– Given x likes(x, chocolate)– Infer: likes(S1, chocolate)– S1 is a “Skolem Constant” that is not
found anywhere else in the KB and refers to (one of) the individuals who likes sun.
41
FOPC
• This choice isn’t completely arbitrary or driven by the needs of practical applications
• FOPC reflects the semantics of natural languages because it was designed that way by human beings
• In particular…
42
Meaning Structure of Language
• The semantics of human languages…– Display a basic predicate-argument
structure– Make use of variables– Make use of quantifiers– Use a partially compositional
semantics
43
Predicate-Argument Structure
• Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates.
• Languages display a division of labor where some words and constituents function as predicates and some as arguments.
44
Example
• Mary gave a list to John• Giving(Mary, John, List)• More precisely
– Gave conveys a three-argument predicate
– The first arg refers to the subject (the giver)
– The second is the recipient, which is conveyed by the NP in the PP
– The third argument refers to the thing given, conveyed by the direct object
45
Is this a good representation?
• John gave Mary a book for Susan– Giving (john,mary,book,susan)
• John gave Mary a book for Susan on Wednesday – Giving (john,mary,book,susan,wednesday)
• John gave Mary a book for Susan on Wednesday in class– Giving (john,mary,book,susan,wednesday,inClass)
• John gave Mary a book for Susan on Wednesday in class after 2pm– Giving
(john,mary,book,susan,wednesday,inClass,>2pm)
46
Reified Representation
• Exist b,e (ISA(e,giving) ^ agent(e,john) ^ beneficiary(e,sally) ^ patient(e,b) ^ ISA(b,book))
• “That happened on Sunday”• Add later (assuming S2 is the
skolem for e): – happenedOnDay(s2,sunday)
47
Representing Time in Language
• I ran to Oakland• I am running to Oakland• I will run to Oakland• Now, all represented the same:• Exist w (ISA(w,running) ^
agent(w,speaker) ^ dest(w,oakland))
48
Representing Time
• Events are associated with points or intervals in time.
1. Exist w,i (ISA(w,running) ^ agent(w,speaker) ^ dest(w,oakland) ^ interval(w,i) ^ precedes(i,now))
2. … member(i,now)3. … precedes(now,i)
49
Determining Temporal Relations is complex (largely unsolved)
• Ok, so for Christmas, we fly to Dallas then to El Paso (refers to the future, but the tense is present)
• Let’s see, flight 1390 will be at the gate an hour now (refers to an interval starting in the past using the future tense)
• I take the bus in the morning but the incline in the evening (habitual – not a specific morning or evening)
50
Reference Point
• Flight 2020 arrived• Flight 2020 had arrived • What’s the difference?• What do you expect in the second
example?
51
Reference Point
• Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E)
• Example:– When Mary's flight departed, I made the
call– When Mary's flight departed, I had made
the call
• Departure event specifies reference point.
52
Reichenbach Applied to Tenses
S S R,S
S,R,E S,R S
We refer to the S,R,E notation as a Basic Tense Structure (BTS)
53
Aspect: A property of Verb Phrases
• Combo of several qualities of events• Statives: express a property of the
experiencer; do not imply anything about the time bounds– “I like the Yankees in the World Series”– “I want to go first class”– “I have the flu”
• Tests for statives: – cannot be used as commands: *“Want to go
first class!”– Odd in progressive: “I am wanting to go first
class”
54
Aspect (continued)
• Activities: have a time span, but not necessarily an end point– Running, playing the piano, looking for an
umbrella
• Tests: (progressive and commands are fine)– Odd when modified by temporal expressions
with “in”: *He was running in 5 minutes
• Subinterval property: e.g., any subinterval of playing the piano is playing the piano
55
Aspect (continued)
• Accomplishments: have a well-defined end point, and cause a new state of affairs to exist– “I booked a reservation”
• Test: Cannot be modified by “stopped” without changing the meaning
• Property: “in” adverbials ok: “I booked a reservation in 5 minutes”
56
Aspect (continued)
• Achievements: occur at a singular point in time; cause a new state of affairs to exist– “Lost an umbrella”, “noticed the
picture”, “arrived at the station”
• Test: cannot occur over a time span– *I lost an umbrella for three months– Note: “I noticed the picture for three
months” has an iterative meaning (this is not a simple achievement)