LING 438/538Computational Linguistics

Sandiway Fong

Lecture 23: 11/14

2

Administrivia

• Reading– Chapter 9: Context-Free Grammars for English

• Homework 5– due tonight

• 538 Presentations– (tentative) November 30th and December 5th– last two lectures of the semester– announcement about chapters on Thursday

3

Today’s Topic

• Syntax and Phrase Structure Grammars

4

9.1 Constituency

• constituents– represented by nodes in a

phrase structure tree

• question– How do we decide what tree

to assign for a given sentence (in a principled way)?

• Syntax• Answer (partial):

– we have to decide what the basic categories are, e.g. nouns, verbs, adjectives, adverbs, complementizers, tense ...

– we have to decide what the phrases in the sentence are

examples:who that John knows does he like?who does he like?

[Output from the PAPPIsystem from my webpage]

5

9.1 Constituency

• tests for constituency– diagnostics– sources:

• textbook: section 9.1

• introductory syntax textbook, e.g. Radford (1997)

• http://www.ling.upenn.edu/~beatrice/150/ch2.html (used here)

• substitution test– Noun Phrase (NP) - pronouns

• The little boy fed the cat. ---> He fed her

• Black cats detest green peas ---> They detest them

• A sequence of words can be a noun phrase in one environment but not pass the test in another:

• The little boy from next door fed the cat without a tail ---> * He from next door fed her without a tail

• These black cats detest those green peas ---> * These they detest those them

6

9.1 Constituency

• substitution test– Preposition Phrase (PP) - adverbs: here, there

• Put it on the table ---> Put it there

• Put it over on the table ---> Put it over there

• Put it over on the table ---> Put it there

• Put it on the table that's by the door ---> * Put it there that's by the door

• Put it over on the table that's by the door ---> * Put it over there that's by the door

• Put it over on the table that's by the door ---> * Put it there that's by the door

– Adjective Phrase (AP) - so• I am very happy, ... ... and Linda is so, too.

• I am very fond of Lukas, ... ... and Linda is so, too

• I am very fond of my nephew, ... * ... and Linda is so of her niece

– Clauses (CP) - so, it• I { know, suspect } that they're invited ---> I { know, suspect } it

• I { imagine, think } that they're invited ---> I { imagine, think } so

7

9.1 Constituency

• movement test– certain sequences of words move together as a unit (constituent)– textbook: preposed or postposed constructions– Noun Phrase (NP) - Topicalization

• I fed the cats. ---> The cats, I fed ___ (The dogs, I didn't) • I fed the cats with long, fluffy tails ---> The cats with long, fluffy tails, I fed ___ (The other cats,

I didn't)

– other phrases - Preposing• PP: The cat strolled across the porch with a confident air ---> With a confident air, the cat

strolled across the porch ___• AP: Ali Baba returned from his travels wiser than before ---> Wiser than before, Ali Baba

returned from his travels ___• AdvP: They arrived at the concert hall more quickly than they had expected ---> More quickly

than they had expected, they arrived at the concert hall ___• I fed the cats with long, fluffy tails ---> *The cats, I fed ___ with long, fluffy tails• The cat strolled across the porch with a confident air ---> *With a, the cat strolled across the

porch ___ confident air• Ali Baba returned from his travels wiser than before ---> *Wiser than, Ali Baba returned from his

travels ___ before• They arrived at the concert hall more quickly than they had expected. ---> *More quickly than

they, they arrived at the concert hall ___ had expected

8

9.1 Constituency

• question and sentence fragments test– sequences of words may function as a sentence fragment in response to a question– NP: What do you like?

• The cats• Cats with long, fluffy tails• The cats with long, fluffy tails

– PP: How did the cat stroll across the porch? • With a confident air.

– PP: Where did Ali Baba go? • On a long journey• To New York

– AP: How did Ali Baba return? • Wiser than before • Fairly jeg-lagged

– AdvP: How did they do? • Not badly• Surprisingly well• Much better than they had expected

9

9.1 Constituency

• It cleft focus test– Ordinary cats detest the smell of citrus fruits ---> It is ordinary cats that detest the

smell of citrus fruits– Ordinary cats detest the smell of citrus fruits ---> It is the smell of citrus fruits that

ordinary cats detest– PP: The cat strolled across the porch with a confident air ---> It was with a confident

air that the cat strolled across the porch ___– AP: Ali Baba returned from his travels wiser than before ---> It was wiser than before

that Ali Baba returned from his travels ___– AdvP: They arrived at the concert hall more quickly than they had expected ---> It was

more quickly than they had expected that they arrived at the concert hall ___– Ordinary cats detest the smell of citrus fruits ---> * It is the smell of that ordinary cats

detest ___ citrus fruits– The cat strolled across the porch with a confident air ---> * It was with a confident that

the cat strolled across the porch ___ air– Ali Baba returned from his travels wiser than before ---> * It was wiser than that Ali

Baba returned from his travels ___ before– They arrived at the concert hall more quickly than they had expected ---> * It was

quickly than they had expected that they arrived at the concert hall more ___

10

9.1 Constituency

• Verb Phrases (VP)– substitution test: do so– question/sentence fragment test: do what

• She will write a book ---> ok She will do so• The two boys could order tuna salad sandwiches ---> ok The two boys could do so • What will she do? ---> ok Write a book • What could the two boys do? ---> ok Order tuna salad sandwiches

– movement test: only nonfinite VP pass• nonfinite VPs:

– (She said that) she will write a book ---> ok (and) write a book, she will ___– though she may write a book ---> ok write a book though she may ___

• finite VPs: – (She said that) she wrote a book ---> * (and) wrote a book, she ___

– though she wrote a book ---> * wrote a book though she ___ – it cleft test: only nonfinite VP pass

– She will write a book ---> ? It is write a book that she will ___– She wrote a book ---> * It is wrote a book that she ___

11

9.2 Context-Free Rules and Trees

• recap– Prolog Definite Clause Grammar (DCG) Rules

• example– Formal grammar Prolog format Notes:– S aB s --> [a],b. Start symbol: S– B aB b --> [a],b. Non-terminals: {S,B,C}– B bC b --> [b],c. (uppercase letters)– B b b --> [b]. Terminals: {a,b}– C bC c --> [b],c. (lowercase letters)– C b c --> [b].

• Prolog format– terminals and non-terminal symbols begin with lowercase letters

• Variables begin with an uppercase letter (or underscore)– --> is the rewrite symbol– terminals are enclosed in square brackets (list notation)– The comma (,) is the concatenation symbol– a period (.) is required at the end of a DCG rule

12

9.2 Context-Free Rules and Trees

• Context-free grammar rules:1. exactly one non-terminal on the LHS of the rule2. (unrestricted RHS) zero or more terminals and non-terminals on the RHS

of the rule• cf. regular grammars

– one non-terminal and one terminal (in per grammar fixed order) on the RHS of the rule – exactly one terminal on the RHS of the rule

• NP rules:1. np --> det, nominal.2. np --> propernoun.3. nominal --> noun | noun, nominal.4. det --> [a].5. det --> [the.]6. noun --> [flight].

• Terminology:– production = rule– a flight can be derived from the non-terminal NP

Disjunction

13

9.2 Context-Free Rules and Trees

• NP DCG rules– np --> det, nominal.– np --> propernoun.– nominal --> noun | noun, nominal.– det --> [a].– det --> [the].– noun --> [flight].

• query– ?- np([a,flight],[]).– yes

• query (with tracing):– ?- trace.– ?- np([a,flight],[]).– 1 1 Call: np([a,flight],[]) ? (return)– 2 2 Call: det([a,flight],_971) ? s (s=skip)– ? 2 2 Exit: det([a,flight],[flight]) ? – 3 2 Call: nominal([flight],[]) ? – 4 3 Call: noun([flight],[]) ? s– 4 3 Exit: noun([flight],[]) ? – ? 3 2 Exit: nominal([flight],[]) ? – ? 1 1 Exit: np([a,flight],[]) ? – yes

14

9.2 Context-Free Rules and Trees

• Add an extra argument to the DCG rules to generate a parse tree explicitly

• NP DCG rules:– np(np(D,N)) --> det(D), nominal(N).– np(np(PN)) --> propernoun(PN).– nominal(nom(N)) --> noun(N).– nominal(nom(N1,N)) --> noun(N1), nominal(N).– det(det(a)) --> [a].– det(det(the)) --> [the].– noun(noun(flight)) --> [flight].

• query– ?- np(Tree,[a,flight],[]).– Tree = np(det(a),nom(noun(flight)))

15

9.2 Context-Free Rules and Trees

• A Word on the Domain– transcribed ATIS-3 corpus– speech corpus for the Air Travel Information Services (ATIS)

domain– Examples:

• I'D LIKE TO SEE ALL FLIGHT FROM <city> TO <city> AGAIN• PLEASE SHOW ME AIRLINE WITH FLIGHT FROM <city> TO <city>

LIST ALL FLIGHT ON <day> FROM <city> TO <city> NONSTOP• DOES <airline> FLY FROM <city> TO <city> • SHOW ME LIST OF FLIGHT ON <month> <day> LEAVING <city> TO

<city> • FIND FLIGHT FROM <city> TO <city> ON <day>• HOW DO I GET TO <city> DOWNTOWN FROM AIRPORT

16

9.2 Context-Free Rules and Trees

• example grammar L0

17

9.2 Context-Free Rules and Trees

• example grammar L0 (in Prolog):– s(s(NP,VP)) --> np(NP), vp(VP).– np(np(N)) --> pronoun(N).– np(np(PN)) --> propernoun(PN).– np(np(D,N)) --> det(D),

nominal(N).– nominal(nom(N)) --> noun(N).– nominal(nom(N1,N)) -->

noun(N1), nominal(N).– vp(vp(V)) --> verb(V).– vp(vp(V,NP)) -->

verb(V),np(NP).– vp(vp(V,NP,PP)) -->

verb(V),np(NP),pp(PP).– vp(vp(V,PP)) -->

verb(V),pp(PP).– pp(pp(P,NP)) -->

preposition(P), np(NP).

• example lexicon L0 (in Prolog):– noun(noun(flight)) --> [flight].– noun(noun(breeze)) --> [breeze].– noun(noun(trip)) --> [trip].– noun(noun(morning)) -->

[morning].– pronoun(pro(i)) --> [i].– pronoun(pro(me)) --> [me].– pronoun(pro(you)) --> [you].– pronoun(pro(it)) --> [it].– propernoun(propn(alaska)) -->

[alaska].– propernoun(propn(baltimore)) -->

[baltimore].– propernoun(propn('los angeles'))

--> [los,angeles].– propernoun(propn(chicago)) -->

[chicago].– propernoun(propn(united)) -->

[united].– propernoun(propn(american)) -->

[american].

18

9.2 Context-Free Rules and Trees

• example lexicon L0 (in Prolog):– det(det(a)) --> [a].– det(det(an)) --> [an].– det(det(the)) --> [the].– det(det(this)) --> [this].– det(det(these)) --> [these].– det(det(that)) --> [that].– verb(verb(is)) --> [is].– verb(verb(prefer)) -->

[prefer].– verb(verb(like)) --> [like].– verb(verb(need)) --> [need].– verb(verb(want)) --> [want].– verb(verb(fly)) --> [fly].– preposition(prep(from)) -->

[from].– preposition(prep(to)) --> [to].– preposition(prep(on)) --> [on].– preposition(prep(near)) -->

[near].

• query– ?- s(Tree,[i,prefer,a,morning,flight],[]).

– Tree = s(np(pro(i)),vp(verb(prefer),np(det(a),nom(

noun(morning),nom(noun(flight))))))

19

9.2 Context-Free Rules and Trees

• Derivation:?- s(Tree,[i,prefer,a,morning,flight],[]).1 1 Call: s(_436,[i,prefer,a,morning,flight],[]) ?2 2 Call: np(_1084,

[i,prefer,a,morning,flight],_1079) ? 3 3 Call: pronoun(_1478,

[i,prefer,a,morning,flight],_1079) ? s? 3 3 Exit: pronoun(pro(i),

[i,prefer,a,morning,flight],[prefer,a,morning,flight]) ?

? 2 2 Exit: np(np(pro(i)),[i,prefer,a,morning,flight],[prefer,a,morning,flight]) ?

4 2 Call: vp(_1085,[prefer,a,morning,flight],[]) ? 5 3 Call: verb(_3028,[prefer,a,morning,flight],[]) ? f5 3 Fail: verb(_3028,[prefer,a,morning,flight],[]) ? 6 3 Call: verb(_3035,

[prefer,a,morning,flight],_3030) ? s? 6 3 Exit: verb(verb(prefer),

[prefer,a,morning,flight],[a,morning,flight]) ? 7 3 Call: np(_3036,[a,morning,flight],[]) ? 8 4 Call: pronoun(_4220,[a,morning,flight],[]) ? f8 4 Fail: pronoun(_4220,[a,morning,flight],[]) ? 9 4 Call: propernoun(_4220,[a,morning,flight],[]) ? f9 4 Fail: propernoun(_4220,[a,morning,flight],[]) ? 10 4 Call: det(_4227,[a,morning,flight],_4222) ? s10 4 Exit: det(det(a),[a,morning,flight],

[morning,flight]) ?

11 4 Call: nominal(_4228,[morning,flight],[]) ?

12 5 Call: noun(_5412,[morning,flight],[]) ? f

12 5 Fail: noun(_5412,[morning,flight],[]) ?

13 5 Call: noun(_5419,[morning,flight],_5414) ? s

13 5 Exit: noun(noun(morning),[morning,flight],[flight]) ?

14 5 Call: nominal(_5420,[flight],[]) ?

15 6 Call: noun(_6601,[flight],[]) ? s

? 15 6 Exit: noun(noun(flight),[flight],[]) ?

? 14 5 Exit: nominal(nom(noun(flight)),[flight],[]) ?

? 11 4 Exit: nominal(nom(noun(morning),nom(noun(flight))),[morning,flight],[]) ?

? 7 3 Exit: np(np(det(a),nom(noun(morning),nom(noun(flight)))),[a,morning,flight],[]) ?

? 4 2 Exit: vp(vp(verb(prefer),np(det(a),nom(noun(morning),nom(noun(flight))))),[prefer,a,morning,flight],[]) ?

? 1 1 Exit: s(s(np(pro(i)),vp(verb(prefer),np(det(a),nom(noun(morning),nom(noun(flight)))))),[i,prefer,a,morning,flight],[]) ?

Tree = s(np(pro(i)),vp(verb(prefer),np(det(a),nom(noun(morning),nom(noun(flight))))))

20

9.2 Context-Free Rules and Trees

• note– this Prolog grammar can also be directly used as a generator – (as well as a parser)

• examples• ?- s(s(np(pro(i)),vp(verb(prefer),np(det(a),nom(noun(morning),nom(noun(flight)))))),Y,[]).• Y = [i,prefer,a,morning,flight]

• ?- s(s(np(pro(i)),vp(verb(prefer),np(det(a),nom(noun(morning),nom(noun(X)))))),Y,[]).• X = flight,• Y = [i,prefer,a,morning,flight] ? ;• X = breeze,• Y = [i,prefer,a,morning,breeze] ? ;• X = trip,• Y = [i,prefer,a,morning,trip] ? ;• X = morning,• Y = [i,prefer,a,morning,morning] ? ;• no

21

9.3 Sentence-Level Constructions

• Sentence– declarative

• s(s(NP,VP)) --> np(NP), vp(VP). • Ex:

– I prefer a morning flight

– imperative

• s(s(VP)) --> vp(VP). • commands and suggestions in the ATIS domain• examples

– Show me the lowest fare– List all flights from Burbank to Denver– Please list the flights from Charlotte to Long Beach arriving after lunch

time

22

9.3 Sentence-Level Constructions

• Sentence– yes-no questions

• s(s(Aux,NP,VP)) --> aux(Aux), np(NP), vp(VP). • subject-aux inversion • Exs:

– Do any of these flights have stops? No– Can you give me the same information for United? Yes (not really yes-no)– Is … ?– Will … ?

– wh-questions• s(s(NP,VP)) --> whNP(NP), vp(VP). % subject wh-question• s(s(NP,Aux,VP)) --> whNP(NP), aux(Aux), vp(VP). % object wh-question

• Exs: – What airlines fly from Burbank to Denver?– Which flights serve breakfast?– What flights do you have from Burbank to Tacoma Washington?– *What flights you have from Burbank to Tacoma Washington?

23

9.4 The Noun Phrase

• Word Sequence• pre-nominal modifiers

– head noun• post-nominal modifiers

• Pre-nominal modifiers– determiners

• a stop the flights that fare this flight• those flights any flights some flights

– no determiner• plural nouns

– examples » show me flights from SF to Denver» *show me flight from SF to Denver

• mass nouns– examples: water, snow, breakfast, lunch, dinner– Test: much ___

24

9.4 The Noun Phrase

• Word Sequence– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Pre-determiners: e.g. all– all the flights all flights

• Post-determiners:– cardinals

• the two friends a one stop flight

– ordinals• the first one the second leg• the next day the last flight• the other American fare

25

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-determiners– quantifiers

• (the) many fares (a) few flights (the) several stops

– adjectives• a first-class fare a non-stop flight• the longest layover the earliest lunch flight

– adverb + adjective• the least expensive fare

26

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Prenominal NP DCG rule– np --> (det), (card), (ord), (quant), (ap), nominal.– (denote optionality)– cannot use pure brackets in Prolog– shorthand for 32 rules– np --> det, card, ord, quant, ap, nominal.– np --> card, ord, quant, ap, nominal.– np --> ord, quant, ap, nominal.– np --> det, ord, quant, ap, nominal.– np --> det, card, quant, ap, nominal.– np --> card, quant, ap, nominal.– np --> det, ord, quant, ap, nominal.– np --> quant, ap, nominal.– etc …

27

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Prenominal NP DCG rule– np --> (det), (card), (ord), (quant), (ap), nominal.– np(np([D,C,O,Q,AP],Nom)) -->

• (det(D)|[]), • (card(C) |[]), • (ord(O)|[]), • (quant(Q)|[]), • (ap(AP)|[]), • nominal(Nom).

– |[]- means OR the empty terminal sequence (i.e. nothing)

28

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-nominal Modifiers:– prepositional phrases (PP)

• examples – all flights from Cleveland– all flights from Cleveland to Newark– arrival in San Jose before seven p.m.– a reservation on flight six oh six from Tampa to Montreal

– non-finite clauses • example: all flights [arriving after eleven a.m.]

– relative clauses • example: a flight [that serves breakfast]

• Post-nominal NP DCG rule (1 to 3 prepositional phrases)– nominal --> nominal, pp, (pp|[]), (pp|[]).

29

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-nominal Modifiers:– non-finite clauses

• gerundive (-ing)• examples:

– all flights [arriving after eleven a.m. ]– any of those [leaving on Thursday]– flights [arriving within thirty minutes of each other]– flights [leaving Tucson next Monday]

• Post-nominal NP rule: (gerundives)– nominal --> nominal, gerundvp.– gerundvp --> gerundv, np | gerundv, pp | gerundv | gerundv, np, pp.– gerundv --> [being]| [preferring] | [arriving] | [leaving].

30

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-nominal Modifiers:– non-finite clauses

• infinitival VP• example:

– the last flight [to arrive in Boston]

• past-participle VP• examples:

– I need to have dinner served– Which is the aircraft used by this flight?

• Post-nominal NP DCG rule: (infinitival)– nominal --> nominal, [to], infvp.– infvp --> infv, np | infv, pp | infv | infv, np, pp.– infv --> [arrive] | [leave].

31

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-nominal Modifiers:– relative clauses

• examples: (subject)– a flight [that serves breakfast]– *a flight [serves breakfast] (as NP)– flights [that leave in the morning]– the United flight [that arrives in San Jose around ten p.m.]– the one [that leaves at ten thirty five]

• examples: (object)– the earliest American Airlines flight [that I can get]– the earliest American Airlines flight [I can get]

• Post-nominal NP rule: (subject relative clauses)– nominal --> nominal, relclause.– relclause --> ( [who] | [that]), vp.– plus additional rules for object relative clauses allowing optional relative pronoun

semantics (subject relative clause)λx.serves(x,breakfast)

(λx.serves(x,breakfast))(flight)

semantics (object relative clause)λx.possible(get(i,x))

32

9.4 The Noun Phrase

• Word Sequence:– pre-determiners

• determiner– post-determiners

– head noun• post-nominal modifiers

• Post-nominal Modifiers– may be combined

• examples – a flight [PP from Phoenix] [PP to Detroit] [CP that serves breakfast]– I need a flight [PP to Seattle][S leaving from Baltimore ][S making a

stop in Minneapolis]– a friend [S living in Denver ] [CP that would like to visit me here [PP in

Washington DC]]

33

9.5 Coordination

• NP coordination with and (or, but):– examples:

• Please repeat [the flights] and [the costs]• I need to know [the [aircraft] and [flight numbers]]• I would like to fly from Denver stopping [PP in [Pittsburgh] and [Atlanta]]

• NP coordination DCG rule:– np --> np, [and], np.

• other phrases: VP, PP, S– can be a test for constituency– examples:

• What flights do you have [leaving Denver] and [arriving in San Francisco]?• planes leaving from [Denver] and [San Francisco]• *planes arriving [in San Francisco] and [flight numbers]• [I’m interested in a flight from Dallas to Washington] and [I’m also interested in going to

Baltimore]

• VP and S coordination DCG rules:– vp --> vp, [and], vp.– s --> s, [and], s.

34

Next Time

• Finish Chapter 9

• Start Chapter 10: Parsing with Context-Free Grammars