lecture 7: language structure: grammarcs6501: nlp 4 a better model cs6501: nlp 5 language is...
TRANSCRIPT
![Page 1: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/1.jpg)
Lecture 7:Language Structure:
Grammar
Kai-Wei ChangCS @ UCLA
Couse webpage: https://uclanlp.github.io/CS269-17/
1ML in NLP
![Page 2: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/2.jpg)
Basic sentence structure
CS6501: NLP 2
![Page 3: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/3.jpg)
A Markov Model
v I eat shshi; I eat meat; you eat banana…
vGreat, it covers many sentences
CS6501: NLP 3
![Page 4: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/4.jpg)
Words take different arguments
v [Good] I eat sushiv [Bad] I run sushiv [Bad] I give sushi
v Intransitive verbs (sleep): no objectv Transitive verbs (eat): take one direct objectv Ditransitive verbs (give): take an additional
indirect object.
CS6501: NLP 4
![Page 5: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/5.jpg)
A better model
CS6501: NLP 5
![Page 6: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/6.jpg)
Language is recursive
CS6501: NLP 6
Adjectivescanmodifynouns.Wecanhaveunlimitedmodifiers(intheory)
![Page 7: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/7.jpg)
We know how to model the simple one
CS6501: NLP 7
![Page 8: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/8.jpg)
Recursion can be more complex
CS6501: NLP 8
Wecanhaveanothernounphraseinpreposition
![Page 9: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/9.jpg)
Syntactic parsing
v Idea: model language as a recursive generating processvOften use a tree structurevDecompose a sentence
CS6501: NLP 9
![Page 10: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/10.jpg)
What is grammar?
vA compact way to define and describe the structure of sentences
vWhy we need grammar?vNumber of C++ programs?
CS6501: NLP 10
976pages.
C++ standard (2014)
ISO/IEC 14882:2014
1358 pages
![Page 11: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/11.jpg)
Can we define a program that generates all English sentences?
CS6501: NLP 11FromJuliaHockenmaier,IntrotoNLP
![Page 12: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/12.jpg)
What is sentence structure
vSentence structure is hierarchicalvA sentence consists of phrases (or
constituents)
CS6501: NLP 12
![Page 13: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/13.jpg)
Can have complex constituents
CS6501: NLP 13
![Page 14: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/14.jpg)
Can have complex constituents
vSyntactically, constituents behave like simple ones
CS6501: NLP 14
![Page 15: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/15.jpg)
Constituency
vGroups of words that behave as a single unit or phrasevE.g., Noun phrases: the man, a girl with glassesvPrepositional phrases: with classes, on a tablevVerb phrase: eat sushi, sleep, sleep soundly
vPhrases has a head:vOther parts called dependentsvE.g, the man, a girl with glasses
CS6501: NLP 15
![Page 16: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/16.jpg)
Properties of constituents
vSubstitutionvHe talks [in class] ⇒ He talks [there]
v It can move around in a sentencevHe talks [in class] ⇒ [In class], he talks.
vCan be used as an answer:vWhere does he talk? – [In class]
CS6501: NLP 16
![Page 17: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/17.jpg)
Types of dependencies
vPhrases has a head:vOther parts called dependentsvE.g, the man, a girl with glasses
vDependents can be arguments or adjunctsvArguments are obligatory
vE.g., [John] likes [Mary]
vAdjuncts are optionalvE.g., John runs [fast] vAdverbs, PPs, Adjectives…
CS6501: NLP 17
Allargumentshavetobepresentandcannotbeoccupiedmultipletimes
Canbeanarbitrarynumberofadjuncts
![Page 18: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/18.jpg)
How to represent the structure
CS6501: NLP 18
![Page 19: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/19.jpg)
Dependency Trees
vDependency grammar describe the structure of sentences as a graph (tree)vNodes represent wordsvEdges represent dependencies
CS6501: NLP 19
![Page 20: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/20.jpg)
Phrases structure trees
vCan be modeled by Context-free grammars
CS6501: NLP 20
![Page 21: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/21.jpg)
Context-free grammars
CS6501: NLP 21
![Page 22: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/22.jpg)
Parse tree defined by CFG
CS6501: NLP 22
![Page 23: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/23.jpg)
Generate sentences by CFG
CS6501: NLP 23
![Page 24: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/24.jpg)
Example: Noun Phrases
CS6501: NLP 24
![Page 25: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/25.jpg)
Example: verb phrase
CS6501: NLP 25
![Page 26: Lecture 7: Language Structure: GrammarCS6501: NLP 4 A better model CS6501: NLP 5 Language is recursive CS6501: NLP 6 Adjectives can modify nouns. We can have unlimited modifiers (in](https://reader033.vdocuments.mx/reader033/viewer/2022053006/5f09f7507e708231d4295d06/html5/thumbnails/26.jpg)
Sentences
CS6501: NLP 26