cs4705 part of speech tagging - columbia universitykathy/nlp/2019/classslides/class8-pos/cla… ·...
TRANSCRIPT
CS4705PartofSpeechtagging
9/30
/19
1 Some slides adapted from: Dan Jurafsky, Julia Hirschberg, Jim Martin
Announcements• Readingfortoday:• Readingfornext0me:• Weareturningtosyntax.Today:part-of-speechtagging
• Homework2isout
2
Gardenpathsentences• Theolddogthefootstepsoftheyoung.
• Thehorseracedpastthebarnfell.
• Theco5onclothingismadeofgrowsinMississippi.
3
GardenpathsentencesN• Theolddog|thefootstepsoftheyoung.
• Thehorseracedpastthebarnfell.
• Theco5onclothingismadeofgrowsinMississippi. 4
GardenpathsentencesV• Theolddog|thefootstepsoftheyoung.
• Thehorseracedpastthebarnfell.
• Theco5onclothingismadeofgrowsinMississippi. 5
Gardenpathsentences• Theolddogthefootstepsoftheyoung.
VBD• Thehorseracedpastthebarn|fell.
• Theco5onclothingismadeofgrowsinMississippi. 6
Gardenpathsentences• Theolddogthefootstepsoftheyoung.
VBNVBD• Thehorseracedpastthebarn|fell.
• Theco5onclothingismadeofgrowsinMississippi. 7
Gardenpathsentences• Theolddogthefootstepsoftheyoung.
• Thehorseracedpastthebarnfell.
• Theco5onclothingismadeofgrowsinMississippi.
8
9
10
Whatisawordclass?• Wordsthatsomehow‘behave’alike:
• Appearinsimilarcontexts• Performsimilarfunc0onsinsentences• Undergosimilartransforma0ons
• 9(orso)tradi0onalpartsofspeech• Noun,verb,adjec0ve,preposi0on,adverb,ar0cle,interjec0on,pronoun,conjunc0on,
11
POSexamples• N nounchair,bandwidth,pacing• V verb study,debate,munch• ADJadjec0ve purple,tall,ridiculous• ADVadverb unfortunately,slowly,• P preposi0onof,by,to• PROpronoun I,me,mine• DETdeterminer the,a,that,those
12
POSTagging:DeDinition• Theprocessofassigningapart-of-speechorlexicalclassmarkertoeachwordinacorpus:
13
the koala put the keys on the table
WORDS TAGS
N V P DET
WhatisPOStagginggoodfor?• IsthefirststepofavastnumberofCompLingtasks• Speechsynthesis:
• Howtopronounce“lead”?• INsult inSULT• OBject obJECT• OVERflow overFLOW• DIScount disCOUNT• CONtent conTENT
• Parsing• NeedtoknowifawordisanNorVbeforeyoucanparse
• Wordpredic0oninspeechrecogni0on• Possessivepronouns(my,your,her)followedbynouns• Personalpronouns(I,you,he)likelytobefollowedbyverbs
• MachineTransla0on
14
15
16
Openandclosedclasswords• Closedclass:arela0velyfixedmembership
• Preposi0ons:of,in,by,…• Auxiliaries:may,can,willhad,been,…• Pronouns:I,you,she,mine,his,them,…• Usuallyfunc0onwords(shortcommonwordswhichplayaroleingrammar)
• Openclass:newonescanbecreatedallthe0me• Englishhas4:Nouns,Verbs,Adjec0ves,Adverbs• Manylanguageshaveall4,butnotall!• InLakhotaandpossiblyChinese,whatEnglishtreatsasadjec0vesactmorelikeverbs. 17
Openclasswords• Nouns
• Propernouns(ColumbiaUniversity,NewYorkCity,ElsbethTurcan,MetropolitanTransitCenter).Englishcapitalizesthese.
• Commonnouns(therest).Germancapitalizesthese.• Countnounsandmassnouns
• Count:haveplurals,getcounted:goat/goats,onegoat,twogoats• Mass:don’tgetcounted(fish,salt,communism)(*twofishes)
• Adverbs:tendtomodifyac0onsorpredicates• Unfortunately,Johnwalkedhomeextremelyslowlyyesterday• Direc0onal/loca0veadverbs(here,home,downhill)• Degreeadverbs(extremely,very,somewhat)• Manneradverbs(slowly,slinkily,delicately)
• Verbs:• InEnglish,havemorphologicalaffixes(eat/eats/eaten)• Ac0ons(walk,ate)andstates(be,exude)
18
• Manysubclasses,e.g.• eats/V⇒eat/VB,eat/VBP,eats/VBZ,ate/VBD,eaten/VBN,ea0ng/VBG,...
• Reflectmorphologicalform&syntac0cfunc0on
19
Howdowedecidewhichwordsgoinwhichclasses?
• Nounsdenotepeople,placesandthingsandcanbeprecededbyar0cles?But…
Mytypingisverybad.*TheMarylovesJohn.
• Verbsareusedtorefertoac0ons,processes,states• ButsomeareclosedclassandsomeareopenIwillhaveemailedeveryonebynoon.• Adverbsmodifyac0ons• IsMondayatemporaladverboranoun?
20
21
DeterminingPart-of-Speech• Ablueseat/Achildseat:nounoradj?
• Sometests• Syntac0c
• Ablueseat Achildseat• Averyblueseat *Averychildseat• Thisseatisblue *Thisseatischild
• Morphological• Bluer *childer
• Blueisanadjec0ve,butchildisanoun 22
DeterminingPart-of-Speech• Preposi0onorpar0cle?A. Hethrewoutthegarbage.B. Hethrewthegarbageoutthedoor.
C. HethrewthegarbageoutD. *Hethrewthegarbagethedoorout.
• outinAisapar0cle,inBisapreposi0on 23
ClosedClassWords• Idiosyncra0c• Closedclasswords(Prep,Det,Pron,Conj,Aux,Part,Num)areeasier,sincewecanenumeratethem….but• Partvs.Prep
• Georgeeatsuphisdinner/Georgeeatshisdinnerup.
• Georgeeatsupthestreet/*Georgeeatsthestreetup.
• Ar0clescomein2flavors:definite(the)andindefinite(a,an)
24
POStagging:Choosingatagset• TodoPOStagging,needtochooseastandardsetoftagstoworkwith
• Couldpickverycoarsetagsets• N,V,Adj,Adv.
• BrownCorpus(Francis&Kucera‘82),1Mwords,87tags
• PennTreebank:hand-annotatedcorpusofWallStreetJournal,1Mwords,45-46tags• Commonlyused• setisfinergrained,
• Evenmorefine-grainedtagsetsexist25
PennTreeBankPOSTagset
26
UsingtheUPenntagset• The/DTgrand/JJjury/NNcommmented/VBDon/INa/DTnumber/NNof/INother/JJtopics/NNS./.
• Preposi0onsandsubordina0ngconjunc0onsmarkedIN(“although/INI/PRP..”)
• Exceptthepreposi0on/complemen0zer“to”isjustmarked“to”.
27
POSTagging• WordsoqenhavemorethanonePOS:back• Thebackdoor=JJ• Onmyback=NN• Winthevotersback=RB• Promisedtobackthebill=VB
• ThePOStaggingproblemistodeterminethePOStagforapar0cularinstanceofaword. 28
These examples from Dekang Lin
HowdoweassignPOStagstowordsinasentence?WhatinformationdoyouthinkwecouldusetoassignPOSinthefollowingsentences?
• Timeflieslikeanarrow.• Time/[V,N]flies/[V,N]like/[V,Prep]an/Detarrow/N
• Time/Nflies/Vlike/Prepan/Detarrow/N• Fruit/Nflies/Nlike/Va/DETbanana/N• Fruit/Nflies/Vlike/Prepa/DETbanana/N• The/Detflies/Nlike/Va/DETbanana/N 29
HowhardisPOStagging?Measuringambiguity
30
Canyouthinkofsevensentenceswhereineachone“well”isusedwithadifferentpartofspeech?
31
PotentialSourcesofDisambiguation• ManywordshaveonlyonePOStag(e.g.is,Mary,very,smallest)
• Othershaveasinglemostlikelytag(e.g.a,dog)• Buttagsalsotendtoco-occurregularlywithothertags(e.g.Det,N)
• Inaddi0ontocondi0onalprobabili0esofwordsP(w1|wn-1),wecanlookatPOSlikelihoodsP(t1|tn-1)todisambiguatesentencesandtoassesssentencelikelihoods
32
HiddenMarkovModelTagging• UsinganHMMtodoPOStagging
• AspecialcaseofBayesianinference
• Relatedtothe“noisychannel”modelusedinMT,ASRandotherapplica0ons
9/30
/19
33
POStaggingasasequenceclassiDicationtask• Wearegivenasentence(an“observa0on”or“sequenceofobserva0ons”)• Secretariatisexpectedtoracetomorrow
• Whatisthebestsequenceoftagswhichcorrespondstothissequenceofobserva0ons?
• Probabilis0cview:• Considerallpossiblesequencesoftags• Choosethetagsequencewhichismostprobablegiventheobserva0onsequenceofnwordsw1…wn.
9/30
/19
34
GettingtoHMM• Outofallsequencesofntagst1…tnwantthesingletagsequencesuchthatP(t1…tn|w1…wn)ishighest.
• Hat^means“oures0mateofthebestone”
• Argmaxxf(x)means“thexsuchthatf(x)ismaximized”
9/30
/19
35
GettingtoHMM• Thisequa0onisguaranteedtogiveusthebesttagsequence
• Intui0onofBayesianclassifica0on:
• UseBayesruletotransformintoasetofotherprobabili0esthatareeasiertocompute
9/30
/19
36
UsingBayesRule
9/30
/19
37
Likelihoodandprior
n
Twokindsofprobabilities(1)• Tagtransi0onprobabili0esp(ti|ti-1)
• Determinerslikelytoprecedeadjsandnouns• That/DTflight/NN• The/DTyellow/JJhat/NN• SoweexpectP(NN|DT)andP(JJ|DT)tobehigh• ButP(DT|JJ)tobe:
• ComputeP(NN|DT)bycoun0nginalabeledcorpus:
Twokindsofprobabilities(2)• Wordlikelihoodprobabili0esp(wi|ti)
• VBZ(3sgPresverb)likelytobe“is”• ComputeP(is|VBZ)bycoun0nginalabeledcorpus:
9/30
/19
40
AnExample:theverb“race”
• Secretariat/NNPis/VBZexpected/VBNto/TOrace/VBtomorrow/NR
• People/NNScon0nue/VBto/TOinquire/VBthe/DTreason/NNfor/INthe/DTrace/NNfor/INouter/JJspace/NN
• Howdowepicktherighttag?
9/30
/19
41
Disambiguating“race”
9/30
/19
42
• P(NN|TO)=.00047• P(VB|TO)=.83• P(race|NN)=.00057• P(race|VB)=.00012• P(NR|VB)=.0027• P(NR|NN)=.0012• P(VB|TO)P(NR|VB)P(race|VB)=.00000027• P(NN|TO)P(NR|NN)P(race|NN)=.00000000032• Sowe(correctly)choosetheverbreading,
9/30
/19
43
SummaryPartsofspeech• What’sPOStagginggoodforanyhow?• Tagsets• Sta0s0csandPOStagging• Next0me:
• HMMTagging
44
Homework2
45