CSA2050:Introduction to Computational
Linguistics
Part of Speech (POS) Tagging I
Introduction Tagsets Approaches
April 2005 CLINT Lecture IV 2
Acknowledgment
Most slides taken from Bonnie Dorr’s course notes: www.umiacs.umd.edu/~bonnie/courses/cmsc723-03
In turn based on Jurafsky & Martin Chapter 8
Bibliography
R. Weischedel , R. Schwartz , J. Palmucci , M. Meteer , L. Ramshaw, Coping with Ambiguity and Unknown Words through Probabilistic Models, Computational Linguistics 19.2, pp 359--382,1993 [pdf]
Samuelsson, C., Morphological tagging based entirely on Bayesian inference, in 9th Nordic Conference on Computational Linguistics, NODALIDA-93, Stockholm, 1993. (see [html])
A. Ratnaparkhi, A maximum entropy model for part of speech tagging. Proceedings of the Conference on Empirical Methods in Natural Language, 1996 Processing [pdf]
April 2005 CLINT Lecture IV 4
Outline
The tagging task Tagsets Three different approaches
April 2005 CLINT Lecture IV 5
Definition: PoS-Tagging
“Part-of-Speech Tagging is the process of assigning a part-of-speech or other lexical class marker to each word in a corpus” (Jurafsky and Martin)
thegirlkissedtheboyonthecheek
WORDSTAGS
NVPDET
April 2005 CLINT Lecture IV 6
Motivation
Corpus analysis of tagged corpora yields useful information
Speech synthesis — pronunciation CONtent (N) vs. conTENT (Adj)
Speech recognition — word class-based N-grams predict category of next word.
Information retrieval stemming selection of high-content words
Word-sense disambiguation
English Parts of Speech
1. Pronoun: any substitute for a noun or noun phrase
2. Adjective: any qualifier of a noun3. Verb: any action or state of being4. Adverb: any qualifier of an adjective
verb5. Preposition: any establisher of relation
and syntactic context6. Conjunction: any syntactic connector7. Interjection: any emotional greeting (or
"exclamation"),
April 2005 CLINT Lecture IV 8
Tagsets: how detailed?
Swedish SUC 25
Penn Treebank 46
German STTS 50
Lancaster BNC 61
Lancaster Full 146
April 2005 CLINT Lecture IV 9
Penn Treebank Tagset
PRPPRP$
April 2005 CLINT Lecture IV 10
Example of Penn Treebank Tagging of Brown Corpus SentenceThe/DT grand/JJ jury/NN commented/VBD on/IN a/DT number/NN of/IN other/JJ topics/NNS ./.
VB DT NN .Book that flight .
VBZ DT NN VB NN ?Does that flight serve dinner ?
2 Problems
Multiple tags for the same word Unknown words
April 2005 CLINT Lecture IV 12
Multiple tags for the same word
1. He can can a can.
2. I can light a fire and you can open a can of beans. Now the can is open, and we can eat in the light of the fire.
3. Flying planes can be dangerous.
April 2005 CLINT Lecture IV 13
Multiple tags for the same word
Words often belong to more than one word class: this This is a nice day = PRP (pronoun) This day is nice = DT (determiner) You can go this far = RB (adverb)
Many of the most common words (by volume of text) are ambiguous
April 2005 CLINT Lecture IV 14
How Hard is the Tagging Task?
In the Brown Corpus 11.5% of word types are ambiguous 40% of word tokens are ambiguous
Most words in English are unambiguous. Many of the most common words are
ambiguous. Typically ambiguous tags are not equally
probable.
April 2005 CLINT Lecture IV 15
Word Class Ambiguity(in the Brown Corpus)
Unambiguous (1 tag): 35,340 types
Ambiguous (2-7 tags): 4,100 types
.
2 tags 3,760
3 tags 264
4 tags 61
5 tags 12
6 tags 2
7 tags 1(Derose, 1988)
April 2005 CLINT Lecture IV 16
3 Approaches to Tagging
1. Rule-Based Tagger: ENCG Tagger(Voutilainen 1995,1999)
2. Stochastic Tagger: HMM-based Tagger
3. Transformation-Based Tagger: Brill Tagger(Brill 1995)
Unknown Words
1. Assume all unknown word is ambiguous amongst all possible tagsAdvantage: simplicity
Disadvantage: ignores the fact that unknown words are unlikely to be closed class
2. Assume that probability distribution of unknown words is same as words that have been seen just once.
3. Make use of morphological information
Combining Features
The last method makes use of different features, e.g. ending in -ed (suggest verb) or initial capital (suggests proper noun).
Typically, a given tag is correlated with a combination of such features. These have to be incorporated into the statistical model.
Combining Tag-Predicting Features in Unknown Words
HMM Models Weischedel et. al. (1993): for each feature f and
tag t (e.g. proper noun) build a probability estimator p(f|t). Assume independence and multiply probabilities together
Samuelsson (1993), rather than preselecting features, considers all possible suffixes up to length 10 as features for predicting tags
Combining Tag-Predicting Features in Unknown Words
Maximum Entropy (ME) Models. A ME model is a classifier which assigns a class to
an observation by computing a probability from an exponential function of a weighted set of features of the observation
An MEMM uses the Viterbi Algorithm to extend the application of ME to labelling a sequence of observations.
For further details see Ratnaparkhi (1996)
Summary
External parameters to the tagging task are (i) the size of the chosen tagset and (ii) the coverage of the lexicon which gives possible tags to words.
Two main problems: (i) disambiguation of tags and (ii) dealing with unknown words
Several methods are available for dealing with (ii): HMMs and MEMMs