july 9, 2003acl 2003 1 an improved pattern model for automatic ie pattern acquisition kiyoshi sudo...
Post on 20-Dec-2015
219 views
TRANSCRIPT
July 9, 2003 ACL 20031
An Improved Pattern Model for Automatic IE Pattern Acquisition
Kiyoshi Sudo
Satoshi Sekine
Ralph Grishman
New York University
July 9, 2003ACL 20032
Automatic Pattern Acquisition
The cost of manual construction of extraction patterns is very high.
The cost of preparation of annotated data for supervised learning is still high.
The recent trend of the researches on pattern acquisition is un- (semi-) supervised learning.
July 9, 2003ACL 20033
Information Extraction
Identifying entities from source text and mapping from source text to pre-defined table.
“A smiling Palestinian suicide bomber triggered a massive explosion in the heavily policed heart of downtown Jerusalem today, …”
Date:Location:Perpetrator:
downtown Jerusalem
A … suicide bomber
today
July 9, 2003ACL 20034
Local Context
Local contexts provides a useful information to identify entities.
“A smiling Palestinian suicide bomber triggered a massive explosion in the heavily policed heart of downtown Jerusalem today, …”
Date:Location:Perpetrator:
downtown Jerusalem
A … suicide bomber
today
July 9, 2003ACL 20035
Extraction Pattern
Generalize each instance of entity and its local context into an extraction pattern.
“A smiling Palestinian suicide bomber triggered a massive explosion in the heavily policed heart of downtown Jerusalem today, …”
triggered a massive explosion<person>NE category
Association Rule Perpetrator:
July 9, 2003ACL 20036
Dependency Tree forPattern Model
Introducing syntax (dependency tree) clarify the relation of arguments with predicates.
triggered
a massive explosionA smiling Palestiniansuicide bomber
heart
heavily policed downtown Jerusalem
today
SBJOBJ
ADV
IN
July 9, 2003ACL 20037
Extraction Pattern models
Predicate-Argument model (Yangarber et al. 2000)
– Based on direct relation with a predicate
Chain model (Sudo et al. 2001)
– Based on a chain of modifiers of a predicate
triggered
<person> explosion
triggered
<person>
triggered
heart
downtown Jerusalem
July 9, 2003ACL 20038
Predicate-Argument model
Predicate-Argument model is based on the direct relation of a predicate and its arguments.
triggered
a massive explosion<person> heart
heavily policed downtown Jerusalem
<date>
SBJOBJ ADV
IN
July 9, 2003ACL 20039
Chain model
Chain model can capture the chain of modifier with an arbitrary depth in the tree, regardless phrasal or clausal boundary.
triggered
a massive explosion<person> heart
heavily policed <location>
<date>
SBJOBJ ADV
IN
(Sudo et al. 2001) reported 5% gain in recall with same level of precision over Predicate-Argument model.
July 9, 2003ACL 200310
Problem
Chain model contains only one node at each level of the tree.
triggered
a massive explosion<person> heart
heavily policed downtown Jerusalem
<date>
SBJOBJ ADV
IN
July 9, 2003ACL 200311
Problem
Lack of the context can make a pattern too general, causing a false match on irrelevant text.
triggered
a national financial crisisthe Mexican peso last week
SBJOBJ ADV
“The Mexican peso was devalued and triggered a national financial crisis last week.”
July 9, 2003ACL 200312
Subtree model
Generalization of Predicate-Argument and Chain model
– Any connected subtree of a
dependency tree will be considered
as a candidate of extraction pattern.
– Give reliable contexts as Predicate-Argument
model does– Capable to capture long-distance relationship
in dependency tree
July 9, 2003ACL 200313
Subtree model
Subtree model can provide more relevant contexts, as well as have a flexibility in traversing arbitrary depth in the tree.
triggered
a massive explosion<person> heart
heavily policed downtown Jerusalem
<date>
SBJOBJ ADV
IN
July 9, 2003ACL 200314
Experiment
Entity Extraction task– Identify if an NE instance is involved in scenario or not
Management Succession– Person, Organization, Post (Position_Title)
Murder Arrest– Arresting Agency (Organization), Suspect (Person), Charge
– Source: Japanese newspaper 117,109 articles (Mainichi 1995)– Test: accumulated from Mainichi 1994
Succession 148 documents Arrest 205 documents
July 9, 2003ACL 200315
Acquisition Method
The target scenario is specified by TREC-like narrative description
– “Management Succession at the level of executives of a company. The topic of interest should not be limited to the promotion inside the company mentioned, but also includes hiring executives from outside the company of their resignation.” [Translated from Japanese]
Preprocessing– Dependency Analysis, NE-tagging
Document RetrievalR
July 9, 2003ACL 200316
Acquisition Method
Count all possible subtrees in R– subtree-mining algorithm (Zaki et al. 2002)
– make a Pattern List of those that conform the pattern model Rank each subtree
R
iii df
Ntfscore log
For each subtree i,
number of times subtree ioccurred in the documents in R
July 9, 2003ACL 200317
Acquisition Method
Count all possible subtrees in R– subtree-mining algorithm (Zaki et al. 2002)
– make a Pattern List of those that conform the pattern model Rank each subtree
R
iii df
Ntfscore log
For each subtree i,
number of documents inthe source which contain subtree i
July 9, 2003ACL 200318
Overlapping patterns
Pattern List contains many overlapping patterns
– (19) (<organization> report)((<organization>-wa) Happyo_suru)
– (480) (<organization> report that … be appointed)((<organization>-wa) (Shunin_suru-to) Happyo-suru)
works as a weight on patterns with more relevant context
[Translated from Japanese]
July 9, 2003ACL 200319
comparison
July 9, 2003ACL 200320
Unsupervised Parameter Tuning
Unsupervised text classification task by pattern matching
– retrieved … 300 documents retrieved– random … 300 randomly selected
– For each precision-recall curve for , calculate the area that the curve covers.
Pearson correlation coefficient– rp = 0.80 with 2% confidence
July 9, 2003ACL 200321
ExtractionPerformance
July 9, 2003ACL 200322
Lessons learned
Subtree vs. Chain– Too-general patterns got more penalized for
Subtree model Penalize by Inversed Document Frequency (Subtree, Chain) More scenario-specific patterns got promoted (Subtree)
July 9, 2003ACL 200323
Lessons learned
Subtree vs. Predicate-Argument– Patterns with nominalized predicates
Extraction patterns for headlines e.g. (promotion of <post> <person>)
((<person> <post>-no) Shokaku)
– Noun phrase patterns with chain of modifiers e.g. (<post> with ministerial authority)
(((Daihyoken-no (Aru- (<post>)))
[Translated from Japanese]
July 9, 2003ACL 200324
Lessons to be learned
Enhanced scoring function by modern IR technique.– Some techniques directly helps pattern acquisition
e.g. relevance feedback– However, note the crucial difference between Pattern
acquisition and IR Same pattern does not appear twice in a document.
Generic variable instead of sticking to Named Entity categories as place holder.
– How robust can a pattern be without semantic restriction?
July 9, 2003ACL 200325
Conclusion
We proposed Subtree model as a generalization of– Predicate-Argument model– Chain model
Subtree model patterns overly performed better than other models in Entity Extraction tasks.
Scoring function needs a special consideration for overlapping patterns.
Unsupervised parameter tuning by text classification task.