tree-edit crfs for rte
Post on 31-Dec-2015
45 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
Tree-edit CRFs for RTE
Mengqiu Wang and Chris Manning
2
Tree-edit CRFs for RTE
• Extension to McCallum et al. UAI2005 work on CRFs for finite-state String Edit Distance
• Key attractions:• Models the transformation of dependency parse trees
(thus directly models syntax), unlike McCallum et al. ’05, which only models word strings
• Discriminatively trained
3
TE-CRFs model in details
• First of all, let’s look at the correspondence between alignment (with constraints) and edit operations
BushNNP
person
metVBD
FrenchJJ
location
presidentNN
Jacques ChiracNNP
person
whoWP
qword
leaderNN
isVB
theDT
FranceNNP
location
Q: A:$
root$
root
root
subj obj
det of
root
subj with
nmod
nmod
substitute
substitute
substitute
substitute
delete
insert
Fancy
substitute
5
TE-CRFs model in details• Each valid tree edit operation sequence that
transforms one tree into the other corresponds to an alignment. A tree edit operation sequence is models as a transition sequence among a set of states in a FSM
S1 S2
S3D, S, I
D, S, I D, S, I
D, E, I
D, S, I
D, S, I
D, S, I
substitute delete substitute substitute insert substitute
S1 S2 S1 S1 S1 S3 S1
S2 S3 S2 S1 S2 S1 S3
S3 S2 S1 S3 S3 S2 S2
… … … …… ……
6
FSM
This is for one edit operation sequence
substitute delete substitute substitute insert substitute
S1 S2 S1 S1 S1 S3 S1
S2 S3 S2 S1 S2 S1 S3
S3 S2 S1 S3 S3 S2 S2
… … … …… ……
delete substitute substitute substitute insert substitute
S1 S2 S1 S1 S1 S3 S1… … … …… ……
substitute delete substitute substitute substitute insert
S1 S2 S1 S1 S1 S3 S1… … … …… ……
substitute substitute delete substitute insert substitute
S1 S2 S1 S1 S1 S3 S1… … … …… ……
There are many other valid edit sequences
7
FSM cont.
S1 S2
S3D, S, I
D, S, I D, S, I
D, S, I
D, S, I
D, S, I
D, S, I
Start Stop
ε ε
S1 S2
S3D, S, I
D, S, I D, S, I
D, S, I
D, S, I
D, S, I
D, S, I
Positive State Set
Negative State Set
εε
8
FSM transitions
S3S2
S1 S1S3
S2
S2
Start
S2 S3 S3S3 S1
S1S2 S2
S2S2 S1
S2S1
S3S3
S3
… …… …
S2… …
…
…
…
…
…
Stop
S3S2
S1 S1S3
S2
S2S2 S3 S3
S3 S1
S1S2 S2
S2S2 S1
S2S1
S3S3
S3
… …… …
S2
… …
…
…
…
…
…
Positive State Set
Negative State Set
9
What is the semantic interpretation of the FSM states?
• At this moment since all the states in the FSM are all fully-connected, it’s unclear what they mean. We fix the number of states to 3, and experiments shows that setting it to 1 or 6 hurts performance.
• We are running new experiments with more meaningfully designed FSM topologies, e.g., each states deterministically corresponds to a particular edit operation.
10
Parameterization
S1 S2substitute
positive or negative
positive and negative
11
Training using EM
E-stepM-step
Using L-BFGS
Jensen’s Inequality
12
Features for RTE
• Substitution• Same --Word/WordWithNE/Lemma/NETag/Verb/Noun/Adj/Adv/Other• Sub/MisSub -- Punct/Stopword/ModalWord• Antonym/Hypernym/Synonym/Nombank/Country• Different – NE/Pos• Unrelated words
• Delete• Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/
Conditional/If• Insert
• Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/Conditional/If
• Tree• RootAligned/RootAlignedSameWord• Parent,Child,DepRel triple match/mismatch
• Date/Time/Numerical• DateMismatch, hasNumDetMismatch, normalizedFormMismatch
13
Tree-edit CRFs for Textual Entailment
• Preliminary results• Trained on RTE2 dev, tested on RTE2 test.• model taken after 50 EM iterations• acc:0.6275, map:0.6407
• SUM, acc=0.675• QA, acc=0.64• IR, acc=0.615• IE, acc=0.58
14
Work in progress
• Implementing a unordered tree-edit algorithm, which would allow swapping of sub-trees
• Use Stanford Parser dependency structure. Need to getting rid of cycles in CollapsedDependencyGraph (almost there, only have a few self-loops now).
• Experiment with deterministic topologies• More features!! • Training a separate model for each sub-task (is
task information given at test time?)
top related