the learning non-isomorphic tree mappings for machine translation jason eisner - johns hopkins univ....
TRANSCRIPT
the
Learning Non-Isomorphic Tree Mappings for Machine Translation
Jason Eisner - Johns Hopkins Univ. a
b
A
B
events of
misinform
wrongly
report
to-John
events
him
“wrongly report events to-John” “him misinform of the events”
2 words become 1
reorder dependents
0 words become 1
0 words become 1
Syntax-Based Machine Translation
• Previous work assumes essentially isomorphic trees– Wu 1995, Alshawi et al. 2000, Yamada & Knight 2000
• But trees are not isomorphic! – Discrepancies between the languages
– Free translation in the training data
the
a
b
A
B
events of
misinform
wrongly
report
to-John
events
him
Two training trees, showing a free translation from French to English.
Synchronous Tree Substitution Grammar
enfants(“kids”)
d’(“of”)
beaucoup(“lots”)
Sam
donnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
kids
Sam
kiss
quite
often
“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”
enfants(“kids”)
kids
NPd’
(“of”)
beaucoup(“lots”)
NP
NP
SamSam
NP
Synchronous Tree Substitution Grammar
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
quitenullAdv
oftennullAdv
nullAdv
“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”
Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.
enfants(“kids”)
kids
Adv
d’(“of”)
beaucoup(“lots”)
NP
SamSam
NP
Synchronous Tree Substitution Grammar
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NPquite
often
“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”
Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.A much worse alignment ...
enfants(“kids”)
kids
NPd’
(“of”)
beaucoup(“lots”)
NP
NP
SamSam
NP
Synchronous Tree Substitution Grammar
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
quitenullAdv
oftennullAdv
nullAdv
“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”
Two training trees, showing a free translation from French to English.A possible alignment is shown in orange.
enfants(“kids”)
kids
NPd’
(“of”)
beaucoup(“lots”)
NP
NPquitenull
Adv
oftennullAdv
nullAdv
SamSam
NP
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
Synchronous Tree Substitution Grammar
“beaucoup d’enfants donnent un baiser à Sam” “kids kiss Sam quite often”
Start
Two training trees, showing a free translation from French to English.A possible alignment is shown in orange. Alignment shows how trees are generated synchronously from “little trees” ...
SamSamNP
Grammar = Set of Elementary Trees
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
idiomatictranslation
enfants(“kids”)
kids
NP
enfants(“kids”)
kids
NP
SamSamNP
Grammar = Set of Elementary Trees
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
idiomatictranslation
SamSamNP
enfants(“kids”)
kids
NP
Grammar = Set of Elementary Trees
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
SamSamNP
enfants(“kids”)
kids
NP
d’(“of”)
beaucoup(“lots”)
NP
NP
“beaucoup d’” deletes inside the tree
d’(“of”)
beaucoup(“lots”)
NP
NP
“beaucoup d’” deletes inside the tree
Grammar = Set of Elementary Trees
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
SamSamNP
enfants(“kids”)
kids
NP
enfants(“kids”)
kids
NPd’
(“of”)
beaucoup(“lots”)
NP
NP
“beaucoup d’” matches nothing in English
Grammar = Set of Elementary Trees
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
SamSamNP
enfants(“kids”)
kids
NP
SamSamNP
enfants(“kids”)
kids
NPquitenull
Adv
Grammar = Set of Elementary Trees
oftennullAdv
nullAdv
d’(“of”)
beaucoup(“lots”)
NP
NP
kissdonnent (“give”)
baiser(“kiss”)
un(“a”)
à (“to”)
Start
NP
NP
nullAdv
adverbial subtree matches nothing in French
Probability model similar to PCFG
Probability of generating training trees T1, T2 with alignment A
P(T1, T2, A) = p(t1,t2,a | n)probabilities of the “little”
trees that are used
p(is given by a maximum entropy model
wrongly
misinform
NP
NP
reportVP | )VP
FEATURES• report+wrongly misinform?
(use dictionary)
• report misinform? (at root)
• wrongly misinform?
Maxent model of little tree pairs
• verb incorporates adverb child?
• verb incorporates child 1 of 3?
• children 2, 3 switch positions?
• common tree sizes & shapes?
• ... etc. ....
p(wrongly
misinform
NP
NP
reportVP | )VP
Inside Probabilities
the
a
b
A
B
events of
misinform
wrongly
report
to-Johnevents
him
VP
( ) = ...misinformreport VP
* ( ) * ( ) + ...
p( | )VP
Inside Probabilities
the
a
b
A
B
events of
misinform
wrongly
report
to-Johnevents
him
VP
NP
NP
( ) = ...misinformreport VP
events ofNP to-John himNP* ( ) * ( ) + ...
p( | )VP
NP
misinform
wrongly
report VP
NP
only O(n2)
An MT ArchitectureViterbi alignment yields output T2
dynamic programming engine
Probability Model p(t1,t2,a) of Little Trees
score little tree pair
propose translations t2of little tree t1
each possible (t1,t2,a)
inside-outsideestimated counts
update parameters
for each possible t1, various (t1,t2,a)
each proposed (t1,t2,a)
DecoderTrainerscores all alignmentsof two big trees T1,T2
scores all alignmentsbetween a big tree T1
& a forest of big trees T2