multilingual relation extraction using compositional ... relation extraction using compositional...

40
Multilingual Relation Extraction using Compositional Universal Schema Haw-Shiuan Chang, Abdurrahman Munir, Ao Liu, Johnny Tian-Zheng Wei, Aaron Traylor, Ajay Nagesh, Nicholas Monath, Patrick Verga, Emma Strubell, Andrew McCallum These slides are mostly prepared by Patrick Verga (Team ID: UMass_IESL)

Upload: trinhduong

Post on 28-Jun-2018

252 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Multilingual Relation Extraction using Compositional Universal Schema

Haw-Shiuan Chang, Abdurrahman Munir, Ao Liu, Johnny Tian-Zheng Wei, Aaron Traylor, Ajay Nagesh,

Nicholas Monath, Patrick Verga, Emma Strubell, Andrew McCallum

These slides are mostly prepared by Patrick Verga

(Team ID: UMass_IESL)

Page 2: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Wei Li studies at Xinghua U. Her 2008 publications include W. Li. “Scalable NLP” ACL, 2008.

Entity Extraction

(NER)Search

Text docs

Entity Mentions

Entities, Relations

Relation Mentions

Wei Li W. Li

Xinghua U.

Member( Wei Li,

Xinghua U.)

Wei Li W. Li

Xinghua U.

Text docs

Text docs

(PER)

(PER)

(ORG)

Slot Filling (SF)

2

Relation Extraction

Query Expansion

Wei Li, Member, ______?

Wei Li W. Li

Entity Aliases

Queries

Wei Li, Member,

Xinghua U. …

Page 3: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

KB

Wei Li studies at Xinghua U. Her 2008 publications include W. Li. “Scalable NLP” ACL, 2008.

Entity Extraction

(NER)

Resolution (Coref)

Xinghua U.

Text docs

Entity Mentions

Entities, Relations

Relation Mentions

Wei Li W. Li

Xinghua U.

Member( Wei Li,

Xinghua U.)

Wei Li W. Li

Xinghua U.

Text docs

Text docs

(PER)

(PER)

(ORG)

Knowledge Base (KB) Construction

3

Relation Extraction

Wei Li, Member, ______?

…Queries

Page 4: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

KB

Wei Li studies at Xinghua U. Her 2008 publications include W. Li. “Scalable NLP” ACL, 2008.

Entity Extraction

(NER)

Resolution (Coref)

answer

Text docs

Entity Mentions

Entities, Relations

Relation Mentions

Wei Li W. Li

Xinghua U.

Member( Wei Li,

Xinghua U.)

Wei Li W. Li

Xinghua U.

Text docs

Text docs

(PER)

(PER)

(ORG)

Resources

4

Relation Extraction

RelationFactorySpanish?

Wei Li, Member, ______?

…Queries

Page 5: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

KB

Wei Li studies at Xinghua U. Her 2008 publications include W. Li. “Scalable NLP” ACL, 2008.

Entity Extraction

Resolution (Coref)

query

answer

Text docs

Entity Mentions

Entities, Relations

Relation Mentions

Wei Li W. Li

Xinghua U.

Wei Li W. Li

Xinghua U.

Text docs

Text docs

(PER)

(PER)

(ORG)

Relation Extraction

5

Member( Wei Li,

Xinghua U.)

Relation Extraction

Page 6: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Compositional Universal Schema

Verga, P., Belanger, D., Strubell, E., Roth, B., and McCallum, A. (2016). Multilingual relation extraction using compositional universal schema. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).

Page 7: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

Relation Types

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Melinda, Bill

Steve, Microsoft

Bill, MicrosoftBill, Steve

Microsoft, SeattleMelinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

structured textual 50k columns

7

Universal Schema

Page 8: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

for which similarity is difficult to be captured by an Ope-nIE approach because of their syntactically complex con-structions. This motivates the technique in Section 3.2,which uses a deep architecture applied to raw tokens, in-stead of rigid rules for normalizing text to obtain patterns.

Sentence (context tokens italicized) OpenIE patternKhan ’s younger sister, AnnapurnaDevi, who later married Shankar, de-veloped into an equally accomplishedmaster of the surbahar, but custom pre-vented her from performing in public.

arg1’s * sisterarg2

A professor emeritus at Yale, Mandel-brot was born in Poland but as a childmoved with his family to Paris wherehe was educated.

arg1 * moved with* family to arg2

Kissel was born in Provo, Utah, buther family also lived in Reno.

arg1 * lived inarg2

Table 1: Examples of sentences expressing relations.Context tokens (italicized) consist of the text occurringbetween entities (bold) in a sentence. OpenIE patterns areobtained by normalizing the context tokens using hand-coded rules. The top example expresses the per:siblingsrelation and the bottom two examples both express theper:cities of residence relation.

2.4 Universal SchemaWhen applying Universal Schema (Riedel et al., 2013)(USchema) to relation extraction, we combine the Ope-nIE and link-prediction perspectives. By jointly mod-eling both OpenIE patterns and the elements of a targetschema, the method captures broader relational structurethan multi-class classification approaches that just modelthe target schema. Furthermore, the method avoids thedistant supervision alignment difficulties of Section 2.2.

Riedel et al. (2013) augment a knowledge graph froma seed KB with additional edges corresponding to Ope-nIE patterns observed in the corpus. Even if the user doesnot seek to predict these new edges, a joint model over alledges can exploit regularities of the OpenIE edges to im-prove modeling of the labels from the target schema.

The data still consist of (s, r, o) triples, which can bepredicted using link-prediction techniques such as low-rank factorization. Riedel et al. (2013) explore a varietyof approximations to the 3-mode (s, r, o) tensor. Onesuch probabilistic model is:

P ((s, r, o)) = �

�u

>s,o

v

r

�, (1)

where �() is a sigmoid function, us,o

is an embeddingof the entity pair (s, o), and v

r

is an embedding of therelation r, which may be an OpenIE pattern or a rela-tion from the target schema. All of the exposition and re-sults in this paper use this factorization, though many ofthe techniques we present later could be applied easily to

the other factorizations described in Riedel et al. (2013).Note that learning unique embeddings for OpenIE rela-tions does not guarantee that similar patterns, such as thefinal two in Table 1, will be embedded similarly.

As with most of the techniques in Section 2.1, the dataonly consist of positive examples of edges. The absenceof an annotated edge does not imply that the edge is false.In fact, we seek to predict some of these missing edges astrue. Riedel et al. (2013) employ the Bayesian Person-alized Ranking (BPR) approach of Rendle et al. (2009),which does not explicitly model unobserved edges asnegative, but instead seeks to rank the probability of ob-served triples above unobserved triples.

Recently, Toutanova et al. (2015) extended USchemato not learn individual pattern embeddings v

r

, but insteadto embed text patterns using a deep architecture appliedto word tokens. This shares statistical strength betweenOpenIE patterns with similar words. We leverage this ap-proach in Section 3.2. Additional work has modeled theregularities of multi-hop paths through knowledge graphaugmented with text patterns (Lao et al., 2011; Lao et al.,2012; Gardner et al., 2014; Neelakantan et al., 2015).

2.5 Multilingual EmbeddingsMuch work has been done on multilingual word embed-dings. Most of this work uses aligned sentences fromthe Europarl dataset (Koehn, 2005) to align word embed-dings across languages (Gouws et al., 2015; Luong et al.,2015; Hermann and Blunsom, 2014). Others (Mikolovet al., 2013; Faruqui et al., 2014) align separate single-language embedding models using a word-level dictio-nary. Mikolov et al. (2013) use translation pairs to learna linear transform from one embedding space to another.

However, very little work exists on multilingual re-lation extraction. Faruqui and Kumar (2015) performmultilingual OpenIE relation extraction by projecting alllanguages to English using Google translate. However,as explained in Section 2.3 the OpenIE paradigm is notamenable to prediction within a fixed schema. Further,their approach does not generalize to low-resource lan-guages where translation is unavailable – while we usetranslation dictionaries to improve our results, our experi-ments demonstrate that our method is effective even with-out this resource.

3 Methods3.1 Universal Schema as Sentence ClassifierSimilar to many link prediction approaches, (Riedel et al.,2013) perform transductive learning, where a model islearned jointly over train and test data. Predictions aremade by using the model to identify edges that were un-observed in the test data but likely to be true. The ap-proach is vulnerable to the cold start problem in collab-

Relation Typesstructured textual 50k columns

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

8

Page 9: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###~y

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

9

Page 10: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

10

Page 11: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

11

Page 12: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

####

###

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

Pattern Encoder

####

###

####

###

####

###

####

###

####

###

####

###

####

###

12

January 15, 2000

Tech pioneer Bill Gates stepped down today as chief executive officer of Microsoft, the Seattle-headquartered software giant. His long-time friend, Steve Balmer, will take over as CEO of Microsoft. Gates will now focus on the charitable foundation he runs with his wife Melinda French Gates. Bill and Melinda were married in a ceremony in Hawaii, rather than her hometown of Dallas.Gates moved his family into their 55,000-square-foot, $54 million house on the shore of Lake Washington, just outside Seattle.

LSTM

Page 13: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

13

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

####

###

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

Pattern Encoder

####

###

####

###

####

###

####

###

####

###

####

###

####

###

LSTMMelinda,

Bill

Entity pair Relation

spouse

married to

#######

####

###

and

####

###

####

###

LSTM

Page 14: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Joe NewEntity worked with his wife Jane NewEntity on relation extraction.

JoeJane

Joe NewEntity worked with his wife Jane NewEntity on relation extraction.

14

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

####

###

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

Pattern Encoder

####

###

####

###

####

###

####

###

####

###

####

###

####

###

spousequery:

LSTM

Page 15: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Joe NewEntity worked with his wife Jane NewEntity on relation extraction.

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Pattern Encoder

Joe NewEntity worked with his wife Jane

NewEntity on relation extraction.

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

15

JoeJane

LSTM

spousequery:

Page 16: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

####

###

Joe NewEntity worked with his wife Jane NewEntity on relation extraction.

Gates moved his family into their 55,000-

square-foot, $54 million house on the shore

of Lake Washington, just outside Seattle.

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

head

-of-s

tate

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Hawaii

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

Pattern Encoder

Joe Jane

Joe NewEntity worked with his wife Jane NewEntity on relation extraction.

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

16

Joe NewEntity worked with his wife Jane

NewEntity on relation extraction.

####

###

####

###

####

###

LSTM

####

###

spousequery:

####

###

Page 17: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Spanish Relation Extraction

Page 18: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

Steve, Microsoft

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Obama, MichelleObama, US

18

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Bill, MicrosoftMelinda, Dallas

Bill, SeattleObama, Michelle

English

Spanish

Multilingual Relation Extraction

Page 19: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

Steve, Microsoft

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

rBill, Microsoft

Bill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

19

Multilingual Universal Schema

Relation Typesstructured English

textual

Spanish

textual

Page 20: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

Steve, Microsoft

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

rBill, Microsoft

Bill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

20

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

LSTM

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Page 21: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

21

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

2121

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

##############

####

###

####

###

####

###

LSTM

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

##############

Page 22: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

22

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

22

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

22

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTM

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddingdistributedrepresentation

ofsemantics

learnedparameters

#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTM

CEO

chair

man

pr

esid

ent

leade

r-of

head

-of

mar

ried-

to

HQ-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Page 23: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

23

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Melinda, Bill

Entity pair Relation

Michelle, Obama

spouse

married to

esposa

Page 24: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

####

##resident

####

####

####

esposaspouse

####

##

cat

####

##

married

####

##

resident

####

##

presidente

####

##

casado

####

##

gato

24

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

Page 25: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

25

####

##

####

##

esposa

spouse

####

##

cat

####

##

married

####

##

resident

####

##

casado

####

##

gato

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

####

##

presidente

Page 26: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

26##

#### ####

##

esposaspouse

####

##

cat

####

##

married

####

##

presidente

####

##

casado

####

##

gato

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

####

##

resident

Page 27: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

####

####

####

esposaspouse

####

##

cat

####

##

married

####

##

casado

####

##

gato ####

##

spouse / esposa

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

####

##

resident

####

##

presidente

Page 28: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

####

##

spouse / esposa

####

##

cat

####

##

married

####

##

casado

####

##

gato

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

##############

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

##############

####

###

####

###

####

###

Steve, Microsoft

Bill, MicrosoftBill, Steve

Melinda, BillMicrosoft, Seattle

Melinda, Dallas

Bill, SeattleObama, Michelle

Obama, US

vectorembeddindistributerepresentati

ofsemantilearnedparamete#######

~x

~y

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

####

###

#######

####

###

#######

####

###

LSTMCE

O ch

airm

an

pres

iden

t lea

der-o

f he

ad-o

f m

arrie

d-to

HQ

-in

spou

se

born

-in

frien

dpe

r:co-

work

er

per:l

ives-

in

org:t

op-m

embe

rs

org:m

embe

r

espo

sa

pres

iden

te

vive e

n re

siden

te

fund

ador

####

##

resident

####

##

presidente

Page 29: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Search Engine Supervision

Page 30: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

KB

Wei Li studies at Xinghua U. Her 2008 publications include W. Li. “Scalable NLP” ACL, 2008.

Entity Extraction

Resolution (Coref)

query

answer

Text docs

Entity Mentions

Entities, Relations

Relation Mentions

Wei Li W. Li

Xinghua U.

Member( Wei Li,

Xinghua U.)

Wei Li W. Li

Xinghua U.

Text docs

Text docs

(PER)

(PER)

(ORG)

Search Engine as a Resource

30

RelationFactory

Relation Extraction

Page 31: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Barack Obama Michelle Obama

Entity 1 Entity 2

Corpus

Barack Obama Michelle Obama…...

…...

Barack Obama Michelle Obama…...

Distant Supervision

per:spouse wife

and

president

lives with

understands

married to

Key phrases for per:spouse

Page 32: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

High frequency signal

Low frequency signal

Global background noise

Entity background noise

High correlation noise

Not expressing a relation

Freebase selection bias

wife

and

president

lives with

married to

Only valid in some entity pairs

Other relation noiseunderstands They might have other relations

Limitations of Distant Supervision

Barack Obama Michelle ObamaEntity 1 Entity 2

Page 33: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

wife

and

president

lives with

married to

Corpus

Barack Obama Michelle Obama?

Key phrases

understands

Entity 1 Entity 2

P( ___ = Michelle Obama)

Asking in another direction?

Barack Obama … wife … ______

Barack Obama … and … ______

P( ___ = Michelle Obama)

Hard to accurately estimate this when corpus is not large enough

Page 34: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Asking Google!

Page 35: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

wife "his wife"

his married husband "wife of"

"married to" "her husband"

to her

wife "his wife" married husband

"married to" "wife of"

"is married" "is married to" “her husband”

“( wife”

Distant supervision (after applying several noises

removal techniques)

After applying search engine supervision

Noise Filtering Examples

Lower their influence on Universal Schema

Page 36: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Results

Page 37: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

KB, XLING Hop0 Hop1 Total

Prec Recall F1 Prec Recall F1 F1

USchema 0.475 0.015 0.028 0.071 0.003 0.005 0.021

LSTM 0.421 0.038 0.069 0.106 0.018 0.030 0.057

The recall is low because we don’t include Chinese query and Chinese corpus

TAC 2016 Component Testing

SF, ENG Hop0 Hop1 Total

Prec Recall F1 Prec Recall F1 F1

DS 0.244 0.168 0.199 0.500 0.013 0.025 0.159

SES+DS 0.229 0.203 0.215 0.444 0.013 0.025 0.174

Page 38: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

SF Hop0 (ENG: 350, SPA: 150) Hop1 (ENG: 185, SPA: 53) Total

Prec Recall F1 Prec Recall F1 F1

2015 system 0.302 0.154 0.204 0.081 0.088 0.084 0.155

2016 system 0.229 0.203 0.215 0.444 0.013 0.025 0.174

Spanish 0.1364 0.285 0.187 N/A N/A N/A 0.160

KB Hop0 Hop1 Total

Prec Recall F1 Prec Recall F1 F1

2015 system 0.390 0.128 0.192 0.253 0.078 0.119 0.168

2016 system 0.271 0.154 0.196 0.039 0.071 0.051 0.127

Spanish 0.280 0.121 0.169 0.060 0.048 0.053 0.124

TAC 2016 ResultsLDC Max All Micro Scores

2015 KB evaluation

2015 system 0.227 0.149 0.192 0.038 0.097 0.054 0.120

? ?

Page 39: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

• Compositional Universal Schema (LSTM) improves recall significantly.

• Search engine supervision cleans training data well.

• Universal Schema can be effectively applied to the language with low resources.

• Our 2015 system is scored much better in this year.

• Need further investigation.

Result Summary

Page 40: Multilingual Relation Extraction using Compositional ... Relation Extraction using Compositional Universal Schema ... relation extraction using compositional universal ... sults in

Q & A