opusparcus–open subtitles paraphrase ......ru: Я не хочу в Европу . ↮ Ты...
TRANSCRIPT
Faculty of Arts Mathias Creutz
OPUSPARCUS – OPEN SUBTITLESPARAPHRASE CORPUS FOR SIX LANGUAGES
Department of Digital HumanitiesFaculty of Arts
University of Helsinki
Miyazaki, Japan, 10 May 2018
10 May 2018LREC 2018
1
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
2
DATA SOURCE
https://www.opensubtitles.org
http://opus.lingfil.uu.se/
https://korp.csc.fi/download/opusparcus/Opusparcus
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
3
Youwannastart?
en
THE PROCESS: DISCOVERING PARAPHRASES FOR ”YOU WANNASTART?”
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
4
Youwannastart?
Willst duanfangen?
Tu veuxcommencer ?
Vill dubörja?
Хочешьначать?
Aloitatko?Haluatkoaloittaa?
fr
sv
de
fi
ru
Faculty of Arts Mathias Creutz 10 May 2018
LREC 2018
5
Do you want to go first?Or else you can start.You wanna break?
Do you wanna start?Do you wanna volley for serve?Do you want to go first?Do you want to start?Hey, you want to tap out?The new dad's gonna.Want to go first?You wanna get started?You wanna go first?You want to get into this right now?
Do you want to go first?Do you want to proceed?No, I think you should.Okay, you wanna go first?Wh-When did you talk to him?You wanna start us off?You want to be first?You want to go first?
Did you wanna go first?Do you want to go first?Want to start?
D'you want to go first?Do you know the opening?Do you want to go first?Do you want to start?This calls for a joint.Why don't you start us off.Would you like to start?You wanna go first?You want to go first?
You
wanna
start?
Willst du
anfangen?Tu veux
commencer ?
Vill du
börja?
Хочешь
начать?
Aloitatko?
Haluatko
aloittaa?
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
6
Do you want to go first?Or else you can start.You wanna break?
Do you wanna start?Do you wanna volley for serve?Do you want to go first?Do you want to start?Hey, you want to tap out?The new dad's gonna.Want to go first?You wanna get started?You wanna go first?You want to get into this right now?
Do you want to go first?Do you want to proceed?No, I think you should.Okay, you wanna go first?Wh-When did you talk to him?You wanna start us off?You want to be first?You want to go first?
Did you wanna go first?Do you want to go first?Want to start?
D'you want to go first?Do you know the opening?Do you want to go first?Do you want to start?This calls for a joint.Why don't you start us off.Would you like to start?You wanna go first?You want to go first?
Youwannastart?
Willst duanfangen?
Tu veuxcommencer ?
Vill dubörja?
Хочешьначать?
Aloitatko?Haluatkoaloittaa?
Faculty of Arts Mathias Creutz
de: Das haben wir schon versucht . ↔ Wir haben das bereits probiert .en: We hope you enjoy the flight . ↔ We hope you have a pleasant flight .
fi: Mikä on noin hauskaa ? ↔ Mitä hirnut siinä ?fr: Je ne fais pas de bruit . ↔ Je suis très silencieux .
ru: Может , поговорим об этом ? ↔ Хочешь об этом поговорить ?sv: Alla vinner på det . ↔ Det tjänar alla på .
10 May 2018LREC 2018
7
SOME PARAPHRASES FOUND
Faculty of Arts Mathias Creutz
de: Willkommen , Professor Liu . ↮ Wo ist meine Tochter ?
en: I don ’t get it . ↮ Why isn ’t your brother here ?
fi: Mitä me teemme ? ↮ Viemme sairaalaan .
fr: On est des chiens . ↮ Qu' avons-nous fait ?
ru: Я не хочу в Европу . ↮ Ты знаешь , где она это взяла ?
sv: Han kunde inte komma . ↮ Var är Jack ?
10 May 2018LREC 2018
8
SOME SUGGESTED SENTENCE PAIRSTHAT ARE NOT PARAPHRASES
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
9
And I can 't ever thank you .
I can 't thank you enough .
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
10
ANNOTATION CATEGORIES
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
11
MANUALLY ANNOTATED”DEVELOPMENT” AND ”TEST” SETS
Language Total number of annotations
“Good” or“Mostly good” paraphrases
German (de) 3483 2060
English (en) 3088 1997
Finnish (fi) 3703 1921
French (fr) 3847 2004
Russian (ru) 4381 2088
Swedish (sv) 4178 1931
Total 22,680 12,001
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
12
ESTIMATED QUALITY OF AUTOMATICALLY RANKED “TRAINING” SETS
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
13
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
14
RANKING FORMULAE
Conditional probability:
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
15
RANKING FORMULAE
Conditional probability:
Joint probability:
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
16
RANKING FORMULAE
Conditional probability:
Joint probability:
Pointwise mutual information:
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
17
RANKING FORMULAE
Conditional probability:
Joint probability:
Pointwise mutual information:
Joint probability and PMI combined:
Faculty of Arts Mathias Creutz 10 May 2018LREC 2018
18
RANKING FORMULAE
Conditional probability:
Joint probability:
Pointwise mutual information:
Joint probability and PMI combined:
PMI summed over multiplepivot language corpora:
Faculty of Arts Mathias Creutz
de: Danke für Ihre Zeit . ↔ Vielen Dank für ihre Zeit .en: Thank you for listening . ↔ Thank you for your attention .fi: Kiitos , kun tulitte . ↔ Kiitos kun pääsitte tulemaan .fr: Je vous remercie d' être venus . ↔ Merci infiniment d' être venus .ru: В любом случае , спасибо . ↔ Всё равно - спасибо .sv: Tack , det var allt . ↔ Tack ska ni ha , allihop .
10 May 2018LREC 2018
19
FINALLY
I am grateful for the valuable contributions by: Mikko Aulamo, Thomas de Bluts, Aleksandr Semenov, Eetu Sjöblom, OliviaEngström, Janine Siewert, Carola Carpentier, Svante Creutz, Yves Scherrer, Anders Ahlbäck, Sami Itkonen, RiikkaRaatikainen, Kaisla Kajava, Tiina Koho, Oksana Lehtonen, Sharid Loáiciga Sánchez, Tatiana Batanina, Jörg Tiedemann,Mietta Lennes, Hanna Westerlund, and Martin Matthiesen, as well as the anonymous reviewers.