the alst project technologies for audiovisual translaon · 2016. 1. 8. · the alst project...
TRANSCRIPT
TheALSTprojectTechnologiesforaudiovisualtransla8onAnnaMatamala(UniversitatAutònomadeBarcelona)
[email protected](TC37-AsLing2015)London,26-27November2015Funding:FFI-2012-31024,2014SGR0027
1
AnnaMatamala(2015).“Technologiesforaudiovisualtransla>on:theALSTproject”Aspresentedatthe37thConferenceTransla1ngandtheComputer,London,UK,November26-27,2015.©AsLing,TheInterna>onalAssocia>onforAdvancementinLanguageTechnology,2015.Distribu>onwithouttheauthorisa>onfromASLINGisnotallowed.AsLingaskingtobeinformedofsuchpos>ngs,includingURLsorURIswhereavailable,totheemailaddress:<presenta>[email protected]>."
2
The ALST project: aims
• Speechrecogni>on• Machinetransla>on• Speechsynthesis
inaudiodescrip>on(AD) invoice-over(VO)
3
Project ra5onale
• Implementa>onofexis>ngtechnologies• Twomodeschosenasinstancesoflinguis>c/sensorialaccessibility• Oralmodes• Focusontheprofessionalsandontheaudiences
• Limita>ons:3years,limitedfunding• Projectteamand2PhDstudents
4
Speech Technologies
5
Speech recogni5on in voice-over (I)
• Par>cipants:10professionaltranscribers• 3condi>ons:• Manualtranscrip>on• Respeaking• Revisionofanautoma>cally-generatedscript(ASR)
• Quan>ta>vedata:>mera>o,outputquality(NER)• Qualita>vedata:opinionsonusefulness,speed,accuracy,overallquality,effort,boredom.
Source:Matamala,Romero-Fresco&Daniluk(forthcoming)6
Speech recogni5on in voice-over (II)
• Speed:manual,respeaking,ASRrevision.• Accuracy:manual,ASRrevision,respeaking.• Respeakingallowedthehighestnumberofpar>cipantstofinish.
• Respeaking,begerscoresinself-reportedeffortandboredom.• Manualtranscript,begerscoresinaccuracyandoverallquality.• Respeaking:impressedwithmanypossibili>esandmorejobsa>sfac>onbutneedforspecifictrainingandfurtherresearch.
7
Speech recogni5on in audio descrip5on
• Soundtrackextrac>on,speechac>vitydetec>on,speakerdiariza>on,andspeech-to-texttranscrip>on
• Film“Closer”(English,Catalan)
• DERandWERmeasures:lowperformancebecauseoftrainingcondi>onsofthesystemandemployedADmaterials.
Source:Delgado,Matamala&Serrano(forthcoming)
8
Speech synthesis in voice-over
• On-going
9
Speech synthesis in audio descrip5on (I)
• Pre-testtoselectthe“best”human/naturalmale/femalevoices
• Mainexperiment:67blindandvisually-impairedvolunteers• Ques>onnaireinspiredbyITU(1994)
Source:Fernández-Torné&Matamala(2015)
10
Speech synthesis in audio descrip5on (II)
• Naturalvoiceshavehighervaluesthanar>ficialvoices
• Nosta>s>caldifferencesbetweenmale/femalenaturalvoices
• Overallimpression/acceptance:meanhigherthan3.2(ona5-pointscale)• 94%par>cipants:TTSisanalterna>veacceptablesolu>onalthoughnotthepreferredone
Source:Fernández-Torné&Matamala(2015)
11
Machine Transla5on
12
Machine transla5on in VO: effort
• Par>cipants:12AVTMAstudents• Condi>ons:
• humantransla>on• post-edi>ng
• Materials:twoshortwildlifedocumentaryexcerpts(En>Es)• Temporal,technical,cogni>veeffortusingInputlog(www.inputlog.net/)
Source:Or>z-Boix&Matamala(forthcoming)
13
Machine transla5on in VO: results
• Resultsobtainedfor:bothexcerpts,excerpt1,excerpt2
• Temporaleffort:post-edi>ngfasteralthoughresultsareonlysta>s>callysignificantinexcerpt1
• Technicalandcogni>veeffort:post-edi>ngrequireslesseffort,althoughdifferencesareonlysta>s>callysignificantgloballyandinexcerpt1
Source:Or>z-Boix&Matamala(forthcoming)
14
Machine transla5on in VO: QA • QAbyexperts:• Pre-correc>ongrading/Correc>onbasedonMQM/Ques>onnaire/Finalmark/Isitatransla>onortheresultofapost-edi>ng?
• QAinDubbingstudio:observa>onalnotes,correc>onsmadebydubbingdirector
• QAbyend-users:pre-taskques>onnaires,viewing,post-taskques>onnaires(comprehension,self-reportedenjoyment,preferencesandinterest)
Source:Or>z-Boix&Matamala(2015)
15
Machine transla5on in VO: results (I)
• QAbyexperts:passesround1(45/72transla>on,37/72post-edi>ng),round2(41and38/72,respec>vely)
• QAbyexperts:correc>ons(12.862transla>onvs.17.957PE)
• QAbyexperts:iden>fytransla>onsassuchmoreeasily
• Transla>onsareslightlybeger Source:Or>z-Boix&Matamala(2015)
16
Machine transla5on in VO: results (II)
• QAbydubbingstudio:
• excerpt1(5changesintransla>onvs6inPE),• excerpt2(4intransla>on,noneinPEduetobadquality)
• Transla>onisbegerin2ndexcerpt,minimaldifferencesinthe1st
Source:Or>z-Boix&Matamala(2015)
17
Machine transla5on in VO: results (III)
• QAbyusers:• Translatedversionsslightlybegerunderstoodthanpost-editedversionsinglobalanalysisandexcerpt1(post-editedisbegerinexcerpt2).
• Begerresultsfortransla>on(enjoyment,preferencesandinterest)
• Transla>onqualityisslightlyhigherinallcondi>onsbutresultsarenotsta>s>callysignificantandtherearedifferencesaccordingtoexcerpts/groups.
Source:Or>z-Boix&Matamala(2015)
18
Machine transla5on in audio descrip5on (I)
• Pre-testtoselecttheMTengine:GoogleTranslate.
• Par>cipants:5volunteers• 5MTengines• Materials:clipfrom“Closer”(240wordsin14ADunits,3’09’’)• Tool:PET
Source:Fernández-Torné&Matamala(2014)
19
Machine transla5on in audio descrip5on (II)
• Mainexperiment:• Par>cipants:12AVTMAstudents• Condi>ons:humancrea>on/transla>on/post-edi>ng• Languagepair:EnglishintoCatalan• Materials:comparableexcerptsfrom“Closer”• Tool:Sub>tleWorkshop
• Temporal,technicalandcogni>veeffort(Inputlog,www.inputlog.net)• Subjec>veopinions(pre-taskandpost-taskques>onnaires)
Source:Fernández-Torné&Matamala(inprep.)
20
MT in AD: results
• Temporaleffort:nosta>s>caldifferences.• Technicaleffort:ADcrea>onandADtransla>on,morekeyboardac>onthanpost-edi>ng.ADtransla>onandpost-edi>ng,highernumberofmousescrollsthanADcrea>on.• Cogni8veeffort:sta>s>callyhigherinADcrea>on.
Source:Fernández-Torné&Matamala(inprep.)
21
MT in AD: results (II)
• Subjec8vedata(ona10-pointLikertscale):
Source:Fernández-Torné&Matamala(inprep.)
22
ADcrea8on ADtransla8on ADpost-edi8ng
Pre Post Pre Post Pre Post
Effort 8.25 7.17 6.17 5.58 6.50 7.50
Crea>vityimpairment
3.09 3.82 7.45 7.27 8.45 9.36
Boredom 2.09 1.82 4.18 4.18 6.73 7.27
Calques 1.25 2.00 5.25 5.42 6.93 8.33
Conclusions
• Small-scaleexploratoryproject,exis>ngtechnology
• Moreresearchneededwithwidersamples,moreexcerpts,otherlanguagepairs
• Notonly>me(produc>vity)butalsoopinionsofprofessionalsandendusers
23
TheALSTprojectTechnologiesforaudiovisualtransla8onAnnaMatamala(UniversitatAutònomadeBarcelona)
[email protected](TC37-AsLing2015)London,26-27November2015Funding:FFI-2012-31024,2014SGR0027
24