Towards a Translation Assessment Assistant Tom Cheesman Assessment Criteria for Target Texts Accuracy (in relation to the Source Text) Appropriate Fluency (in the Target Language) Der Silicon-Valley-Gre Peter Thiel (44) The Silicon Valley great Peter Thiel, 44, The Silicon Valley giant Peter Thiel (44) The Silicone-Valley investor Peter Thiel Peter Thiel, 44, of the Silicon Valley circles Silicon-Valley prominent figure, Peter Thiel The 44-year-old Silicon-Valley-sized Peter Thiel Peter Thiel, the superstar Silicon Valley investor Analogue Translation Assessment Workflow Trainer selects, prepares Source Text (ST) Students create Target Texts (TTs) (with or without use of dictionaries, online resources, or Computer-Assisted Translation tools) Trainer manually assesses the TTs and gives individual written feedback (+ maybe a model translation) Draft Digital Workflow with TAA* System Source Text (ST) is segmented. TT upload template is created. ST is machine-analysed for translatability >> metrics + in-text annotations [option: show to students]. E.g.: 1.Text metrics (readability, type/token ratio, word/sentence lengths, etc) 2.Morphosyntactic analysis: parts of speech, grammar 3.Linguistic corpus-related analysis semantic domains, registers, sentiments, etc 4.Cultural item labelling: named entities, idiomatic expressions (use WordNets + cultural reference resources) Students create aligned Target Texts (TTs). TTs are analysed >> metrics + annotations o Compare TT analyses with ST analyses (as 1 4 above): more distant = less accurate or less appropriate o Target Language errors in word formation, collocations, grammar: more errors = less fluent o Corpus variation analysis identifies which ST segments are more/less variously translated: more various = more challenging and which students produce more typical or outlier translations: outlier = very good or very bad? Trainer assesses TTs on 1 screen. Can override automated annotations o Metrics and annotations identify common problems and individual outliers o Smooth, easy navigation in the corpus. Assessment per segment as well as per document. o Problems specified: SL or TL knowledge gaps? lexis? grammar? inappropriate register or domain? etc o Recurrent errors need only be manually annotated once Feedback advantages o Automated metrics + annotations + manual comments, which are ST-specific + TT-specific + cohort/corpus-related o Feedback to individual students and to the cohort is coordinated Research data generated: learning/teaching, text analytics / NLP, HCI, viz, etc Additional student learning: o Text analysis tools and techniques o Editing, self-testing: students can edit TTs, re-submit, compare new metrics+annotations o Shared learning: students can compare, revise each others TTs, can compile best of versions, etc * Further Applications of TAA? Compare, analyse multiple versions of: Scripture, literature, philosophy Translation competition entries Practical documents: legal texts, contracts, tenders, product descriptions, multi-author papers, etc. Multi-modal texts: written + spoken versions, dubbing, subbing, interpreting, international news stories, etc ??? Possible partner: SDL (Bristol) SDL Trados Computer-Assisted Translation system (Kevin Flanagan) Towards a Translation Assessment Assistant for Tom Cheesman