ketra. cognitive aspects of translation and post-editing: empirical investigations erik angelone,...
TRANSCRIPT
![Page 1: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/1.jpg)
KeTra
![Page 2: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/2.jpg)
Cognitive aspects of translation and post-editing: empirical investigations
Erik Angelone, Isabel Lacruz and Greg Shreve
KENT STATE UNIVERSITY
![Page 3: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/3.jpg)
KeTra’s Three Periods
Early: 2007-2009
Recent: 2010-2013
Future: 2013-onwards
![Page 4: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/4.jpg)
Early period (2007-2009)
Erik Angelone & Greg Shreve
Observational studies using Think Aloud Protocols and Screen Recordings
Two main projects
![Page 5: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/5.jpg)
Uncertainty Management and Problem Solving in Translation
Questions investigated:
1. When, where, and how are problem solving bundles employed in managing uncertainty while translating?
2. How do the degree and contour of metacognitive activity associated with the bundles vary between professional and non-professional translators?
3. In what situations are the metacognitive activities associated with uncertainty management accounted for by TAPs, and in what situations are they not?
![Page 6: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/6.jpg)
Main Findings
The professionals engaged in sequential, uninterrupted problem recognition - solution proposal - solution evaluation bundling to a greater extent than the students.
![Page 7: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/7.jpg)
Problem-Solving Strategies and Error Mitigation in Translation
Questions investigated:
1. What impact does metacognitive bundling have on overall translation quality, as indicated by the type and frequency of errors in the target text?
2. Are certain forms of uncertainty management behavior more conducive than others to limiting translation errors?
3. What is the correlation between the textual level of overall uncertainty management and the textual level of translation errors?
4. Do apparent process-oriented expertise effects in uncertainty management behavior result in product-oriented expertise effects, i.e., improved translation quality?
![Page 8: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/8.jpg)
Main Findings
Metacognitive bundling (uninterrupted problem recognition - solution proposal - solution evaluation sequences) resulted in fewer overall errors and fewer errors of each textual level type.
![Page 9: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/9.jpg)
Recent Period (2010-2013)
Erik Angelone, Isabel Lacruz, Greg Shreve
Experimental studies using eye-tracking, key-logging, and reaction time methodologies
![Page 10: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/10.jpg)
Cognitive Effort, Syntactic Disruption, and Visual Interference in a Sight Translation
Task
Questions investigated:
1. Does syntactic complexity of the source text affect sight translation performance?
2. Does syntactic complexity of the source text affect written translation?
3. Is sight translation more effortful than bilingual reading?
![Page 11: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/11.jpg)
Methodologies
Eye-tracking
Keystroke logging
One authentic version and one modified version of each text
11 translators took part in these experiments
![Page 12: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/12.jpg)
Main Findings
Syntactic complexity affected sight translation: sight translation was more effortful in the syntactically complex texts than in the non-complex texts
Syntactic complexity did not affect written translation
Sight translation was more effortful than bilingual reading
![Page 13: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/13.jpg)
Sight Translation and Speech Disfluency: Performance Analysis as a Window to Cognitive Translation
Processes
Questions investigated:
1. Is there a relationship between oral disfluency phenomena and effort indicators in eye-tracking metrics?
2. What types of linguistic problems result in disfluencies?
![Page 14: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/14.jpg)
Main Findings
Promising methodology: Triangulation of eye-tracking data and disfluency data
There seems to be an association between speech disfluencies and cognitive effort in sight translation
Disfluencies signal production problems resulting from lexical, syntactic, and translation-strategic problems
![Page 15: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/15.jpg)
Efficacy of Screen Recordings in the Other-Revision of Translations
Questions investigated:
1. Are screen recordings efficacious to assist other-revision?
2. Are screen recordings more efficacious than IPDR records in other-revision?
3. Is there a difference in efficacy depending on error type?
![Page 16: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/16.jpg)
Methodologies
8 Spanish/English translators and 4 German/English translators
First session: Each participant translated one text using Screen Recordings and another one using IPDR
Second session: Each participant edited one text translated by another translator and had access to either the Screen Recordings or the IPDR
The dependent variable was proportion of errors mitigated in the edited texts
Errors were classified by type
![Page 17: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/17.jpg)
Main Findings
Screen recordings were more effective than IPDR in mitigating errors in other-revision: The percentage of errors left in the edited text was significantly lower for SR than IPDR
This effect was significant for both German and Spanish translators
SR were more efficacious only in four of the six error categories
![Page 18: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/18.jpg)
Overall Errors Mitigated by Process Protocol
IPDR log Screen recording0
20
40
60
80
100
120
Pre-editingPost-editing
Err
or
Fre
quency
![Page 19: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/19.jpg)
Percentage of Errors Mitigated by Error Type
Punctuation Spelling Lexis Syntax Style Mistranslation0
10
20
30
40
50
60
70
80
90
100
IPDR logScreen recording
Error type
Perc
enta
ge
![Page 20: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/20.jpg)
Pauses and Cognitive Effort in Post-Editing
Questions investigated:
1. Do pause patterns in keystroke log reports indicate cognitive effort in post-editing? If so, can these patterns be quantified by metrics suitable for automatic analysis?
2. Can levels of cognitive effort in post-editing be detected through pause timing data generated from keystroke log reports?
3. Can levels of cognitive effort in post-editing be detected through word timing data generated from keystroke log reports?
4. Can levels of cognitive effort in post-editing be detected through pause count data generated from keystroke log reports?
![Page 21: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/21.jpg)
Methodologies
Keystroke logging
Post-editing of English-Spanish and Spanish-English MT
Assessment of actual cognitive effort from actions detected, rather than by inferences made from objective assessments of cognitive demand
Two observational studies: one pilot study and one follow-up using three participants with different language and professional characteristics
![Page 22: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/22.jpg)
Main Findings
Clusters of short pauses associated with high levels of cognitive effort
Average pause ratio (= average pause time/average word time) decreases as cognitive effort increases
Average pause time decreases as cognitive effort increases
Average time to process a word increases as cognitive effort increases
Pause to word ratio (= number of pauses/number of words) increases as cognitive effort increases
![Page 23: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/23.jpg)
Future Directions
Experimental studies into the nature of cognitive effort in post-editing and translation
Comparison of editing performance and behavior of non-translator subject-matter-experts and professional non-subject-matter-expert translators
Comparisons of MT revisions and human translations
How is the mind of the translator different from that of the bilingual/monolingual language user? How are mental representations for language different for a translator?
![Page 24: KeTra. Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY](https://reader038.vdocuments.mx/reader038/viewer/2022110116/551655455503469d698b4d2f/html5/thumbnails/24.jpg)
Future Directions
Both experimental and observational studies using Eye-tracking in combination with keystroke logging Reaction time paradigms such as:
Lexical decision tasks Word translation tasks Translation verification tasks both for words and for
sentences Paper and pencil tasks such multiple choice tests Screen Recordings Dual task paradigms