muñoz 2009 the way they were

8/8/2019 Muoz 2009 The Way They Were

1/22

Published in Mees, Alves & Gperich, eds. Methodology, Techonology and Innovationin Translation Process Research. Copenhagen: Samfundslitteratur, pp. 87-108.

The way they were:Subject profiling in translation process research

Ricardo Muoz Martn

AbstractMethodology is arguably an area where translation process research (TPR)

could be improved, and one of the aspects in need of attention is subject

profiling. A characterization of informants mental abilities and language

skills is proposed by means of the Wechsler Adult Intelligence Scale

(WAIS) and TOEFL (Test of English as a Foreign Language) subtests.

Results of a study which tested 17 subjects who also translated four texts

using Translog moderately supports that applying these tests may be useful

for both filtering out irregular subjects and for ranking regular

informants according to parameters relevant for TPR. Ways to simplify,shorten and standardize test administration are also discussed.

1. IntroductionBack in 1995, when Arnt Lykke Jakobsen developed Translog (Jakobsen1998), research by means of TAPs had already attracted some criticism,casting doubts on the reliability of data, as evidenced in works which could

only be read later (e.g. Bernardini 1999, 2001; Li 2004; Jakobsen 2003).But by that time, Jakobsen probably did not have a clear notion of howinfluential his initiative would turn out to be. From then on, empiricalresearch into translation processes (TPR) has expanded into a thrivingbranch of Translatology, witness the variety of topics and methodssummarized by Gpferich (2008). Ever since Translog made it possible torecord and measure typists keyboard activities, it has been one of thefavorite data collection tools.


2/22

Subject profiling in translation process research

Advances in, and diversification of, data collection procedures havecome with a sound, growing concern with methodological issues (e.g.Kumaul & Tirkkonen-Condit 1995; Dancette & Mnard 1996; Fraser1996; Kumaul 1998; Orozco 1999; Lauffer 2002; Neunzig 2002; Williams

& Chesterman 2002; Alves 2003b; Hansen 2003, 2006; Lrscher 2005;OBrien 2005; PACTE 2005; Campbell & Wakim 2007; Bayer-Hohenwarter 2008; Gpferich 2008; Muoz 2009). However, some of theproblems faced by our forerunners in the eighties and the nineties remainunsolved.

Some TPR problems seem associated with the need for a consistenttheoretical framework to interpret data1 (cf. Risku 2000; Rydning 2001;

Muoz 2006), others are related to the nature of our research designs.There is still enormous variation between TPR projects in terms of researchfoci, research protocols, ecological validity, subjects categorization, sourcetexts (cf. Fraser 1996; Rodrigues 2002), and data interpretation. While it istrue that some of these factors may be irrelevant for certain research goals,others are pervasive and threaten to maintain current endeavors as aheterogeneous set of juxtaposed attempts which cannot be interconnected,nor built upon. In short, progress in methodology has not lived up to

improvements in data collection procedures.Following Gpferichs (2008: 916) classification criteria, we might

describe most current TPR efforts as qualitative endeavors focusing(mainly) on online processes by means of methodologies inspired bypsycholinguistics and collecting data at one point in time only, althoughthere are some longitudinal studies underway (e.g. Hansen, on sources ofinterference, TransComp and PACTE, on the acquisition of translationcompetence). A different, popular classification (Campbell & Stanley

1963; Cook & Campbell 1979) distinguishes between pre-experimental,quasi-experimental, and true experimental designs. Pre-experimentaldesigns adhere to the scientific method, but they do not make use of controlgroups. Quasi-experimental designs employ diverse means of comparinggroups, but they fail to randomize the choice of subjects or their assignment

1 In an intelligent move which temporarily sets aside the lack of an uncontestedframework, some research teams, such as LETRA and TransComp, have developedstorage and codification systems which will make data available for new analyses inthe future.


3/22


to control and study groups. True experimental designs satisfy bothconditions. Thus, the main difference between these categories relates tothe way subjects are dealt with.

Many TPR projects belong to the category of pre-experimental designs,

which comprise case studies, one-group pre-test/post-test strategies andstatic group comparisons (cross-sectional studies). Case studies are consideredto be the least reliable, since they tend to prematurely assume links betweencauses and effects. Pre-test/post-test designs provide measurements ofchanges, but their conclusions may be biased since there is no control groupto compare with the experimental groups and thus the researcher cannot besure whether it was really the experimental manipulation what caused the

observed differences between the subjects. Finally, static group comparisonsoften fail to discern pre-experimental conditions in the subjects; moreover,the purported similarities within groups (e.g. translation students vs.professional translators) compromise the inferred differences betweenthem. In my opinion, much TPR runs into one or the other of theseproblems which, again, bring the handling of subjects to the forefront.

As far as subjects are concerned, control groups are a must, but weneed to introduce some other cautions in order to reduce systemic bias and

to enhance internal validity. Since random selection of subjects may provedifficult and random assignment to study and control groups may not befeasible in many studies, we need to discuss and agree on ways to reduceuncontrolled variables in the subjects of our experiments. Ideally, weshould find a way to supplement our data on the subjects with reliablenumerical indexes expressing differences in relevant variables, to allow usto (a) screen out subjects on the grounds of atypical scores; (b) refine dataanalyses by taking into account relevant characteristics of the subjects,

whether within the same experiment or compared to those from other,unrelated trials, and (c) triangulate them with hard process data, such astime, segmentation, and recursiveness measurements.

At present these criteria are only desiderata, and this article aims toprompt a discussion about them by proposing a two-fold characterization oftest subjects both in terms of their mental abilities and their linguistic skills(in English). Blasco (2007) shows that using the Wechsler Adult

Intelligence Scale (WAIS) and TOEFL tests may enable us to profile the


4/22


subjects but, in order to illustrate the approach, some research findings ofan ad hocresearch project will be summarized in Section 3.

2. Testing mental abilitiesMany TPR projects use psychological concepts such as strategy, expertise,competenceand the like, so a psychological approach to profiling subjectsin our experiments should ensure conceptual coherence. Psychological testsseem the best option, for they often render numerical, scaled results.Neuropsychological and personality tests are ruled out because they usuallylook for disorders in the subjects. Achievement tests do not seem to allow

for comparisons between experiments with different original texts; they maybe used in a specific project, but their results would not be transferable tothe next. Occupational tests try to match subjects interests with those ofpersons in known careers, but they do not seem very useful in our case(they would not be very informative of subjects abilities); nor do aptitudetests since, more often than not, the nature of aptitudes and skills intranslating is the very object of study. That leaves us with intelligence tests.

When dealing with intelligence, psychologists do not seem to agree

about much more than its basic definition (cf. Neisser et al. 1996). Forinstance, Gottfredson (2002) asserts that different abilities involved inintelligence merge into a single, general factor known as g, whereasWittmann & S (1997) insist that g is only the apex of a hierarchicalstructure. Converting intelligence into a score is rightfully mistrusted notonly in psychology2 but, crucially, most major intelligence tests arecomposed of around a dozen subtests which measure specific cognitivedimensions and constructs. Rather than classifying subjects according to

their IQ, we should be interested in some of the cognitive abilities that add

2 Gould (1981: 2425) argued that psychometrics is a form of scientific racism sincethe abstraction of intelligence as a single entity, its location within the brain, itsquantification as one number for each individual, and the use of these numbers torank people in a single series of worthiness, invariably find that oppressed anddisadvantaged groups races, classes, or sexes are innately inferior and deserve theirstatus. The second edition of his book, The Mismeasure of Man(1996), was enlarged toinclude an explicit rejection ofThe Bell Curve (Herrnstein & Murray 1994), whichargued that intelligence is a better predictor than educational level and parentssocioeconomic status of factors such as financial income, job performance, and crime.


5/22


to it, because a more detailed account might be more suited to controllingand profiling subjects and triangulating translation process data.

Two composite intelligence tests stand out from the rest as far asreliability and popularity are concerned: the Stanford-Binet Intelligence

Scale and the Wechsler Adult Intelligence Scale (WAIS). Stanford-Binetmeasures working memory, visual-spatial processing, knowledge,quantitative reasoning, and fluid reasoning in subjects between 2 and 23years old, and it is better than WAIS at ferreting out extremes (geniuses,mental retardation). WAIS measures working memory, processing speed,verbal comprehension, and perceptual organization in adults between 16and 89 years old, and its reliability is outstanding (Kaufman &

Lichtenberger 1999). Since many subjects in TPR are older than 23, WAISseems better suited for subject profiling.The Perceptual Organization Index (POI) in the WAIS test measures

fluid reasoning (matrix reasoning subtest), spatial processing, attentivenessto detail and visual-motor integration. It is primarily a measure of perceptual(visual) processing, and it is not clear whether it is important for ourpurposes, perhaps except when eye tracking is used. Since POI subtestshave been thoroughly revised in the WAIS4,3 and since indexing subjects

according to mental abilities should not be confused with IQ testing, it isperhaps best to omit this section (see Section 6). Several research projects,such as the TOP Ullevl 600 study of the University of Oslo, make use of alimited number of the WAIS3 subtests only to measure specific abilities.Let us have a glimpse at working memory (WM) and processing speed (PS).

2.1 Working Memory

WM is a theoretical construct (Daneman & Carpenter 1980) to refer to theability to keep important information active in mind while comprehending,thinking, and acting. It is involved in higher-order mental processes such asproblem solving (Bhner et al. 2008), tasks such as writing (Kellog 2001)

3 The WAIS test has been subjected to several revisions. The latest version, WAIS4,was released in 2008. The version discussed here is WAIS3 (1997), for whichstandardized versions are already available for many major languages, includingChinese (cf. Yao et al. 2007), French (EIWA-3, Wechsler 2000), German (HAWIE-3,cf. Von Aster et al. 2006), Portuguese (cf. do Nascimento & de Figueiredo 2002), andSpanish (EIWA-3, 1999; cf. Garca et al. 2003).


6/22


and text editing (Hayes & Chenoweth 2006), as well as actions such assaccadic eye movements (Barton et al. 2006; Bays & Husain 2008). In astudy of more than 3,000 schoolchildren (ages 511), Alloway et al. (2009)found that roughly 10 % had low WM scores which might explain

problems such as deficient monitoring of the quality of their work, anddifficulties in generating new solutions to problems.

WM seems to be able to predict some translation-relevant skills, suchas L1 fluency (Daneman 1991) and recall (Braga Tomitch 1999). WM alsointeracts with foreign language skills (Van den Noort et al. 2006). In L2,for example, WM seems to correlate positively4 with L2 fluency, accuracy,and complexity (Mota 2003). WM also seems to be a good predictor of

success in mathematical problem solving (Holmes & Adams 2006; DeSmedt et al. 2009) and insight problem-solving, i.e., the phenomenon ofsuddenly solving an apparently difficult problem (Murray & Byrne 2005),and in reasoning (S et al. 2002). Furthermore, WM may also be used topredict multitasking performance (Konig et al. 2005), where it can predicterrors of commission, i.e., errors related to addressing a task demandincorrectly, as opposed to errors of omission, i.e., allowing a task demandto lapse (Oberlander et al. 2007). The fragmentary way of reading while

translating might also be influenced by WM, for high-span readersperformance on hypertexts seems to equal that of low-span readers onlinear texts (Fontanini 2005).

Bajo et al. (2001), Rothe-Neves (2003), Liu et al. (2004), Dragsted(2005), Macizo & Bajo (2005), Mizuno (2005), Padilla et al. (2007: 5979)and Timarov (2008), among others, have already underscored theimportance of WM in translation and interpreting processes. Since, ingeneral, working memory may even be the primary determinant of

proficiency in cognitive domains, at least when the influence of priorknowledge and experience is minimal (Hambrick & Engle 2003: 190), theabove-mentioned factors suggest that WM scores might have a strongexplanatory power in TPR as well, where they might become a predictor oftranslation problem-solving success.

4 Fortunately, like other cognitive abilities, WM develops over time and can also be

improved under certain circumstances. For example, Klein & Boals (2001) suggestthat expressive writing can increase WM capacity, and Ransdell et al. (2001) showthat fluency in another language may confer long-term working memory benefits.


7/22


2.2 Processing speedProcessing speed (PS) in the WAIS3 test refers to the speed at whichcognitive processes can be carried out, and it relates to the ability to

automatically perform easy or overlearned cognitive tasks. PS does notseem related to successful problem solving (Welsh et al. 1999), although itdoes correlate with errors caused by omissions (Oberlander et al. 2007).Nevertheless, PS is interesting for other reasons. For instance, it seems tobe related to WM efficiency. A higher PS might reduce demands on WM,thereby fostering reasoning and learning (Zhu & Weiss 2005: 315); a lowerPS, on the other hand, may degrade cognitive performance because theresult of previous processing may have faded away when needed in

subsequent mental operations (Salthouse 1996). So, PS seems to have anindirect effect on the way the mind works, and not only on its pace, asexperimental results indicate for aspects such as verbal ability (Wiedel &Schwartz 1982), creativity (Rindermann & Neubauer 2004), abstractreasoning and, indeed, problem solving in tasks where subjects do not needto retrieve information from their long-term memories (fluid intelligence;Sheppard & Vernon 2008).

From a developmental point of view, PS becomes faster until ages 25to 30, and then starts a slow, steady decline (McGrew & Woodcock 2001).Expertise may reduce this decline (Nunes & Kramer 2009), so that abilitiesof older high-level experts may exceed those of younger persons withlower levels of expertise (Horn & McArdle 2007). Since expertise levelsare relevant for many TPR projects, and Horn & McArdle (2007: 241)found that expertise speed and cognitive speed were collinear, PS mightbecome an important touchstone against which to contrast translation

expertise levels.There are other reasons why PS could be important for TPR studies.

Some TPR projects reach conclusions said to be related to translation speed(e.g. automation, expertise level) but which have been arrived at throughtyping speed. However, the translation behavior of (mainly trainee)subjects in our studies is often hindered by their lack of typing skills (that


8/22


is, by slower movementtimes5), which does not necessarily correlate withlower PS scores but, so far, we have no way to account for this difference.Indexing subjects as to their mental PS might do the job, and thus thismight have been an interesting supplement to studies where timing was

important, such as De Rooze (2003), Rothe-Neves (2003), Dimitrova(2005), Dragsted (2005), Jakobsen (2005), and Sharmin et al. (2008).

Furthermore, in TPR we use the terms planning and reviewing tobroadly refer to macrotextual phases of text production, as usual in writingresearch (Jakobsen 2002). In psychology, Hayes & Flower (1986) alsoidentified planning, actual text generation, and reviewing as the three keyprocesses in L1 writing, but at a microtextual level. Levy & Ransdell

(1995) found that the median time for text generation was 7.5 seconds,whereas planning, and reviewing/revising took about 2.5 seconds each (thatis, activities alternated at these rates). Figures should be different whentranslating, due to the interplay with the original text, but one of thecharacteristics associated to better performance when translating is routineinteriorization, i.e. procedural automation as evidenced by uneventful, fastrenderings. Differences between monolingual writing and the automatedtranslation of segments might yield information as to the specifics of

translating. Hence, this microtextual approach might also be veryinformative in segmentation and pause analyses. Under this scope, PSmight turn out to be a reliable index to triangulate microtextual translationplanning, generating, and reviewing/revising phases in subjects as well.

3. Testing language commandA second area of interest to profiling subjects in translation research is

language command, since there is enormous variation between all subjects,not only between trainees. TPR researchers seem very conscious of theimportance of the native tongue for translating successfully, but there arevery few samples of research projects where this variable has beencontrolled. The WAIS3 is usually administered in the subjects nativetongue and its Verbal Comprehension Index (VCI) results from aggregating

5 Ericsson (2003: 5758) summarizes research findings which point to reading inadvance and preparing movements ahead as the main means of distinguishing expertfrom regular typists. Both characteristics may improve with higher PS.


9/22


scores from subtests focusing on verbal reasoning, concept formation,comprehension, acquired knowledge, and attention to verbal stimuli. Here,subtest scores are probably interesting per se, but the composite VCI mayalso be used as an indicator of the subjects language skills. Since WAIS3

has been normalized for many languages, using VCI as an index of nativetongue command might also make it possible to compare results of TPRprojects working with different languages.6

In foreign language skills, the point of departure should probably bethe Common European Framework of Reference for Languages (CEFR;Council of Europe 2001). Most language tests correlated to the levels ofskill in the CEFR are criterion-referenced, i.e., they assess subjects

abilities without ranking them against those of their peers.

7

These levelsmight be mapped onto numerical scales to convert them into computableindexes, but their results would be rather uninformative because only thetwo, maybe three, highest levels seem relevant for TPR subjects (B2,independent user with limited operational proficiency; C1, proficient userwith effective operational proficiency; C2, proficient user withcomprehensive mastery; Council of Europe 2001: 2229). That is, exceptfor some cases with entry-level trainees, the lower levels in the six-fold

scale would be useless, for no subject in a TPR test would score that low.Therefore, the higher levels would not provide a clear image of inter-subject variation: all professionals and most advanced trainees, forinstance, are likely to fall into the C1 and C2 categories. In other words, itwould help researchers divide subjects into simply those with good or betterlanguage skills (which is fine, but not enough), without any furtherrefinement being possible. So here we seem to reach a dead end, except forthe English TOEFL exam, which should be reliable enough for our purposes.8

6 Verbal comprehension subtests should not be used to test foreign language skills,

since psychologists do not consider it appropriate to administer the test in languagesother than the subjects own. See, however, results for subject X below.

7 This is the case for Cambridge exams for English, CIEP/Alliance Franaise forFrench, Goethe Institut/TELC for German, and DELE for Spanish.

8 The Educational Testing Service (ETS), which administers the TOEFL, states on itsweb page that more than 6000 colleges, universities, and licensing agencies in 136countries use it as indicator of the English skills of their applicants. More than 20million test takers (cumulative figure) let ETS publish statistics summarizing resultsby study level, gender, native tongue, and country.


10/22


At the end of 2005, the Educational Testing Service (ETS) madeavailable an Internet-based version of the test (IBT), which also introducedvariations in its contents and structure. The IBT writing section comprisestwo parts: writing on what has been read and listened to, and a free

composition. Compositions are reviewed by two ETS evaluators. In orderto ensure reliability, our research subjects would need to register for thetest, but the IBT is not administered in all countries. Also, costs would befar too high, so this would rule out this option in most cases. That leaves uswith training copies of the paper-based version (PBT).

The PBT is divided into three sections: listening comprehension,structure and written expression, and reading comprehension, and they take

the form of batteries of multiple-choice questions. Oral language skills areimportant for a good command of the language but they are not soimmediately important for translating.9 The section on structure and writtenexpression (T2) seems more closely related to formal knowledge of theEnglish language, whereas reading comprehension (T3) does not dependonly on language command. Rather, good comprehenders are characterizedby a better use of metacognitive strategies (Dickson et al. 1998). Hence, forTPR purposes, we might be interested in both sections.

4. Case study4.1 Materials and methodsIn order to put this proposal to the test, 21 4th-year translation students atthe University of Granada were requested as part of their training tasks totranslate four texts under normal computerized classroom conditions using

Translog. They also took the Verbal Comprehension, Working Memory andProcessing Speed subtests of the WAIS3, and sections T2 and T3 of atraining PBT. Four subjects were male (A, E, I and O) and the rest werefemale, and they were all right-handed. All originals were complete texts ofapproximately 450 words, and they were all taken from The Economist.Translations of excerpts or full articles from The Economist are oftenpublished in the Spanish press, so the assignments were acceptably

9 Anyway, Dunkel & Davies (1994: 65) have shown that TOEFL scores do not capturethe real abilities in oral comprehension in ESL University students.


11/22


realistic. Translations were performed against the clock: subjects had to tryto complete them in one hour, although they were told in advance theyshould finish their translations even if they took longer. Subjects times tocomplete the four texts were averaged and are displayed in Table 1 undertranslation time(TT).

Translations were revised by a translation teacher, a professionaltranslator with more than 10 years of in-house experience in a translationand localization company, and a translation graduate/PhD student who hadstarted to work as a freelance translator. Quality evaluation followedguidelines summarized in Muoz & Conde (2007). Evaluators wererequested to perform the task to the best of their knowledge and without

any specific instructions except for the requirement to later classify eachtranslation into one of four categories: very bad, bad, good, and very good.These categories were then converted into numerical scores and results forthe four texts were averaged, converted into a 100 scale, and rounded. InTable 1, they are shown under average grade(AG).

In order to triangulate data, reformulationswere defined as changesin the copy which include either syntactic modifications or a different lexicalchoice for at least one item, and repetitions were defined as deletions

followed by the typing of exactly the same string, so that they would alsocomprise those of just one character, including punctuation marks, but notblank spaces. These parameters were chosen because of their objectivenature and their potential relationship with problem solving in translation.Two evaluators independently computed reformulations and repetitions inthe texts. No significant differences were found, and results were averaged.

Two of the subjects had technical problems and did not manage torecord all translations, and a third one was ill for a short period of time and

was therefore not able to translate all four texts. Subject X was a foreignlanguage native speaker, so her data were not computed, although they areshown in Table 1 below to illustrate the contrast with native speakers.

4.2 Results and discussionThe results for 18 students are summarized in Table 1. Scores for theTOEFL and grades have been converted to a 100 point scale, to ease the

analysis. WAIS3 scores are already averaged on a 100 point scale from the


12/22


start, with an SD of 15, so scores below 85 and above 115 may safely beconsidered meaningful.

Table 1. Subjects ages (Age) and scores for working memory (WM), processing

speed (PS), verbal comprehension (VC), TOEFL sections 2 and 3 (T2 and T3),average translation time (TT), and average grade (AG); subjects in each grouparranged according to their WM scores

Subj A B C D E F G I H J

Age 21 21 21 34 21 21 23 22 22 21

WM 117 118 122 124 125 127 129 132 134 139

PS 123 114 125 132 123 93 132 134 140 117

VC 111 123 110 108 114 103 100 121 106 110

T2 92 88 57 85 83 72 82 87 63 82

T3 95 93 85 87 90 81 88 93 90 100

TT 48:43 48:21 40:28 48:03 51:42 48:42 51:41 45:01 43:55 48:57

AG 95 95 75 60 80 90 85 85 70 80

Subj K L O M N P Q X

Age 23 23 21 21 22 21 23 26

WM 81 83 100 100 102 107 108 129

PS 98 117 112 117 125 118 106 101

VC 102 110 104 110 102 119 104 98

T2 78 83 83 85 75 92 83 85T3 50 92 90 87 93 95 88 88

TT 58:30 53:47 44:32 46:18 46:07 45:47 49:23 50:22

AG 75 75 80 70 75 85 50 60

From the point of view of screening out informants (our first goal), subjectsK and L showed significant low scores for WM. When all subjects areconsidered, this amounts to roughly 10 %, the same rate Alloway et al.

(2009) found in children. These subjects probably never had specialtraining to improve their WM, so they might run into some of theaforementioned troubles found in children as well. Although the averagegrade for their translations did not seem particularly affected, they were theones who took longest to perform the tasks, owing to both more frequentand longer pauses. Hence, when behavior is to be observed and time is arelevant factor, subjects with WM scores below 85, such as K and L,


13/22


should probably be omitted from the study,10 for their behavior might beaffected by this special circumstance, as evidenced by the low Englishverbal comprehension scores (T3) in subject K whose scores were lowfor both WM and PS though higher than those of subjects C, F, H, and N

in formal English command (T2).Subject X took the WAIS3 subtest battery in Spanish as well, where

she had standard scores in PS, high scores in WM, and slightly low scoresin VC. Her scores for T2 were moderately high (sample av. 80.5) andaverage in T3 (sample av. 88). She also took long to translate the texts, andher results were poor, probably due to her lacking Spanish skills.

Table 2. Average scores for the whole sample (17) and groups with low, standard

and high WM capacities

Sample sd Low sd Standard sd High sd

Age 21.7 0.87 23 0 21.6 0.89 21.4 0.73

WM 114.6 17.17 82.0 1.41 103.4 3.85 126.7 6.99

PS 120.8 10.81 107.5 13.44 115.6 7.09 126.7 8.46

VC 108.4 6.03 106.0 5.66 107.8 6.94 110.6 7.24

T2 83.3 5.49 80.5 3.54 83.7 1.16 79.1 11.42

T3 90.4 4.55 71.0 29.70 90.6 3.36 90.2 5.39

TT 47:35 3:20 56:08 3:20 46:04 0:15 47:33 3:29AG 79.7 9.39 75 0 77.5 6.46 81.5 11.07

As to refining data analyses by taking these factors into account, Table 2presents averages for subjects grouped according to their WM capacities(low, standard, high). Results show that 10 students (Table 1, upper row)had above standard scores for WM, 8 of whom also had high processingspeeds. This amounts to roughly 50 % of all students. Padilla (1995) foundthat advanced simultaneous interpreting trainees also had larger WMcapacities, and she suggested that training in interpretation might contributeto explain this feature, so it is tempting to expand this hypothesis to covertranslation students as well. However, the University of Granadastranslation program has a constant, very high number of applicants, whichin the English major exceeds the capacity of the program. In such cases, the

10 Subjects F, J, L, N and X showed a difference higher than 20 points between WM andPS scores. Dehn (2006: 107) considers this gap both significant and infrequent butBinder et al. (2009) think of it as normal variations within healthy adults.


14/22


public education system will prioritize students according to theirsecondary education and entrance examination grades, which in certainyears means that the students with the lowest grades registering for theirfirst-year courses have a grade point average of B+ or even A-. Since WM

is considered a good predictor of academic achievement, it might well bethe case that these students had high WM capacities from the start.

Data show that translations by the group with high WM scores wereawarded slightly better grades even if their English command was slightlylower, as expressed by T2 (there were important variations within the group,but note subject H). This difference is nearly leveled out in (English)reading comprehension (T3), so their higher mental capacities, including

higher VC (i.e., better language skills in Spanish), are a good candidate forexplaining these small differences. In fact, PS and T3 had a small, positivecorrelation (0.493*), and T3 also had a small, negative correlation with thetime subjects took to translate the texts (-0.537*).

When data were triangulated, the group with standard WM scoresyielded an average of 44.37 reformulations while the group with high WMscores had introduced 25.87 only. Thus, higher WM capacities might havea direct influence on the planning of solutions. No remarkable differences

were observed for repetitions (standard WM: 12; high WM: 13.5). Sincethe group with standard WM scores had slightly significant high PS scores(115.6) as well, repetition rates were recalculated by rearranging groupsaccording to an arbitrary threshold of a 120 PS score. This resulted in onegroup of standard-to-high PS rates (subjects B, F, J, M, O, and P) andanother group with higher PS rates (subjects A, C, D, E, G, H, I, and N).The first group had a rate of 16.75 repetitions, whereas subjects in the newgroup of higher PS scores averaged only 11 repetitions. Hence, high PS

scores might have had an influence on repetitions. No explanation can beadvanced for this, however, and it might just be a product of chance.

5. ConclusionThere is not much to conclude from a proposal based on work in progresswhose aim is to prompt an exchange of ideas, and which is illustrated by

such a small sample. Results suggest that significantly low WM/PS scoresmight distort research results when timed behavior is considered. Some small


15/22


differences in both behavior and results were found which distinguishedsubjects with standard WM scores from those with high WM scores. This,and the possible influence of high WM scores on fewer reformulations and,perhaps, that of high PS scores on fewer repetitions, definitely deserve

further study. Language skills are also variables which need to be controlled toimprove the reliability of data interpretation. Admittedly, they have notbeen a primary concern in this piece of work, but large differences betweensubjects in the study provide some food for thought.

In any case, the study points to the possibility of using WAIS3 indexesand TOEFL scores as adequate indexes to profile subjects in TPR. If datawith larger samples corroborate the differences, TPR researchers might be

able to compare subjects across trials and, crucially, we would also be ableto tap on results of other study populations researched in psychology andeducation. If correlations between test scores and training and expertiselevels were consistent, this strategy might also open up the possibility ofnormalizing the definitions of translation trainees (beginners, novices,advanced, graduates), professionals, and experts by associating them tocertain scores in the battery (e.g., VC, TOEFL; see, for instance, Ronowiczet al. 2005), and also to find some cognitive and linguistic explanations for

some idiosyncratic results. If differences are not replicated, then we willhave to look for another meaningful way to profile subjects.

6. Post ScriptumFurther work seems necessary to ease both testing and the interpretation ofresults, and also to reduce the time they take. The standard administrationof the full WAIS3 takes more than 90 minutes (Ryan et al. 1998), which is

very long.11 WAIS3 tests are designed to be administered orally andindividually, so its application seems clearly impractical. As for the TOEFLtest, the administration of the reading comprehension section takes aboutone hour and the section on structure and written expression, about half anhour. When added to the time WAIS3 subtests take, it means that subjects

11 This has prompted the design of many short forms, either by leaving out the easiestand most difficult items in each subtest or by reducing the number of subtests.However, short forms of the WAIS3 are, simply, less reliable (Jeyakumar et al.2004).


16/22


will have to spend three additional hours with the researchers. This is agrave disadvantage. Furthermore, administering and scoring a battery ofWAIS3 subtests is not an easy task and may result in variations (Ryan &Schnakenberg-Ott 2003; Kaufman & Lichtenberger 2005: 197202). But

there may be other strategies to ease the task, apart from abandoning POIaltogether.

Braden (1999: 2) states that evaluators can change the timing, thesetting, and the formats of both test presentation and response toaccommodate clients with disabilities, and illustrates this with a version ofWAIS3 translated into American Sign Language for the deaf. Also, Rossiet al. (2007) found no significant differences between individual and

collective administration of the WAIS3. Thus, there exists the possibility ofstandardizing a written presentation and response format of the WAIS3 testset as a web page.12 This strategy might make administration easier and,crucially, homogeneous across trials, even when applied to several subjectsat once. The scoring of results can also be automated, in such a way that theresearcher may only need to choose some statistical variables, as in theexample provided in Crawford et al. (2008).

So, there may be ways to shorten and homogenize test administration

and interpretation, in such a way that profiling subjects becomes both morefeasible and more manageable. It will still be time-consuming. However,this should not be seen as an impediment, but just as both a disadvantageand a challenge. Plants and stones dont move, gases and stars dont think,machines and materials dont feel. In the social sciences, the quest forvalidity demands additional work, but that is exactly what makes it sofascinating. And in the last decades, Arnt has taught us that it is worth theeffort.

ReferencesAlloway, T. P., Gathercole, S. E., Kirkwood, H. J. & Elliott, J. E. 2009. The

cognitive and behavioural characteristics of children with low workingmemory. Child Development80: 606621.

Alves, F. (ed). 2003a. Triangulating Translation. Amsterdam: Benjamins.

12 In this project, a PowerPoint and paper answer-sheet formats were used for both

WAIS3 and TOEFL, which allowed for the testing to be carried out collectively.Testing was broken down into four sessions, which preceded each translation task.


17/22


Alves, F. 2003b. Triangulation in process oriented research in translation. In F.Alves (ed). viix.

Bajo, M. T., Padilla, P., Muoz, R., Padilla, F., Gmez, C., Puerta, M. C.,Gonzalvo, P. & Macizo, P. 2001. Comprehension and memory processes intranslation and interpreting. Quaderns. Revista de Traducci6: 2731.

Barton, J. J. S., Kuzin, A., Polli, F. & Manoach, D. S. 2006. The use of workingmemory for task prediction: what benefits accrue from different types offoreknowledge? Neuroscience139/1: 385392.

Bayer-Hohenwarter, G. 2008. Alles nur eine Frage der Zeit? Methodologischeberlegungen zu Zeit(druck) und Translation. JoSTrans9: 108131.

Bays, P. M. & Husain, M. 2008. Dynamic shifts of limited working memoryresources in human vision. Science321/5890: 851854.

Bernardini, S. 1999. Using think-aloud protocols to investigate the translationprocess: methodological aspects. In J. N. Williams (ed.). RCEAL Working

papers in English and Applied Linguistics 6. Cambridge: University Press.179199.

Bernardini, S. 2001. Think-aloud protocols in translation research. Target13/2:241263.

Blasco Mayor, M. J. 2007. La comprensin oral en el desarrollo de la pericia enla intepretacin de conferencias. Granada: Comares.

Binder, L. M., Iverson, G. L. & Brooks, B. L. 2009. To err is human:Abnormal neuropsychological scores and variability are common inhealthy adults. Archives of Clinical Neuropsychology24/1: 3146.

Braden, J. P. 1999. Accommodations in testing: methods to ensure validity.Assessment Focus8/1: 13.

Braga Tomitch, L. M. 1999. Individual differences in working memory capacityand the recall of predicted elements in the text. Lenguas modernas26: 3151.

Bhner, M., Krner, S. & Ziegler, M. 2008. Working memory, visual-spatial-intelligence and their relationship to problem-solving. Intelligence 36/6:672680.

Campbell, D. T. & Stanley, J. C. 1963. Experimental and Quasi-experimentalDesigns for Research. Chicago: Rand McNally.

Campbell, S. & Wakim, B. 2007. Methodological questions about translationresearch: A model to underpin research into the mental processes oftranslation. Target19/1: 119.

Cook, T. D. & Campbell, D. T. 1979. Quasi-experimentation. Chicago: RandMcNally.

Council of Europe. 2001. Common European Framework of Reference forLanguages: Learning, Teaching, Assessment. Cambridge: CambridgeUniversity Press.

Crawford, J. R., Allum, S. & Kinion, J. E. 2008. An index-based short form ofthe WAIS-III with accompanying analysis of reliability and abnormality ofdifferences. British Journal of Clinical Psychology47/2: 215237.

Dancette, J. & Mnard, N. 1996. Modles empiriques et exprimentaux entraductologie: questions dpistmologie. Mta41/1: 139156.


18/22


Daneman, M. 1991. Working memory as a predictor of verbal fluency. Journal of Psycholinguistic Research20/6: 445463.

Daneman, M. & Carpenter, P. A. 1980. Individual differences in workingmemory and reading. Journal of Verbal Learning and Verbal Behavior19:450466.

De Smedt, B., Janssen, R., Bouwens, K., Verschaffel, L., Boets, B. &Ghesquire, P. 2009. Working memory and individual differences inmathematics achievement: A longitudinal study from first grade to secondgrade. Journal of Experimental Child Psychology103/2: 186201.

Dehn, M. J. 2006. Essentials of Processing Assessment. New York: Wiley.Dickson, S. V., Collins, V. L., Simmons, D. C. & Kameenui, E. J. 1998.

Metacognitive strategies: Research bases. In D. C. Simmons & E. J.Kameenu (eds). What Reading Research Tells us about Children withDiverse Learning Needs: Bases and Basics. Mahwah, N.J.: Erlbaum. 295276.

De Rooze, B. 2003. La traduccion, contra reloj. PhD Dissertation. University ofGranada.

do Nascimento, E. & Figueiredo, V. L. Marques de. 2002. WISC-III e WAIS-III:Alteraes nas verses originais americanas decorrentes das adaptaespara uso no Brasil. Psicologia: Reflexo e Crtica15/3: 603612.

Dragsted, B. 2005. Segmentation in translation: differences across levels ofexpertise and difficulty. Target17/1: 4970.

Dunkel, P. & Davis, J. 1994. The effect or rhetorical signalling cues on the recallof English lecture information by speakers of English as a native or secondlanguage. In J. Flowerdew (ed.). Academic Listening: ResearchPerspectives. Cambridge: Cambridge University Press. 5575.

Englund Dimitrova, B. 2005. Expertise and Explicitation in the TranslationProcess. Amsterdam: Benjamins.

Ericsson, K. A. 2003. The acquisition of expert performance as problem solving:Construction and modification of mediating mechanisms through deliberatepractice. In J. E. Davidson & R. J. Sternberg (eds). The Psychology of Problem Solving. New York: Cambridge University Press. 3183.

Fontanini, I. 2005. Working memory and the processing of linear texts and hypertexts.Letra Magna3: 118.

Fraser, J. 1996. The translator investigated: learning from translation processanalysis. The Translator2/1: 6579.

Garca, L. F., Ruiz, M. . & Abad, F. J. 2003. Factor structure of the SpanishWAIS-III. Psicothema15/1: 155160.

Gpferich, S. 2008. Translationsprozessforschung; Stand Methoden Perspektiven(Translationswissenschaft 4). Tbingen: Narr.

Gottfredson, L. S. 2002. g: Highly general and highly practical. In R. J. Sternberg& E. L. Grigorenko (eds). The General Factor of Intelligence. How Generalis it?Mahwah, N. J.: Erlbaum. 331380.

Gould, S. J. 1981. The Mismeasure of Man. New York: Norton.


19/22


Hambrick, D. Z., & Engle, R. W. 2003. The role of working memory in problemsolving. In J. E. Davidson & R. J. Sternberg (eds.). The Psychology of Problem Solving. New York: Cambridge University Press. 207232.

Hansen, G. 2003. Controlling the process. Theoretical and methodologicalreflections on research in translation processes. In F. Alves (ed). 2542.

Hansen, G. 2006. Retrospection methods in translator training and translationresearch. JoSTrans5: 240.

Hayes, J. R. & Chenoweth, N. A. 2006. Is working memory involved in thetranscribing and editing of texts? Written Communication23/2: 135149.

Hayes, J. R. & Flower, L. S. 1986. Writing research and the writer. AmericanPsychologist41: 11061113.

Herrnstein, R. J. & Murray, C. 1994. The Bell Curve: Intelligence and ClassStructure in American Life. New York: Touchstone Books (Free Press).

Holmes, J. & Adams, J. W. 2006. How are pupils working memory skills linked totheir mathematical abilities? Educational Psychology26/3: 339366.

Horn, J. L. & McArdle, J. J. 2007. Understanding human intelligence sinceSpearman. In R. Cudeck & R. C. MacCallum (eds). Factor Analysis at 100.Mahwah, N.J.: Erlbaum. 205247

Jakobsen, A. L. 1998. Logging time delay in translation. In G. Hansen (ed.). LSPTexts and the Process of Translation. Copenhagen: CBS. 73101.

Jakobsen, A. L. 2002. Translation drafting by professional translators and bytranslation students. In G. Hansen (ed). Empirical Translation Studies:Process and Product. Copenhagen: Samfundslitteratur. 191204.

Jakobsen, A. L. 2003. Effects of think aloud on translation speed, revision andsegmentation. In F. Alves (ed). 6995.

Jakobsen, A. L. 2005. Investigating expert translators processing knowledge. InH. V. Dam, J. Engberg & H. Gerzymisch-Arbogast (eds). KnowledgeSystems and Translation. Berlin: Walter de Gruyter. 173189.

Jakobsen, Arnt L. 2006. Research methods in translationTranslog. In K.P.H.Sullivan & E. Lindgren (eds). Computer Keystroke Logging and Writing:Methods and Applications. Oxford: Elsevier. 95105.

Jeyakumar, S. L. E., Warriner, E. M., Raval, V. V. & Ahmad, S. A. 2004.Balancing the need for reliability and time efficiency: short forms of theWechsler Adult Intelligence Scale-III. Educational and PsychologicalMeasurement64/1: 7187.

Kaufman, A. S. & Lichtenberger, E. O. 1999. Essentials of WAIS-IIIAssessment. New York: Wiley.

Kaufman, A. S. & Lichtenberger, E. O. 2005. Assessing Adolescent and AdultIntelligence. New York: Wiley.

Kellogg, R. T. 2001. Competition for working memory among writing processes.The American Journal of Psychology114/2: 175191.

Klein, K. & Boals, A. 2001. Expressive writing can increase working memorycapacity. Journal of Experimental Psychology: General, 130: 520533.

Konig, C. J., Buhner, M. & Murling, G. 2005. Working memory, fluidintelligence, and attention are predictors of multitasking performance, but


20/22


polychronicity and extraversion are not. Human Performance 18/3: 243 266.

Kumaul, P. & Tirkkonen-Condit, S. 1995. Think-aloud protocol analysis intranslation studies. TTR8/1: 177199.

Kumaul, P. 1998. Die Erforschung von bersetzungsprozessen: Resultate undDesiderate. Lebende Sprachen2: 4953.

Lauffer, S. 2002. The translation process. An analysis of observationalmethodology. Cadernos de Traduo10/2: 5974.

Levy, C. M. & Ransdell, S. 1995. Is writing as difficult as it seems? Memory andCognition23: 767779.

Li, D. 2004. Trustworthiness of think-aloud protocols in the study of translationprocesses. International Journal of Applied Linguistics14: 301313.

Liu, M., Schallert, D. L. & Carroll, P. J. 2004. Working memory and expertise insimultaneous interpreting. Interpreting6/1: 1942.

Lrscher, W. 2005. The translation process: Methods and problems of itsinvestigation. Meta50/2: 597608.

Macizo, P. & Bajo, M. T. 2005. Working memory and translation. Cognitiva17/1: 2953.

McGrew, K. S. & Woodcock, R. W. 2001. Technical manual. Woodcock-Johnson III. Itasca, Il.: Riverside.

Mizuno, A. 2005. Process model for simultaneous interpreting and workingmemory. Meta50/2: 739752.

Mota, M. B. 2003. Working memory, capacity and fluency, accuracy,complexity, and lexical density in l2 speech production. Fragmentos 24:69104.

Muoz Martn, R. 2006. Traductologa cognitiva y traductologa emprica. In G.Wotjak (eds). Quo Vadis, Translatologie? Berlin: Frank & Timme. 267278.

Muoz Martn, R. 2009. Expertise and environment in translation. MutatisMutandis2/1: 24-37.

Muoz Martn, R. & Conde Ruano, J. T. 2007. Effects of serial translationevaluation. In P. A. Schmitt & H. E. Jngst (Hrsg). Translationsqualitt.Frankfurt: Peter Lang. 428444.

Murray, M. A. & Byrne, R. M. J. 2005. Attention and working memory in insightproblem-solving. In B. G Bara, L., Barsalou & M. Bucciarelli (eds.),Proceedings of the XXVII Annual Conference of the Cognitive Science

Society. Mahwah, NJ:Erlbaum. 15711575.Neisser, U., Boodoo, G., Bouchard Jr., T. J., Boykin, A. W., Brody, N., Ceci,

S. J., Halpern, D. F., Loehlin, J. C., Perloff, R., Sternberg, R. J. & Urbina,S. 1996. Intelligence: knowns and unknowns. American Psychologist51/2:77101.

Neunzig, W. 2002. Estudios empricos en traduccin: apuntes metodolgicos.Cadernos de Traduo10: 7596.

Nunes, A. & Kramer, A. F. 2009. Experience-based mitigation of age-related

performance declines: evidence from air traffic control. Journal of Experimental Psychology: Applied15/1: 1224.


21/22


Oberlander, E. M., Oswald, F.L., Hambrick, D. Z. & Jones, L.A. 2007.Individual difference variables as predictors of error during multitasking.

Technical Report NPRST-TN-07-9. Available online athttp://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA470466&Location=U2&doc=GetTRDoc.pdf.

OBrien, S. 2005. Methodologies for measuring the correlations between post-editing effort and machine translatability. Machine Translation19: 3758.

Orozco, M. 1999. La metodologa de la investigacin en traductologa.Perspectives7/2: 189198.

PACTE. 2005. Investigating translation competence: Conceptual andmethodological issues. Meta50/2: 609619.

Padilla Bentez, P. 1995. Procesos de memoria y atencin en la interpretacin delenguas. Unpublished PhD dissertation. University of Granada.

Padilla, P., Macizo, P. & Bajo, M. T. 2007. Tareas de traduccin e interpretacindesde una perspectiva cognitiva. Granada: Atrio.

Ransdell, S., Arecco, M. R. & Levy, M. C. 2001. Bilingual long-term workingmemory: the effects of working memory loads on writing quality andfluency. Applied Psycholinguistics22/1: 113128.

Rindermann, H. & Neubauer, A. C. 2004. Processing speed, intelligence,creativity, and school performance: Testing of causal hypotheses usingstructural equation models. Intelligence32: 573589.

Risku, H. 2000. Review of Developing translation competence. Target 16/1:169172.

Rodrigues, C. 2002. A abordagem processual no estudo da traduo: uma meta-anlise qualitativa. Cadernos de traduo10: 2357.

Ronowicz, E., Hehir, J., Kaimi, T., Kojima, K., & Lee, D. S. 2005. Translatorsfrequent lexis store and dictionary use as factors in SLT comprehension andtranslation speed. Meta50/2: 580596.

Rossi Cas, L., Neer, R. & Lopetegui, S. 2007. Transferable skills followingWAIS-III verbal comprehension index. Orientacin y sociedad7: 125134.

Rothe-Neves, R. 2003. The influence of working memory features on someformal aspects of translation performance. In F. Alves (ed). 97119.

Ryan, J. J. & Schnakenberg-Ott, S. D. 2003. Scoring Reliability on the WechslerAdult Intelligence Scale-Third Edition (WAIS-III). Assessment10/2: 151159.

Ryan, J. J., Lpez, S. J. & Wert, T. R. 1998. Administration time estimates forWAIS-III subtests, scales, and short forms in a clinical sample. Journal of Psychoeducational Assessment16/4: 315323.

Rydning, A. F. 2001. Review of Translation Studies: Perspectives on anEmerging Discipline. Languages in Contrast3/2: 292302.

Salthouse, T. A. 1996. The processing-speed theory of adult age differences incognition. Psychological Review103/3: 403428.

Sharmin, S., pakov, O., Rih, K. J. & Jakobsen, A. L. 2008. Effects of timepressure and text complexity on translators fixations. In S. N. Spencer (ed).

Proceedings of the ETRA 2008 Symposium. New York: ACM. 123126.


22/22


Sheppard, L. D. & Vernon, P. A. 2008. Intelligence and speed of informationprocessing: a review of 50 years of research. Personality and IndividualDifferences44: 535551.

S, H. M., Oberauer, K., Wittmann, W. W., Wilhelm, O., & Schulze, R. 2002.Working memory explains reasoning And a little bit more. Intelligence30: 261

288.Timarov, . 2008. Working memory and simultaneous interpreting. In P.

Boulogne (ed). Translation and Its Others. Selected Papers of the CETRAResearch Seminar in Translation Studies 2007. Available athttp://www.kuleuven.be/cetra/papers/papers.html.

Van den Noort, M. W. M. L., Bosch, P. & Hugdahl, K.. 2006. Foreign languageproficiency and working memory capacity. European Psychologist11/4:289296.

Von Aster M. G., Neubauer, A. & Horn, R. 2006. Wechsler-Intelligenztest frErwachsene HAWIE-III. Frankfurt: Harcourt Test Services.

Wechsler, D. 2000. Manuel de la WAIS-III. Paris: Centre de PsychologieApplique.

Welsh, M. C., Satterlee-Cartmell, T., & Stine, M. 1999. Towers of Hanoi andLondon: Contribution of working memory and inhibition to performance.Brain & Cognition41: 231242.

Wiedel, T. C. & Schwartz, S. 1982. Verbal ability and mental processing speed.Current Psychological Research2: 247256.

Williams, J. & Chesterman, A. 2002. The Map. A Beginners Guide to DoingResearch in Translation Studies. Manchester: St. Jerome.

Wittmann, W. W. & S, H. M. 1997. Challenging g-mania in intelligence

research: Answers not given, due to questions not asked. Presentation at theISSID symposium New Directions in Ability Research, July 1923,University of Aarhus. Available at http://www.psychologie.uni-mannheim.de/psycho2/psycho2.en.php?page=publi/papers.en.htm&cat=publi.

Yao, S., Huan, C., Jiang, L. & Tam, W. C. C. 2007. Replication of factorstructure of Wechsler Adult Intelligence Scale-III. Chinese version inChinese mainland non-clinical and schizophrenia samples. Psychiatry andClinical Neurosciences61/4: 379384.

Zhu, J. & Weiss, L. 2005. The Wechsler scales. In D. P. Flanagan & P. L.Harrison (eds). Contemporary Intellectual Assessment. Theories, Tests, andIssues. 2nd edn. New York: Guilford Press. 297324.

muñoz 2009 the way they were

Documents