Part 5 Test AdaptationsPart 5 Test Adaptations

1. Conceptualizations1. Conceptualizations 2. Example2. Example

What is the central issue in What is the central issue in translations/adaptations? translations/adaptations?

Producing instruments that measure Producing instruments that measure target constructs adequately in target constructs adequately in target culturestarget cultures

A Note on TerminologyA Note on Terminology

TranslationTranslation• Conventional term, still often usedConventional term, still often used

AdaptationAdaptation• Has become generic term for modern Has become generic term for modern

translation practicestranslation practices• Based on increased sensitivity for non-Based on increased sensitivity for non-

linguistic factors in translations, such as linguistic factors in translations, such as cultural norms of address, relevance of cultural norms of address, relevance of thorough knowledge of target culturethorough knowledge of target culture

Main Applications of Main Applications of Translations/AdaptationsTranslations/Adaptations

Comparative StudiesComparative Studies• Comparison of construct or mean scores Comparison of construct or mean scores

across culturesacross cultures• High demands on comparability of scoresHigh demands on comparability of scores

Maximizing comparabilityMaximizing comparability

Monocultural studies in target cultureMonocultural studies in target culture• Main issue is ensuring validity in new Main issue is ensuring validity in new

contextcontext• Few demands on comparability scoresFew demands on comparability scores

Maximizing local suitabilityMaximizing local suitability

Translations in Historical PerspectiveTranslations in Historical Perspective Stage 1:Stage 1:

• Close translations were standard practiceClose translations were standard practice• Techniques were developed (e.g., translation back Techniques were developed (e.g., translation back

translation) translation) Important societal developments:Important societal developments:

• Globalization and migration (multi-ethnic societies)Globalization and migration (multi-ethnic societies) Stage 2:Stage 2:

• Increasing appreciation that close translations have Increasing appreciation that close translations have problems, e.g., Grade 12 = Form 6 = ……?problems, e.g., Grade 12 = Form 6 = ……?

• Need for adaptations, localizationsNeed for adaptations, localizations• Need for standardization of adaptation proceduresNeed for standardization of adaptation procedures

What is a Good Translation/ What is a Good Translation/ Adaptation? Adaptation?

Dependent on perspectiveDependent on perspective• Linguistic perspectiveLinguistic perspective• Psychological perspectivePsychological perspective

Mapping problem:Mapping problem:• Translating/adapting can be seen as finding an Translating/adapting can be seen as finding an

optimal mapping of text in two languagesoptimal mapping of text in two languages What is a good mapping?What is a good mapping?

• A good mapping shows equivalence of the A good mapping shows equivalence of the original and translationoriginal and translation

ExampleExample What is the American equivalent of the What is the American equivalent of the

Dutch item “Hoe heet de koningin van Dutch item “Hoe heet de koningin van Nederland?” (Suppose that item is part of a Nederland?” (Suppose that item is part of a test of crystallized intelligence)test of crystallized intelligence)

Literal/close translation: What is the name of Literal/close translation: What is the name of the queen of the Netherlands?” the queen of the Netherlands?” • Problem: Item more difficult for American children Problem: Item more difficult for American children

than for Dutch childrenthan for Dutch children Adaptation: “What is the name of the Adaptation: “What is the name of the

president of the USA?”president of the USA?”• Problem: Queen and president are not equally Problem: Queen and president are not equally

known in their respective countriesknown in their respective countries

What Does “Equivalent” Mean?What Does “Equivalent” Mean?

Eusebius Hieronymus (St. Jerome, famous Eusebius Hieronymus (St. Jerome, famous bible translator from Greek and Hebrew to bible translator from Greek and Hebrew to Latin; ±347—419/420): Latin; ±347—419/420): • 2 types of translations: “words” and 2 types of translations: “words” and

“meanings” (he favored the latter)“meanings” (he favored the latter)

Here two types of equivalence relevant:Here two types of equivalence relevant:linguisticlinguistic

mapping/equivalencemapping/equivalence

psychologicalpsychological}

Linguistic EquivalenceLinguistic Equivalence (Broader than similarity of words)(Broader than similarity of words) Linguistic equivalence refers to Linguistic equivalence refers to

similarity of linguistic features of a text.similarity of linguistic features of a text. Examples of relevant linguistic features Examples of relevant linguistic features

are:are:• Lexical similarityLexical similarity• Grammatical accuracyGrammatical accuracy• In general: emphasis on formal-textual In general: emphasis on formal-textual

characteristics (cf. automatic translations)characteristics (cf. automatic translations)

Psychological EquivalencePsychological Equivalence

Psychological equivalence refers Psychological equivalence refers to similarity of (psychological) to similarity of (psychological) meaning and scoresmeaning and scores

Similarity in a broad sense:Similarity in a broad sense:• Textual, e.g.,Textual, e.g.,

Connotation of words, implied context of textConnotation of words, implied context of text ComprehensibilityComprehensibility

• Metrical:Metrical: Score comparabilityScore comparability

Relationship between Two Relationship between Two PerspectivesPerspectives

Three possible relations between linguistic and Three possible relations between linguistic and psychological features, depending on the psychological features, depending on the overlap:overlap:

Poorly translatable

b. partial

Essentiallynon-translatable

c. none

Translatable

a. complete

psych. linguistic

TranslatabilityTranslatabilityA psychological test/item is A psychological test/item is

Well translatableWell translatable if linguistic and if linguistic and psychological features yield the same translationpsychological features yield the same translation

Poorly translatablePoorly translatable if linguistic and if linguistic and psychological features do not entirely converge psychological features do not entirely converge (e.g., translation of slang: meaning is (e.g., translation of slang: meaning is translatable, but conciseness is lost)translatable, but conciseness is lost)

Non-translatableNon-translatable if there is a complete or if there is a complete or nearly complete nonoverlap (e.g., Jabberwocky) nearly complete nonoverlap (e.g., Jabberwocky)

Jabberwocky (Lewis Carroll, 1871)Jabberwocky (Lewis Carroll, 1871)

'Twas brillig, and the slithy toves'Twas brillig, and the slithy tovesDid gyre and gimble in the wabe;Did gyre and gimble in the wabe;All mimsy were the borogoves,All mimsy were the borogoves,And the mome raths outgrabe.And the mome raths outgrabe.

"Beware the Jabberwock, my son!"Beware the Jabberwock, my son!The jaws that bite, the claws that The jaws that bite, the claws that catch!catch!Beware the Jubjub bird, and shunBeware the Jubjub bird, and shunThe frumious Bandersnatch!"The frumious Bandersnatch!"

Illustration by John Tenniel

Framework for Framework for Translations/AdaptationsTranslations/Adaptations

Need for a theoretical—Need for a theoretical—methodological framework that links methodological framework that links all stages all stages of a projectof a project

Bias and equivalence as key Bias and equivalence as key conceptsconcepts

Steps in Designing Cross-Cultural Tests Steps in Designing Cross-Cultural Tests (Hambleton & Patsula, 1999)(Hambleton & Patsula, 1999)

1.1. Ensure tEnsure that construct equivalence exists in the language and cultural groups hat construct equivalence exists in the language and cultural groups of interest.of interest.

2.2. Decide whether to adapt an existing test or develop a new test.Decide whether to adapt an existing test or develop a new test.

3.3. Select well-qualified translators.Select well-qualified translators.

4.4. Translate and adapt the test.Translate and adapt the test.

5.5. Review the adapted version of the test and make necessary revisions.Review the adapted version of the test and make necessary revisions.

6.6. Conduct a small tryout of the adapted version of the test.Conduct a small tryout of the adapted version of the test.

7.7. Carry out a more ambitious field-test.Carry out a more ambitious field-test.

8.8. Choose a statistical design for connecting scores on the source and target Choose a statistical design for connecting scores on the source and target language versions of the test.language versions of the test.

9.9. If cross-cultural comparisons are of interest, ensure equivalence of the If cross-cultural comparisons are of interest, ensure equivalence of the language versions of the test. language versions of the test.

10.10. Perform validation research, as appropriate.Perform validation research, as appropriate.

11.11. Document the process and prepare a manual for the users of the adapted Document the process and prepare a manual for the users of the adapted tests.tests.

12.12. Train users.Train users.

13.13. Monitor experiences with the adapted test, and make appropriate revisions.Monitor experiences with the adapted test, and make appropriate revisions.

Overview of Common Overview of Common Procedures to Examine Procedures to Examine

Accuracy of Translations/ Accuracy of Translations/ AdaptationsAdaptations

Procedures as opportunities to strengthen Procedures as opportunities to strengthen the quality of a translation/adaptation the quality of a translation/adaptation projectproject

Two taxonomies presented here:Two taxonomies presented here:• common: (back) translations vs. committee common: (back) translations vs. committee

approachapproach• use of existing/new materialuse of existing/new material

Theoretical and Methodological Theoretical and Methodological BackgroundBackground

Crucial concept in translations is Crucial concept in translations is equivalence:equivalence:• LinguisticLinguistic

Mapping of linguistic meaning (word meaning, Mapping of linguistic meaning (word meaning, sentence meaning)sentence meaning)

• PsychologicalPsychological Mapping of psychological meaning (serves the Mapping of psychological meaning (serves the

same psychological function in all languages?)same psychological function in all languages?)

A good translation combines these A good translation combines these considerationsconsiderations

OptionsOptions AdoptionAdoption (Close “literal” translation) (Close “literal” translation)

• Advantage: maintains metric equivalenceAdvantage: maintains metric equivalence• Disadvantage: adequacy (too) readily assumed, should Disadvantage: adequacy (too) readily assumed, should

be demonstratedbe demonstrated Adaptation (changing contents of one or Adaptation (changing contents of one or

more items so as to increase cultural more items so as to increase cultural appropriateness)appropriateness)• Advantage: more flexible, more tailored to the contextAdvantage: more flexible, more tailored to the context• Disadvantage: fewer statistical techniques available to Disadvantage: fewer statistical techniques available to

compare scores across culturescompare scores across cultures Assembly (composing a new instrument)Assembly (composing a new instrument)

• Advantage: very flexibleAdvantage: very flexible• Disadvantage: almost no comparability maintainedDisadvantage: almost no comparability maintained

A Sample of Possible Procedures A Sample of Possible Procedures (after Harkness, 2003)(after Harkness, 2003)

Strength and Weakness of Strength and Weakness of Translations Back TranslationTranslations Back Translation

What is the Best Option?What is the Best Option?

One type is not intrinsically better or One type is not intrinsically better or worse than anotherworse than another

Main question is Main question is

NOTNOT What is globally the best choice?What is globally the best choice?

BUTBUT What is the best choice in a specific What is the best choice in a specific

case?case?

Four Important Perspectives Four Important Perspectives (Harkness & Van de Vijver, in preparation)(Harkness & Van de Vijver, in preparation)::

Adaptation PerspectivesAdaptation Perspectives

IndicatorIndicator

ConstructConstruct

MeasurementMeasurement

LanguageLanguage

IntegrationIntegrationCultureCulture

A good translation/adaptation A good translation/adaptation combines equivalence perspectives combines equivalence perspectives

What is a good translation/ What is a good translation/ adaptation?adaptation?•A translation or adaptation is good A translation or adaptation is good

when it combines high levels of when it combines high levels of construct, cultural, linguistic, and construct, cultural, linguistic, and measurement equivalence.measurement equivalence.

Is There a Best Way to Translate Is There a Best Way to Translate an Instrument? an Instrument?

Simple items often straightforward to Simple items often straightforward to translate translate • Close translations will do well, various kinds of Close translations will do well, various kinds of

equivalence jointly maximizedequivalence jointly maximized More complex items often require choices More complex items often require choices

about which equivalence will be maximized:about which equivalence will be maximized:• Maximizing comparability or cultural Maximizing comparability or cultural

appropriateness ?appropriateness ?

Different perspectives on Different perspectives on equivalence often, but not always equivalence often, but not always compatiblecompatible

Example: cross-cultural differences in modes Example: cross-cultural differences in modes of address of address • Maximizing linguistic equivalence may challenge Maximizing linguistic equivalence may challenge

cultural appropriateness (e.g., requests may be too cultural appropriateness (e.g., requests may be too direct) direct)

• Maximizing cultural appropriateness may Maximizing cultural appropriateness may challenge statistical equivalence (e.g., rephrasing challenge statistical equivalence (e.g., rephrasing may threaten comparability of scores) may threaten comparability of scores)

Taxonomy of AdaptationsTaxonomy of Adaptations

Most examples come fromMost examples come from

Traveling with Cognitive Tests:

Testing the Validity of a KABC-II Adaptation in India

Maike Malda

Fons J. R. van de Vijver

Krishnamachari Srinivasan

Catherine Transler

Prathima Sukumar

Accepted for publication in Assessment.

Kaufman Assessment Battery Kaufman Assessment Battery for Children for Children (second edition)(second edition)

Subtests:Subtests: AtlantisAtlantis Number RecallNumber Recall RoverRover TrianglesTriangles Word OrderWord Order Pattern ReasoningPattern Reasoning Story Completion Story Completion replaced by adaptation of replaced by adaptation of

WISC(/-R/-III) Picture ArrangementWISC(/-R/-III) Picture Arrangement

1. Example Construct-Driven1. Example Construct-Driven

Problems with the behaviors or Problems with the behaviors or attitudes associated with the attitudes associated with the construct or with communication construct or with communication norms pertaining to these behaviors norms pertaining to these behaviors or attitudesor attitudes

Usage of somatic and psychological Usage of somatic and psychological symptoms in depression inventoriessymptoms in depression inventories

Differential norms in allowance to express Differential norms in allowance to express psychological symptoms across culturespsychological symptoms across cultures

Patel, Abas, Broadhead, Todd, & Reeler (2001) Patel, Abas, Broadhead, Todd, & Reeler (2001) • In Zimbabwe, multiple somatic complaints such as headaches In Zimbabwe, multiple somatic complaints such as headaches

and fatigue are the most common presentations of depression. and fatigue are the most common presentations of depression. On inquiry, however, most patients freely admit to cognitive and On inquiry, however, most patients freely admit to cognitive and emotional symptoms. Many somatic symptoms, especially those emotional symptoms. Many somatic symptoms, especially those related to the heart and the head, are cultural metaphors for fear related to the heart and the head, are cultural metaphors for fear or grief. Most depressed individuals attribute their symptoms to or grief. Most depressed individuals attribute their symptoms to “thinking too much” (“thinking too much” (kufungisisakufungisisa), to a supernatural cause, and ), to a supernatural cause, and to social stressors. Our data confirm the view that although to social stressors. Our data confirm the view that although depression in developing countries often presents with somatic depression in developing countries often presents with somatic symptoms, most patients do not attribute their symptoms to a symptoms, most patients do not attribute their symptoms to a somatic illness and cannot be said to have “pure” somatisation. somatic illness and cannot be said to have “pure” somatisation. This means that it is vital to understand the culture specific This means that it is vital to understand the culture specific terminology used by patients and to assess mood in those with terminology used by patients and to assess mood in those with multiple somatic complaints. multiple somatic complaints.

Consequence Consequence • Common western measures of depression will under-diagnose Common western measures of depression will under-diagnose

depression in Shona speakers. depression in Shona speakers.

3333

2. Example Culture-Driven2. Example Culture-Driven

Example: ‘Burglar’ (Picture Arrangement; adapted for use in low-SES children in Bangalore, India )

ProblemsProblems: : 1.1. Unclear whether the burglar was getting in Unclear whether the burglar was getting in

or getting out; or getting out; 2.2. Man not recognized as burglar; Man not recognized as burglar; 3.3. Window was not recognized (vertically Window was not recognized (vertically

moving windows are uncommon in India)moving windows are uncommon in India)

Malda, Van de Vijver, Srinivasan, Transler (2008): Adapting a Western Cognitive Test for a Malda, Van de Vijver, Srinivasan, Transler (2008): Adapting a Western Cognitive Test for a Non-Western Context: The KABC-II in Bangalore, IndiaNon-Western Context: The KABC-II in Bangalore, India

Example: Do you often feel distressed? Example: Do you often feel distressed? Translation to Dutch:Translation to Dutch:

• ““Distressed” does not have an equivalent word in Dutch Distressed” does not have an equivalent word in Dutch • Possible solutions Possible solutions

Composite of different emotions in Dutch; ask for frequency Composite of different emotions in Dutch; ask for frequency of composite (“how often do you feel X and Y?”). Problem: of composite (“how often do you feel X and Y?”). Problem: composite may not be recognizable composite may not be recognizable

Choose a single emotion that is as close as possible; Choose a single emotion that is as close as possible; problem: change of item content if no close match can be problem: change of item content if no close match can be found found

Describe the emotion in the item (e.g., vignette); problem: Describe the emotion in the item (e.g., vignette); problem: may require a similar description in English original may require a similar description in English original

• Need to check adequacy of chosen solution in statistical Need to check adequacy of chosen solution in statistical analysis analysis

• Combination of judgmental and statistical evidence Combination of judgmental and statistical evidence crucial in instruments that are more difficult to crucial in instruments that are more difficult to translate/adapt translate/adapt

3. Example of Language-Driven 3. Example of Language-Driven Adaptation Adaptation

Language and test content:Language and test content:• Adaptation of words in subtestAdaptation of words in subtest Atlantis Atlantis::

Kannada nonsense words Kannada nonsense words

(e.g., English ‘Dablee’ (e.g., English ‘Dablee’ Kannada ‘Ribu’) Kannada ‘Ribu’) Important: number of syllablesImportant: number of syllables

• Adaptation of digits in subtest Adaptation of digits in subtest Number RecallNumber Recall based on number of syllables (1 in English based on number of syllables (1 in English

version; first 2 and then 3 in Kannada version; first 2 and then 3 in Kannada version)version)

3636

4. Example of Measurement-Driven 4. Example of Measurement-Driven Adaptation (Unfamiliarity)Adaptation (Unfamiliarity)

Kaufman ABC used in Bangalore (Kannada-Kaufman ABC used in Bangalore (Kannada-speaking children)speaking children)

Adaptation of words inAdaptation of words in subtest subtest Word Order Word Order based on:based on:

Unfamiliarity and ambiguity of objects and wordsUnfamiliarity and ambiguity of objects and words Number of syllablesNumber of syllables

Original version

Kannada version

Malda, Van de Vijver, Srinivasan, Transler (in review): Adapting a Western Cognitive Test for Malda, Van de Vijver, Srinivasan, Transler (in review): Adapting a Western Cognitive Test for a Non-Western Context: The KABC-II in Bangalore, Indiaa Non-Western Context: The KABC-II in Bangalore, India

Original version

Kannada version

Problem: word for star in Kannada is too long, English word “star” is well known but too short (monosyllabic)

Original version

Problems:

(1) Key was often called ice cream;

(2) English word “key” was often used, which is too short (monosyllabic)

Kannada version

Original version

Problem: original drawing was not easily recognized as house, distinguishing features added

Kannada version

Example: ‘Painting’Example: ‘Painting’

Problem: mirror was not recognizedProblem: mirror was not recognized

RoverRover

Test content:Test content: Additional instructions in subtest Additional instructions in subtest RoverRover One additional instruction in subtest One additional instruction in subtest Pattern Pattern

ReasoningReasoning Slight change of subtest composition and item order Slight change of subtest composition and item order

in in subtest subtest TrianglesTriangles

Sample item Original version Sample item Original version

Sample item Indian version

Problem: original sample item was too difficult; this item has been added as actual test item

Background Reading Background Reading (1) (2)