I

O

Pd

R

a

b

c

d

I

Td

Cf

e(

0d

ntensive and Critical Care Nursing (2010) 26, 64—68

avai lab le at www.sc iencedi rec t .com

journa l homepage: www.e lsev ier .com/ iccn

RIGINAL ARTICLE

reparing research instruments for use withifferent cultures

uth Endacotta,b,∗, Julie Benbenishtyc,1, Myriam Sehad,2

Faculty of Health, University of Plymouth, UKDivision of Nursing and Midwifery, La Trobe University, Melbourne, AustraliaHadassah Hebrew University Medical Center, General Intensive Care Unit, PO Box 12000, 91120 Jerusalem, IsraelSpital Maennedorf, Asylstr, 8708 Maennedorf, Switzerland

Accepted 14 December 2009

KEYWORDS Summary There is a growing requirement to use standardised instruments for collecting

Researchinstruments;Validity;Reliability;Translation;Multicultural;

research data and monitoring patient progress. Two sets of properties should be addressedwhen selecting and adapting research instruments: psychometric properties (validity, appropri-ateness, reliability, and responsiveness) and clinical properties (feasibility and acceptability ofthe instrument). This paper outlines steps necessary to fulfil these requirements when using aresearch instrument in different cultures.© 2009 Elsevier Ltd. All rights reserved.

(

a

International

ntroduction

he requirement to adapt research instruments for use inifferent cultures is evident in two trends:

(a) the increase in people from different cultural back-grounds in the same population (for example, relativesof ICU patients);

∗ Corresponding author at: Faculty of Health, Centre Court, Drakeircus, Plymouth PL4 8AA, UK. Tel.: +44 1752 587488;ax: +44 1752 586748.

E-mail addresses: ruth.endacott@plymouth.ac.uk,ndacottsinoz@hotmail.com (R. Endacott), JULIE@hadassah.org.ilJ. Benbenishty), m.seha@spitalmaennedorf.ch (M. Seha).1 Tel.: +972 2 6778060; fax: +972 2 6430349.2 Tel.: +41 44 922 20 60; fax: +41 44 922 20 67.

(tmotapip(Awo

964-3397/$ — see front matter © 2009 Elsevier Ltd. All rights reserved.oi:10.1016/j.iccn.2009.12.005

b) the increase in studies designed to examine practicesacross different cultural contexts.

In order to compare populations that may be cultur-lly diverse, it is important to use standardised instrumentsSumathipala and Murray, 2006) hence the preference forranslating instruments rather than developing new instru-ents (see for example Stricker et al. (2007) adaptation

f a measure for family satisfaction in ICU and the adap-ation of post-traumatic stress disorder measures in ICUs described by Jones et al. (2007)). The level of detailrovided in reports of studies using translated researchnstruments varies considerably, reflecting the primary pur-

ose of the paper. For example, a paper by Larsson et al.2007) focused on the processes used to translate Confusionssessment Method for ICU (CAM-ICU) for use in Swedish ICUshereas a paper by Jones et al. (2007) focused on the resultsf a study that used a tool translated into three languages.

es 65

Figure 1 Seven-stage translation process (adapted from Wildet al., 2005). Preparation; forward translation/reconciliation;bnfi

oa

CStbSgDi

1

23

miiat

egoTsab

tt

1

Preparing research instruments for use with different cultur

Emphasis is often placed on ensuring adequate and appro-priate translation of instruments; this is not surprising giventhe heavy influence of culture on language. The ETHICATTstudy investigators (Sprung et al., 2007) found that the ques-tionnaire used to examine attitudes to end of life in nurses,physicians, patients and families included expressions forwhich there was no equivalent in some languages.

There are a number of factors to be considered inaddition to instrument translation. In general, two sets ofproperties should be addressed when selecting and adapt-ing research instruments: firstly, psychometric properties(validity, appropriateness, reliability, and responsiveness)and, secondly, clinical properties (for example, feasibil-ity/acceptability of the measure). These will be addressedin this paper, specifically as they relate to measures to beused in different cultures.

Validity

Validity describes the extent to which the instrument mea-sures what it is intended to measure. Research instrumentsin themselves are not inherently valid (Curtis, 2003); validityhas to be established for specific contexts and populations.For example, the ICU Palliative Care Quality Measures weredeveloped in the United States over a 2-year period in con-sultation with over 200 ICU clinicians from 43 ICU teams.They were then subjected to extensive and rigorous pilottesting (for a full description of these processes, see Nelsonet al., 2006). However, validity and reliability establishedthrough this process does not mean that the tool can beautomatically applied in other settings. In consultation withNelson, work is underway by the authors (RE, JB, and MS)to assess validity and reliability for the measure in the UK,Israel and Switzerland.

Traditionally validity has been conceptualised as:

• content validity, which examines the extent to which ameasure contains a comprehensive sample of items thatare relevant to the area of interest;

• criterion validity, which examines the extent to which ameasure provides results that are consistent with a goldstandard;

• construct validity, which involves forming theories aboutthe attribute and then assessing the extent to which themeasure provides results that are consistent with the the-ories. The construct validity of the ICU Palliative CareQuality Measures tool (Nelson et al., 2006) will also beassessed by the authors using focus groups to examinewhat clinicians in the three countries consider to repre-sent a ‘good’ dying process.

Each of these types of validity should be considered whenresearch instruments require translation.

Translating research instruments

The overall goal of instrument translation is to provide evi-dence that the meaning of items in the translated versionis equivalent to items in the original language (Varrichio,2004). Specific guidelines exist for the translation ofresearch instruments designed to measure patient-related

ack translation; back translation review; harmonisation; cog-itive debriefing; review of cognitive debriefing results andnalisation.

utcomes (Wild et al., 2005); these guidelines recommendseven-stage translation process (see Fig. 1).A detailed presentation of these processes to translate

onfusion Assessment Method for ICU (CAM-ICU) for use inwedish ICUs is provided by Larsson et al. (2007). An alterna-ive approach, using translation/back translation followedy a nominal group technique, is described in detail byumathipala and Murray (2006). Achieving consensus is a keyoal of both of these approaches; in order to achieve this,uffy (2006) suggests that those reviewing the translated

nstrument should answer three questions for each item:

. what does this item mean to you? (to confirm item mean-ing);

. how clear is this item to you? (to confirm item clarity);

. how relevant is this item to you? (to confirm item rele-vance).

This last question is particularly culturally dependent anday be overlooked when selecting and adapting research

nstruments. Two authors of this paper (JB/MS) report that its unusual to use ALL questions in a questionnaire developednd validated in English because some items are not relevanto other cultures.

Construct validity (sometimes referred to as ‘conceptualquivalence’) is particularly important if there is no lan-uage/lexical equivalence (Temple, 1997). Some examplesf challenges with language equivalence are presented inable 1. Language is a crucial consideration even when theame word is used; for example, the cultural sensitivitiesttached to the term ‘euthanasia’ are eloquently portrayedy Michalsen and Reinhart (2006).

Birbili (2000) suggests that different factors may affecthe quality of the translation, depending on who undertakeshe translation:

. if the researcher is also the translator, the qual-

ity of translation may be influenced by factors suchas: the background of the researcher—translator; theresearcher’s knowledge of the language and culture ofthe people under study; and the researcher’s fluency inthe language of the write-up.

66 R. Endacott et al.

Table 1 Linguistic challenges encountered when translating research instruments.

Study (authors) Challenges

Translation of CAM-ICU for use in Swedish ICUs (Larsson etal., 2007)

The word ‘delirium’ in Swedish culture can be linked toalcohol abuse‘Confusion’ might be interpreted as an insulting term

Translation of the Bradford Somatic Inventory (Sumathipalaand Murray, 2006)

Translating the phrase ‘heart pounding’ led to two phrasesin Sinhala but back translation meant that the two phrasesmeant ‘speeding’ and ‘a feeling of forceful tapping’

Exploration of spinal cord injury (Chen and Boore, 2010) Translation of ‘suffering’ from Chinese results in the word‘pain’.

ETHICUS study (Benbenishty et al., 2006) The definition of brain death was problematic in someEuropean countries

Physical Restraint in Intensive Care in Europe [PRICE] study Whilst there were no problems translating the research

2

praaita

taocpt

ASaisodEEslm(q(agwo

ETrioaca

T

Iatrobotea

A

Ariepmibmoaow

(Benbenishty and Adam, 2006)

. if the researcher and the translator are not the same per-son, the quality of translation can be influenced by theposition the translator holds in relation to the researcherand the competence/background of the translator. Ifthe translator is not a health care worker, his/herunderstanding of nursing and medical concepts may beinaccurate or incomplete.

The extent to which translators also make decisions aboutunctuation and inflection will depend on the type of mate-ial being translated (Chen and Boore, 2010). It is generallyccepted that any translation is subject to flaws: ‘‘evenn apparently familiar term or expression for which theres direct lexical equivalence might carry ‘emotional conno-ations’ in one language that will not necessarily occur innother’’ (Birbili, 2000).

Research instruments may be subject to copyright henceheir use or amendment requires permission of the originaluthors. Regardless, it is good practice to seek involvementf the original authors in order to assess harmonisation (oronsistency) with the original version of the tool. This isarticularly important if the researchers will seek in futureo compare findings across a number of studies.

voiding biaselection bias can be introduced when decisions are madebout which languages to translate research instrumentsnto; such decisions must be guided by the primary aim of thetudy. For example, if the aim is to explore nurses’ opinionsn a good death in ICU, involving families during CPR, moralistress amongst ICU nurses or conflicts in the ICU acrossurope then the instruments must be translated into as manyuropean languages as possible. However, if the aim of thetudy is to compare practice in two specific countries thenimiting the languages used is acceptable. This decision-aking about choice of language is explicit in some papers

e.g. Latour et al., 2009), whilst other authors leave theseuestions unanswered (Sprung et al., 2007). Latour et al.

2009) describe a pragmatic approach to language decisions:s their respondents were all conference delegates, the lan-uages used for their questionnaire (in addition to English)ere based on location of the conference and country ofrigin for the majority of delegates.

ltCen

instrument, the data collection instructions requiredconsiderable revision

thical acceptabilityhe translation of technical terms for clinicians to completeesearch instruments is relatively straightforward (it is eas-er to achieve consistency where there is a central lexiconf practice); translation of the participant information sheetnd consent form requires researchers to satisfy ethical prin-iples of: informed consent, beneficence/non-maleficencend autonomy/right to withdraw.

ranslating research findings

ssues around translation in research may not arise untilfter data collection, when the researcher seeks to presenthe data in another language. In the case of qualitativeesearch, Chen and Boore (2010) suggest that translationf concepts and categories is undertaken, rather than ver-atim translation of all data transcripts, with involvementf an expert panel to verify translation decisions. However,his remains particularly problematic when qualitative dataxcerpts, often in the patient or relative’s own words, formkey part of a conference presentation.

ppropriateness

ppropriateness is a term used to describe whether theange of the construct measured within the sample is sim-lar to the range which is covered by the instrument. Inssence this reflects how relevant the instrument is to theopulation being examined. For example, a useful outcomeeasure must provide room on the scale to demonstrate

mprovement and deterioration. Appropriateness is assessedy looking at the scale score distributions of the instru-ent with regard to the range, mean, standard deviation

f scores, as well as the floor and ceiling effects. If the floornd ceiling effects are high, it suggests that the spectrumf the scale is too limited to detect some of the changeshich may occur. If a high percentage of people score the

owest possible score, thus producing a floor effect, thenhere is no room for further deterioration to be detected.onversely if a high percentage of people score the high-st possible score, producing a ceiling effect, then there iso room to detect improvements that might be occurring.

es

Er

adcisc

F

Aiititdcb

QosRt

C

CmfD(abppttdtps

R

B

B

B

C

Preparing research instruments for use with different cultur

It therefore follows that high floor and ceiling effects limitthe potential responsiveness of an instrument, particularlywhen the sample is restricted to specific sub-samples or set-tings. Further, if there are differences between responsesby country then data analysis may be restricted to subsetsof the original dataset, requiring large numbers from eachcountry or elaborate re-coding processes.

Reliability

Regardless of the strength of previous reliability testing,it is important to establish reliability when a tool is usedin a different setting. Reliability refers to the extent towhich the measure is consistent and minimises random error(its repeatability). Aspects of reliability to be addressedinclude: instrumental reliability (reliability of measurementdevice), rater reliability (reliability of the person admin-istering the measurement device) and response reliability(reliability/stability of the variable being measured). Theseaspects are addressed through establishing equivalence, sta-bility and internal consistency for the instrument.

Equivalence

Establishes whether an instrument produces consistent mea-surements, for a given entity, in the hands of two or moreinvestigators (raters) or when utilised in two different forms.If data are to be collected on any variable in a study by morethan one investigator, inter-rater reliability needs to be con-sidered. Tools used in clinical practice are almost alwaysrequired to have inter-rater reliability, since different prac-titioners are likely to use them to assess the same individual(for example CAM-ICU).

Stability

Denotes the extent to which an instrument performs consis-tently when used to measure the same entity on repeatedoccasions, i.e. the extent to which measurements arerepeatable. It is important to distinguish between stabilityof the instrument (e.g. CAM-ICU) and stability of the entitythat it seeks to measure (e.g. confusion). Stability is usuallydetermined for a single rater/investigator and is referred toas intra-rater reliability or test—retest reliability. It involvesserial measurements of an entity by a single rater. Attemptsto obtain intra-rater reliability are still prone to errors aris-ing from variation in the investigator’s performance.

Internal consistency

It is a measure of the homogeneity of a multi-item instru-ment and is usually achieved by including at least two itemsthat measure the same aspect of the construct.

Responsiveness

A measure is said to be ‘responsive’ if it is sensitive tointerventions. For example quality of life measures shouldbe responsive to interventions that change quality of life.

C

67

valuating responsiveness requires assessing quality of lifeelative to an external indicator of change

Generic instruments are most useful in discriminatingnd making comparisons of different disease states foretermining severity of disease impact and cross-conditionomparisons. Disease-specific instruments can assess lim-tations or restrictions associated with particular diseasetates and may be more responsive to minimally significanthanges.

easibility and acceptability

lthough a research instrument may be widely acceptednternationally (for example, the generic SF-36 for measur-ng health and well-being) it may not be specific enough forhe culture in which the research is being conducted or thellness under study. In this case, the investigator would usehe generic tool along with a specific tool relevant to theisease/context. The inclusion of a generic measure allowsomparison of the study sample (for example family mem-ers of ICU patients) with other populations.

One aspect of the afore-mentioned review of Palliativeuality Measures for ICU (Nelson et al., 2006) by the authorsf this paper is the feasibility and acceptability of all mea-ures in the three countries (UK, Israel and Switzerland).emoval of some dimensions of the tool is necessary becausehey are not practised in a particular country.

onclusions

linicians are increasingly encouraged to use outcomeeasures to assess the effectiveness of interventions,

or example Short-Form-36 (SF-36), Hospital Anxiety andepression Scale (HADS) or the Impact of Events ScaleIES). It is important to appreciate the steps necessary todapt research instruments for use in different cultures,oth for collecting research data and for monitoring patientrogress. The increasingly multicultural population of ICUatients and families also brings a responsibility to ensurehat adequate understanding is achieved when explaininghe results of assessments to patients and families with aifferent linguistic background. This is particularly impor-ant in countries where the education background of theatient/family may create inequity in the level of under-tanding achieved.

eferences

enbenishty J, Adam S. Physical restraint in European ICUs. Inten-sive Care Med 2006;32(S1):S107.

enbenishty J, Ganz FD, Lippert A, Bulow H-H, Wennberg E, Hender-son B, et al. Nurse involvement in end-of-life decision making:the ETHICUS study. Intensive Care Med 2006;32:129—32.

irbili M. Translating from one language to another. Social ResearchUpdate 2000; 31. Available from: http://www.soc.surrey.ac.uk/sru/SRU31.html [accessed 11.11.2009].

hen HY, Boore JRP. Translation and back-translation in quali-

tative nursing research: methodological review. J Clin Nurs2010;19:234—9.

urtis JR. Measuring health status after critical illness: where are weand where do we go from here? In: Angus DC, Carlet J, editors.Surviving intensive care. Berlin: Springer; 2003.

6

D

J

L

L

M

N

S

S

S

T

V

W

8

uffy ME. Translating instruments into other languages: basic con-siderations. Clin Nurse Spec 2006;20(5):225—6.

ones C, Backman C, Capuzzo M, Flaaten H, Rylander C, Griffiths RD.Precipitants of post-traumatic stress disorder following intensivecare. Intensive Care Med 2007;33:978—85.

arsson C, Granberg, Axel A, Ersson. A confusion assessment methodfor the intensive care unit (CAM-ICU): translation, retransla-tion and validation into Swedish intensive care settings. ActaAnaesthesiol Scand 2007;51:888—92.

atour JM, Fulbrook P, Albarran JW. EfCCNa survey: European inten-sive care nurses’ attitudes and beliefs towards end-of-life care.Nurs Crit Care 2009;14:110—21.

ichalsen A, Reinhart K. ‘‘Euthanasia’’: a confusing term, abusedunder the Nazi regime and misused in present end of life debate.Intensive Care Med 2006;32:1304—10.

elson JE, Mulkerin CM, Adams LL, Pronovost PJ. Improving comfort

and communication in the ICU: a practical new tool for palliativecare performance, measurement and feedback. Qual Saf HealthCare 2006;15:264—71.

prung C, Carmel S, Sjokvist P, Baras M, Cohen SL, Maia P, et al.Attitudes of European physicians, nurses, patients, and fami-

R. Endacott et al.

lies regarding decisions: the ETHICATT study. Intensive Care Med2007;33:104—10.

tricker K, Niemann S, Bugnon S, Wurz J, Rohrer O, Rothen H. Familysatisfaction in the intensive care unit: cross-cultural adaptationof a questionnaire. J Crit Care 2007;22:204—11.

umathipala A, Murray J. New approach to translating instrumentsfor cross-cultural research: a combined qualitative and quanti-tative approach for translation and consensus generation. Int JMethods Psychiatr Res 2006;9(2):87—95.

emple B. Watch your tongue: issues in translation and cross-cultural research. Sociology 1997;31:607—18.

arrichio C. Measurement issues concerning linguistic translations.In: Frank-Stromberg M, Olsen S, editors. Instruments for clin-ical health care research. 3rd ed. Boston: Jones and Bartlett;2004.

ild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-

Lorenz A, et al. Principles of good practice for the translationand cultural adaptation process for patient reported out-come (PRO) measures: report of the ISPOR task force fortranslation and cultural adaptation. Value Health 2005;8:94—104.