Survey participants lost in translation: an experimental study on the use of

dialectal vs. standard language

Track: International and Cross-Cultural Marketing

Mourad Touzani (Rouen Business School, France) / Fatma Smaoui (Institut Superieur de Gestion,

University of Tunis, Tunisia) / Kaouther Kooli (The Business School, Bournemouth University) / Karim

Ben Yahia (ISBAM, LARIME, TUNISIA)

Access to this paper is restricted to registered delegates of the EMAC 2013 Conference

1

Survey participants lost in translation: an experimental study on the use of dialectal vs.

standard language

Abstract

This study investigates the effectiveness of questionnaires’ translation to Dialectal Arabic and

examines the impact of three linguistic modalities on respondents’ cognitive disposition and

the moderating role of acculturation and linguistic preferences. For the purposes of this paper,

an experiment was implemented. A questionnaire designed in three languages French,

Classical Standard Arabic and Dialectal Arabic was given to three samples of 200 individuals

each. The findings show a higher cognitive disposition toward the French-language

questionnaire. Questionnaires in Dialectal Arabic did not generate good cognitive disposition

amongst our respondents. Moreover, there is a significant moderating effect of acculturation

on the relationship between linguistic modalities and respondents' Cognitive Disposition

towards the questionnaire. Preference for Dialectal Arabic moderates the relationship between

the linguistic modality and respondents’ cognitive disposition towards the questionnaire.

Keywords Questionnaire translation, Dialect and language, Cross-cultural research,

Linguistic preferences.

Track International and Cross-Cultural Marketing

2

1. Introduction

In their effort to avoid strategic mistakes, multinational companies cautiously try to gain

insight into each of their target markets. Cross-national marketing research appears to be the

best way to identify the differences and specificities characterizing each country/region.

Concerned about the comparability of their findings, marketing researchers usually translate

data collection tools such as questionnaires. Translation is then assumed to hold in

consideration the linguistic nuances of the target populations. In the Middle-East and North

Africa (MENA) region, marketing researchers have several linguistic options: they may keep

the original version of the questionnaire (in English or French), or translate it whether in

Classical Standard Arabic or in Dialectal Arabic. During the last two decades, throughout the

Arab world, some marketing professionals seemed to have adopted Dialectal Arabic when

designing slogans, labeling products, implementing advertising campaigns and online

communicating with consumers. More recently, assuming that colloquial Arabic is closer to

people’s hearts and minds, several marketing research groups started using colloquial Arabic

in both qualitative and quantitative studies.

However, it is essential for marketing researchers to be able to guarantee that translation will

not affect the equivalence of their tools. The purpose of this research is to investigate the

effectiveness of questionnaires’ translation to Dialectal Arabic. A literature review on

languages/dialects is followed by an experimental study (in the Tunisian setting) including the

three linguistic options mentioned above. The aim is to understand the impact of these

linguistic modalities on respondents’ cognitive disposition and to compare how Dialectal

Arabic compared to Classical Standard Arabic and French affects cognitive disposition. This

research also gave us the opportunity to analyze the moderating role of acculturation and

linguistic preferences.

2. Literature review

2.1. Dialects in the MENA region

The word dialect is commonly used by sociolinguists to refer to any variety of language

which is shared by a group of speakers (Wolfram, 1991). In the Arab world, Dialectal Arabic

is the natural spoken Arabic used in everyday-life communications. It is often opposed to

Classical Standard Arabic which is the formal and standard type of language. While each

country is characterized by one or several forms of Dialectal Arabic speeches used in oral

exchanges on a daily basis, Standard Arabic is a common denominator for all Arab countries

and is used in formal media such as books, newspapers, magazines, TV news, political

speeches, and movie/TV captioning. Classical Standard Arabic is generally used for formally

written communications, whereas dialects are preferred for spoken communication. Each

dialect is a particular phraseology, more or less technical, more or less rich, more or less

imaged according to the various semantic fields it encompasses. Within its linguistic

expression, the originality of dialect is reflected through its technical words that designate

specific categories and activities, as well as expressive wordings conveying visions of the

word, affective states and value systems.

2.2. Dialects and comprehension

Several companies seem to consider that dialect may be used to get closer to consumers

(Appiah & Liu, 2009). Dialect may induce stereotypical judgements in the listener/reader

(Wang et al., 2012), as well as regional and/or social identification (DeShields et al., 1996;

DeShields & Kara, 2011; Gong et al., 2011). Familiarity with a given dialect has been proven

3

to ameliorate comprehension (Gass & Varonis, 1984). However, no significant impact on

intelligibility was highlighted (Matsuura, Chiba, & Fujieda, 1999). In some cases, listeners

associate positive/negative connotations (educational level, social status or region) with

dialect. This will influence the dialect comprehensibility (Eisenstein & Verdi, 1985). Other

variables may affect comprehensibility such as variations in grammar or prosody (Anderson-

Hsieh et al., 1992).

2.3. Research propositions

Several studies show that individuals may answer differently according to the language of the

questionnaire (Harzing, 2006; Hung & Heeler, 1998; Ralston et al., 1995). Richard & Toffoli

(2009) underline the existence of a ‘language effect’ that may constitute a threat to the

equivalence and validity of instruments across several language versions. The question we ask

here is whether this argument is extendable to dialects. Very little research addresses the

problem of using dialects in marketing research studies. Among the rare developments on the

topic, dissimilarities between different linguistic modalities are reported to be critical in

questionnaire development. To illustrate this, Brock (1989) gives the example of the

differences between classic and colloquial Cantonese in Hong Kong that often necessitate

rewriting questionnaires. In the present study, we posit the following proposition:

P1: Dialectal Arabic leads to better cognitive disposition towards the questionnaire compared to Classical

Standard Arabic or French.

For the particular case of former-colonized countries, the linguistic system is characterized by

bilingualism. During their studies, citizens learn both their language and the one of the former

colonizing country. This is often accompanied by tensions between the two languages (Sam &

Berry, 2006; Hirschman & Touzani, 2011). When individuals speak Dialectal Arabic, it is not

rare that they introduce French (or English) words in their sentences. In case of acculturation,

these linguistic incursions become frequent and individuals will often go back and forth

between the colloquial Dialect Arabic and French (Fitouri, 1983). These considerations lead

us to the following proposition:

P2: Acculturation has a moderating effect on the relationship between the linguistic modality and respondents’

cognitive disposition towards the questionnaire.

Individuals' language capacity has been found to be a factor influencing their willingness to

engage in a survey, their aptitude to answer questions, and the response rate as well (Harzing,

2000). When moving to a less familiar language, morphological, lexical and syntactic

variations may lead lower level of comprehension (Matsuura, 1999). Besides, individuals are

at ease with their own dialect, and will then be able to decrypt cultural cues that are embedded

in this dialect (Appiah & Liu, 2009). Dialect being the most familiar linguistic form, we

formulate the proposition below.

P3: Preference for Dialectal Arabic has a moderating effect on the relationship between the linguistic modality

and respondents’ cognitive disposition towards the questionnaire.

3. Methodology

3.1. Construction of a cognitive disposition index (CDI)

A first qualitative step aimed to determine the criteria leading to respondents’ cognitive

disposition towards a questionnaire. Ten questionnaires, varying in terms of language/dialect,

4

length, quality, and content were designed. Following Kelly’s triadic approach (Kelly, 1955;

Whyte & Bytheway, 1996), respondents were asked to fill in triads of questionnaires and to

proceed to comparisons. A content analysis was conducted to analyze the qualitative data that

emerged from the comparisons. The text was codified and categorized and themes emerged.

After several discussions, the authors selected eleven key criteria as proxies of respondents’

cognitive disposition towards a questionnaire: simplicity, precision, design, involving,

neutrality, comprehensiveness, technical, questions order, length, and easy-to-focus-on. Later,

in our experiment, we proposed to grade each item on a 5-points Likert scale, and to calculate

a cognitive disposition index based on the sum of these grades.

3.2. Experimental design

A professional marketing consultancy company accepted to collaborate with us in this

experimental study. We benefitted from a professional market research intended to study

usages and attitudes in the clothing industry. A survey has been conducted using three

samples of 200 individuals each, selected according to a quota procedure designed with

respect to gender, age, socio-professional categories and geographical area according to the

statistics given by the Tunisian National Institute of Statistics (2012). The questionnaire

designed by the consultancy company was translated from French to Classical Standard

Arabic and Dialectal Arabic by an expert and cultural specificities of each language were

respected (Thelen et al., 2006). They were back translated by two independent experts. The

back translated versions were compared to the initial French version of the questionnaire to

check the translation reliability (Douglas & Craig, 2006). A pre-test of the questionnaire was

carried out and changes were made based on the respondents’ suggestions. The Classical

Standard Arabic version of the questionnaire was answered by sample 1, the Tunisian

Dialectal Arabic version was answered by sample 2 and the French version was answered by

sample 3. Each questionnaire was accompanied by an additional page conceived by the

authors. This page allowed them to measure the cognitive disposition index, acculturation,

and linguistic preferences. The questionnaires were face to face administrated to individuals

during autumn 2012 by professional interviewers. Six hundred questionnaires were

completed.

3.3 Data analysis

SPSS 19 was used to analyze the quantitative data. Principal component analysis and

reliability analysis were performed on the acculturation scale. Then, ANOVA and group

analyses were applied to test the research propositions.

4. Results

4.1. Linguistic modality and respondents’ cognitive disposition toward the questionnaire

The results show a significant effect of the linguistic modality on the CDI. A more detailed

analysis indicates a significant effect on 8 of the 11 criteria composing the CDI (Table 1).

Only length, design and questions order are not influenced by the linguistic modality used in

the questionnaire. When comparing linguistic modalities, our findings show that there’s a

higher cognitive disposition index toward the French-language questionnaire (Table 1).

Design, questions order and length were found to be non significant criteria. Otherwise, for all

the other items, French has the best score for the biggest number of criteria except for

precision. Questionnaires are cognitively apprehended to be more precise when they are in

Classic Standard Arabic. Consequently, Dialectal Arabic did not grade better than Classic

Standard Arabic and Standard Arabic for any of the criteria. Proposition 1 is not supported,

5

since questionnaires in Dialectal Arabic did not generate good cognitive disposition amongst

our respondents.

Table 1. Linguistic modalities and cognitive disposition scores

F Sig Standard

Arabic

mean score

Dialectal

Arabic mean

score

French mean

score

Cognitive Disposition Index (CDI) 27.57 S (0.000) 29.4 29.7 32.61

CDI criteria

Simplicity 45.30 S (0.00) 2.94 3.92 3.06

Design 0.93 NS 3.78 3.79 3.68

Precision 11.99 S (0.00) 4.10 3.75 3.73

Involving 17.71 S (0.00) 2.11 2.74 2.35

Neutrality 17.90 S (0.00) 2.33 2.79 2.16

Comprehensiveness 15.51 S (0.00) 2.39 3.02 2.62

Technical 2.52 S (0.081) 2.18 2.34 2.10

Questions order 0.77 NS 2.26 2.41 2.34

Suggestive-of-an-answer 18.52 S (0.00) 2.03 2.44 1.78

Easy-to-focus-on 2.69 S (0.068) 2.34 2.58 2.34

Length 0.29 NS 2.95 2.85 2.92

4.2. Acculturation as a moderator of the relationship between the linguistic modality and CDI

An exploratory factor analysis was performed in order to identify the dimensionality of the

acculturation scale. Using the eigenvalue-greater-than-1 criterion, our results show that this

scale is one-dimensional and concentrates 64% of the total initial variance. Our data yielded a

Cronbach's alpha of 0.72 as an indicator of the scale's reliability. As noted by Nunnally and

Bernstein (1994), an alpha coefficient greater than 0.70 is moderately reliable. The test of the

moderating effect of acculturation on the relationship between linguistic modalities and

respondents' Cognitive Disposition towards the questionnaire is significant (Sig. 0.007). A

high level of acculturation impacts this relationship negatively for both Classical Standard

Arabic and Dialectal Arabic questionnaires and positively for French-language questionnaires

(Table 2). P1 is then supported.

4.3. Preference for Dialectal Arabic as a moderator of the relationship between the linguistic

modality and CDI

Table 2 shows that preference for Dialectal Arabic (vs. Classical Standard Arabic and French)

moderates the relationship between the linguistic modality and respondents’ cognitive

disposition towards the questionnaire. A feeling of unease with Classical Standard Arabic

(compared to Dialectal Arabic) has a positive influence on our respondents’ cognitive

disposition towards the questionnaire. A similar result has been found when respondents

perceived they had more language capacity with French than with Dialectal Arabic.

Table 2. Interaction effects of acculturation and language preference DF Sum of Squares Mean Square F Sig.

Principal effect

Linguistic modality (LM) 2 1084.157 542.079 20.371 0.000

Interaction effect

LM * acculturation 3 332.816 110.939 4.086 0,007

Interaction effect

LM * Preference for Dialectal

vs. Standard Arabic

3 134.315 131.315 5.048 0.002

LM * Preference for Dialectal

vs. French

3 140.602 0,432 5.284 0.001

6

5. Discussion

The first original contribution of this paper is the construction of an index allowing to

measure respondents’ disposition towards questionnaires. The second contribution is the focus

made on dialect as a possible option for questionnaire survey. An important finding is that

linguistic modalities do influence disposition towards the questionnaire, with a better

disposition level for the French language. This may be explained through the “acculturation in

situ” concept (Hirschman & Touzani, 2011), a phenomenon characterizing not only Tunisia,

but the whole MENA region. A questionnaire, through its hard copy, its content, its design, its

figures and form can be associated to knowledge, science and progress in respondents’ minds.

Unconsciously, individuals may feel that it has a bigger fit with the French language than with

both Arabic linguistic modalities. French is often associated with science since it is the

language used in college and university for scientific and technical courses (Abassi, 2005 ).

Western culture also embodies for many LDCs consumers the idea of modernity, progress and

rationality (Sandickci and Ger, 2001). Our results are also in keeping with the idea that French

is an operational/functional language. Dialectal Arabic is not even preferred to Classic

Standard Arabic. This latter may be considered as deeper, nuanced, specific, and in all cases

allowing a better comprehension. Dialectal Arabic does not generate better cognitive

dispositions since it is not associated with pencil-and-paper situations. When using Dialectal

Arabic in written situations, it remains hard to phonetically transcribe several expressions that

are specific to the oral speech. In spite of the growing presence of Dialect in the social media,

smart phones, and even the advertising landscape since recently, it remains considered as an

oral communication tool.

References

Abassi, D. (2005). Entre Bourguiba et Hannibal – Identité tunisienne et histoire depuis

l’indépendance, Karthala éditions.

Anderson-Hsieh, J., Johnson K., & Koehler, K. (1992). The relationship between native

speaker judgments of non-native pronunciation and deviance in segmentals, prosody, and

syllable structure. Language Learning, 42, 529–555.

Appiah, O., & Liu, Y. I. (2009). Reaching the model minority: Ethnic differences in

responding to culturally embedded targeted- and non-targeted advertisements. Journal of

Current Issues & Research in Advertising, 31, 1, 27–41.

Brock, S. E. (1989). Marketing research in Asia: Problems, opportunities, and

lessons. Marketing Research, 1, 3, 44.

DeShields Jr, O. W., Kara, A., Kaynak., E. (1996). Source effects in purchase decisions: The

impact of physical attractiveness and accent of salesperson. International Journal of Research

in Marketing, 13, 89–101.

DeShields Jr, O. W., & Kara, A. (2011). The varying influence of spokesperson's accent in

communication effectiveness: A comparative study in two different regions of Mexico.

Journal of Targeting, Measurement and Analysis for Marketing, 19, 55–65.

Douglas, S. P., & Craig, C. S. (2006). On improving the conceptual foundations of

international marketing research. Journal of International Marketing, 14, 1, 1–22.

Eisenstein, M., & Verdi G. (1985). The intelligibility of social dialects for working class adult

learners of English. Language Learning, 35, 2, 287–298.

7

Fitouri, C. (1983). Biculturalisme, bilinguisme et éducation, Neuchâtel-Paris Delachaux et

Niestlé.

Gass, S. M., & Varonis, E. (1984). The effect of familiarity on the comprehensibility of non-

native speech. Language Learning, 34, 65–89.

Gong, Z., Zhang, M., & Zhou, G. (2011). Cache-based document-level statistical machine

translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural

Language Processing, 909-919, Edinburgh, Scotland.

Herzing, A. W. K. (2000). Cross-national industrial mail surveys why do response rates differ

between countries? Industrial Marketing Management, 29 (3), 243–254.

Harzing, A. W. K. (2006). Response styles in cross-national survey research: a 26-country

study. The International Journal of Crosscultural Management, 6, 2, 243–66.

Hirschman, E. C., & Touzani, M. (2011). Consumer acculturation in situ: The continuing

legacy of French colonization in North Africa. in Handbook of Islamic Marketing, Özlem,

Sandikci, Gillian Rice, eds. Edward Elgar Publishing Limited, 114–128.

Hung, K., & Heeler R. (1998). Bilinguals' brand perceptions reported on different-language

questionnaires. Advances in Consumer Research, 25, 246–51.

Kelly, G. A. (1955). The Psychology of personal constructs. (Vol. 1), New York: W.W.

Norton & Co.

Matsuura, H., Chiba, R., & Fujieda, M. (1999). Intelligibility and comprehensibility of

American and Irish Englishes in Japan. World Englishes, 18, 1, 49–62.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. Third edition, New York,

McGraw-Hill.

Ralston, D. A., Cunniff, M. K., & Gustafson, D. J. (1995). Cultural accommodation: the effect

of language on the responses of bilingual Hong Kong Chinese managers. Journal of Cross-

Cultural Psychology, 26(6): 714–727.

Richard, M. O., & Toffoli, R. (2009). Language influence in responses to questionnaires by

bilingual respondents: A test of the Whorfian hypothesis. Journal of Business Research, 62,

987–994.

Sam, D. L., & Berry, J. W. (2006). The Cambridge handbook of acculturation psychology.

Cambridge.

Sandıkcı, Ö., & Ger, G. (2002). In-between modernities and postmodernities: theorizing

Turkish consumptionscape. In. Susan M. Broniarczvk and Kent Nakamoto (eds), Advances in

Consumer Research, vol. 29, Valdosta, GA: Association for Consumer Research, pp. 465–70. Thelen, S. T., Ford, J. B., & Honeycutt, E. D. Jr. (2006). The impact of regional affiliation on

consumer perceptions of relationships among behavioral constructs. Journal of Business

Research, 59, 9, 965–973.

Wang, Z., Arndt., A. D., Singh, S. N., Biernat, M., & Liu, F. (2012), “You Lost Me at Hello”:

How and when accent-based biases are expressed and suppressed. . International Journal of

Research in Marketing, In Press.

Whyte G., & Bytheway A. (1996). Factors affecting information system’s success.

International Journal of Service Industry Management, 7, 1, 74–93.

Wolfram, W. (1991). Dialects and American English, Prentice Hall, (Englewood Cliffs, NJ).