satisfaction measurements in a cross-cultural environment: how valid are subjective judgements?...
TRANSCRIPT
University of Cologne
Center for Evaluation Faculty of Management, Economics and Social Sciences
Dr. Christian Bosau
Satisfaction measurements in a cross-cultural environment:
How valid are subjective judgements?
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 2
Background of the study
Are we really measuring „true“ satisfation?
Job satisfaction (JS) in organizations: § keyword: international employee satisfaction survey § the absolute level of JS often has direct consequences for leaders § intercultural measurement problems often are NOT considered
Main question: How can we compare the results of JS across national and cultural borders?
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 3
Cross-cultural JS-results
8,4 8,48,2
8,18,0 7,9 7,9 7,8 7,8 7,7 7,7 7,7 7,7 7,7 7,6 7,6 7,6 7,6 7,6 7,6 7,5 7,5 7,5 7,5
7,3 7,3 7,27,1 7,1 7,1 7,1 7,1 7,0 7,0 7,0 6,9 6,9 6,9
6,7 6,7 6,6 6,6 6,6 6,5
6,26,1
5,9
5,5
5
6
7
8
9
Switz
erla
nd
Mal
ta
D
enm
ark
Nor
way
Can
ada
I
cela
nd
I
rela
nd
A
ustr
ia
U
SA
Swed
en
B
elgi
um
N
orth
Irel
and
Mex
ico
Japa
n
A
rgen
tina
Chi
le
F
inla
nd
Net
herl
ands
Bra
zil
Lux
embo
urg
Pola
nd
N
iger
ia
Po
rtug
al
U
nite
d K
ingd
om
Italy
G
erm
any
- Wes
t
Sl
oven
ia
C
zech
Rep
ublic
H
unga
ry
Sout
h A
fric
a - W
hite
G
erm
any
- Eas
t
Spai
n
Indi
a
Chi
na
Sl
ovak
ia
Fran
ce
Li
thua
nia
Gre
ece
C
roat
ia
E
ston
ia
R
uman
ia
Latv
ia
B
ulga
ria
Kor
ea
Rus
sia
Turk
ey
U
krai
ne
B
elar
us
• results from World-Values-Survey:
• very often: deskriptive intercultural results are published (Slocum & Topichak, 1972; Lincoln et al, 1981; Griffeth & Hom, 1987; Near & Rechner, 1993; Chiu & Kosinski, 1999; van de Vliert & Janssen, 2002; Llorente & Macias, 2005)
Unclear: Can we really just interpret those descriptive differences of JS directly?
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 4
Problems of cross-cultural research
response styles and culture: § different cultures show different response styles, e.g. acquiescence (ARS) (Hui & Triandis, 1989; Johnson et al., 1997; Chen et al.; 1995; Takahashi et al., 2002; Harzing, 2006)
§ different response style should be understood as different communicational behaviour, not just methodological bias (Smith, 2004; Smith & Fischer R., 2006)
standardising of measures (see Fischer, R.; 2004): with-in-subject, group centering, grand mean centering
problems: § measures are not independent from each other anymore § absolute level of measures is lost § interpretation is possible only in relation to standardising value (group or grand mean, etc.)
What can we do about it?
Note: Especially problematic if we want to compare absolute measurements, like JS-levels of countries/subsidiaries/etc.!
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 5
Examples: - Does well-being increase with higher GDP? - The more interesting a job is, the higher the JS is? - etc.
Methodological classificiation
level oriented structure oriented vs.
Examples: - How high is JS in different countries? - In which countries are people more satisfied with their lives? - Are Americans more extroverted than Germans? - Is the image of my product better in Germany compared to Spain? - etc.
construct bias method bias vs. item bias vs.
different understanding of construct across cultural
boundaries
bad item translation or culturally inapropriate wording
different response styles
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 6
The 3 important constructs
What is the relationship looking like between these constructs?
response style
culture
job satisfaction level
? ?
? -
individualism
acquiescent response style
Known: Negative relationship between individualism and acquiescent response style (ARS)
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 7
Method - secondary analysis
§ Data from World-Values-Survey & Eurobarometer
§ operationalisation of acquiescence response style (see Harzing, 2006) : acquiescence-index 5-point-scale: ALL items, having received 5 or 4 MINUS ALL items, having received 2 or 1
§ aggregation to national level
§ correlation of national means (controlled for soziodemographic differences of nations, i.e. age, gender)
Important question: How will the JS-measurement be influenced by response styles?
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 8
Nation-level-results - 1
results: § job satisfaction and ARS are indeed correlated: BUT negatively. § i.e., high measures of job satisfaction only occur in countries/cultures with small or no ARS response tendency norms Keeping in mind: in all countries respondents are on average satisfied with their jobs § Note: if ARS exist, respondents do not report dissatisfaction but only lower satisfaction § Fits with the self-construal results of Markus & Kitayama (1991): interdependent/collectivist people – having higher ARS – are restrained in telling their personal feelings.
-.13 (14)
-.68** (11)
-.59*** (16)
-.71*** (16)
-.68*** (16)
Job Satisfaction (nation-level-mean from
Eurobarometer)1
-.29 (25) ARS-index from Hofstede (2001)
-.47** (23) ARS-index from Harzing (2006)
-.38*** (47) ARS-index 3 from WVS (4-point-scale ‚agree/disagree‘-label)
-.47*** (42) ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label)
-.47*** (46) ARS-index 1 from WVS (5-point-scale ‚agree/disagree‘-label)
Job Satisfaction (nation-level-mean from World-Values-Survey)1
1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses: number of countries
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 9
Nation-level-results - 2
-.13 (14)
-.68** (11)
-.59*** (16)
-.71*** (16)
-.68*** (16)
Job Satisfaction (nation-level-mean from
Eurobarometer)1
-.29 (25) ARS-index from Hofstede (2001)
-.47** (23) ARS-index from Harzing (2006)
-.38*** (47) ARS-index 3 from WVS (4-point-scale ‚agree/disagree‘-label)
-.47*** (42) ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label)
-.47*** (46) ARS-index 1 from WVS (5-point-scale ‚agree/disagree‘-label)
Job Satisfaction (nation-level-mean from World-Values-Survey)1
1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses: number of countries
§ the correlation is higher in european nations (countries of the EU)!
§ Note: european nations have – on average – better working conditions, since economic wealth is higher (compared wordwidely) § we know: job satisfaction measurements are certainly influenced by working conditions as well – not only communicational norms
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 10
Re-test of WVS-data
§ Operationalisation working conditions : GDP (gross domestic product)
§ Median split into good (rich countries) and bad (poor countries) working conditions
Rich countries Poor countries
Job satisfaction (nation-level-mean from World-Values-Survey)
Job satisfaction (nation-level-means from
Word-Values-Survey)
ARS-index 1 from WVS (5-point-scale ‚agree/disagree‘-label) -.51* (22) -.02 (24)
ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label) -.51*** (22) -.34 (20)
ARS-index 3 from WVS (4-point-scale ‚agree/disagree‘-label) -.37* (23) -.04 (24)
ARS-index from Harzing (2006) -.61** (13) .08 (10)
ARS-index from Hofstede (2001) -.34 (19) sample to small
§ result: correlation remains important and significant only in rich countries with good working conditions, almost no correlation within poor countries with bad working conditions § possible interpretation: following communicational norms becomes important only if a sufficient working standard is established
1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses (number of countries)
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 11
Re-test of the WVS-data
Very robust effect:
correlation remains the same by including economic indicators: - HDI - GDP - Quality-of-Life-Index
Acquiescent response style
Wor
ld-V
alue
s-Su
rvey
– jo
b sa
tisfa
ctio
n le
vel Group of countries
rich countries poor countries rich countries poor countries
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 12
- acquiescent response style
individualism
Negative relationship between acquiescent response style and job satisfaction level (in rich countries)
?
? job satisfaction level
Relationship of culture and job satisfaction: JS-Level is higher in individualistic countries (see Bosau, 2008, as well as Hofstede, Judge, etc.)
- +
New finding
The 3 important constructs
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 13
+ -
acquiescent response style
individualism
?
job satisfaction level
-
Hypothesis: Relationship of individualism and JS is only a spurious correlation
Instead: culture à response style à JS
The 3 important constructs
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 14
§ statistical analysis: mediation analysis (Baron & Kenney, 1986)
Test of mediation model
analysis by rich countries of WVS (N=21)
§ interpretation: relationship of culture and JS can be (almost completely) mediated by ARS; thus: culture à response tendency à JS-level
total effect Individualism (Hofstede) job satisfaction β = .39* (p=.08)
mediation model
job satisfaction β = .20 (p=.39)
β = -.49** (p=.03) β = -.39 (p=.11)
Individualism (Hofstede)
ARS
§ same analysis with poor countries showed no total and no mediation effect
§ using several indicators for Individualism, ARS and job satisfaction in 8 different mediation models we always get the same pattern of results
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 15
§ statistical analysis: stepwise multilevel analysis with HLM 6 (Bryk & Raudenbush)
Multi-Level analysis
analysis by rich countries of WVS (N=21)
ARS individual individual level
national level ARS norm
JS
.072 -1.43* Step 2: ARS norm 5.31** (1)
Step 3: ARS individual .07 .078 4.31** (1)
Step 4: ARS individual – random slope 70.60** (1)
§ result: negative relationship with national ARS norm remains significant; on individual level no clear relationship of ARS and JS
§ same analysis with poor countries showed no effects at all
Step 1: Basic Model, controlled for age & gender
unstand. beta SE increase in model fit Chi-Square (df)
Step 5: ARS norm x ARS individual -.04 .889 0.002 (1)
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 16
JS and ARS on the individual level
7.10
7.56
8.03
8.50
8.96
Arbeitszufriedenheit
-1.25 -0.75 -0.25 0.25 0.75
ZustimmungstendenzARS individual
JS in
divi
dual
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 17
Conclusion
Results: § indeed: JS-measurements are influenced by response styles (ARS) § consequence: measurements from different cultures cannot be compared directly § national level: negative relationship, i.e. the higher the norm to agree, the more moderate the JS-measurement § individual level: no clear relationship, i.e. in some countries negative and in some countries positive § individualism-JS-relationship can be understood as a spurious correlation § instead: cultural values and their socialization leeds to a specific communication style that in turn influences the JS-measurements
In conclusion: Cross-culturally, we are not measuring „true“ satisfaction. To a great extent we are getting results that are an expression of culturally socialized communication norms!
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 18
Life satisfaction
All countries together: N-ARS 5-point-scale
‚agree/disagree‘-label WVS 1
Subjective Well-Being (Diener et al., 1995) -.52*** (40)
Life Satisfaction (Diener et al., 2000) -.36** (30)
Life Satisfaction (Suh & Oishi, 2002) -.50*** (29)
Happiness (Inglehart & Klingemann, 2000) -.42*** (51)
Life Satisfaction (Inglehart & Klingemann, 2000) -.54*** (51)
Ideal Life Satisfaction (Suh & Oishi, 2002) -.29 (29)
N-ARS 5-point-scale
‚agree/disagree‘-label WVS 1
- Poor countries -
N-ARS 5-point-scale
‚agree/disagree‘-label WVS 1
- Rich countries -
Subjective Well-Being (Diener et al., 1995) -.08 (11) -.58*** (21)
Life Satisfaction (Diener et al., 2000) -.02 (11) -.20 (11)
Life Satisfaction (Suh & Oishi, 2002) -.05 (11) -.53* (12)
Happiness (Inglehart & Klingemann, 2000) -.02 (22) -.57*** (21)
Life Satisfaction (Inglehart & Klingemann, 2000) -.19 (22) -.61*** (21)
Ideal Life Satisfaction (Suh & Oishi, 2002) -.03 (11) .02 (12)
Split into poor and rich countries:
1 pearson-correlation coefficient; signifikance: *** p < .01, ** p < .05, *p < .10; in parentheses: number of countries
1 pearson-correlation coefficient; signifikance: *** p < .01, ** p < .05, *p < .10; in parentheses: number of countries
Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 19
contact details: Dr. Christian Bosau Dipl.-Psych. & Master of HRM & IR
Center for Evaluation Faculty of Management, Economics and Social Sciences University of Cologne Herbert-Lewin-Str.2 50931 Cologne/Germany Tel. +49 (0)221 470-4120 [email protected]
Thanks for your attention