Differential item functioning analysis of

the COPSOQ across ten countries:

the NEXT Study

Jian LI, MD, MPH, PhD, Dr.rer.sec.

and the NEXT Study Group

Institute of Occupational and Social Medicine

University of Düsseldorf, Germany

The 4th International Workshop on COPSOQ

Gent, Belgium, 19-20 September 2013

World Happiness Report 2013

http://unsdsn.org/files/2013/09/WorldHappinessReport2013_online.pdf

Question:

Can the scores be used to

compare across countries?

Whether questionnaire scales capture the same

concepts across countries?

Whether respondents interpret and respond to

items in the same way?

Differential Item Functioning (DIF) analysis is

traditionally used in Psychology, Education, and

Business/Management disciplines. Recently, it

has been applied to mental health/quality of life

measurement.

International Comparison based on questionnaire scores

Translation inequivalence

Confounding effect

Cultural differences

Main sources of DIF:

Orhede E, et al. Item bias in indices measuring psychosocial work

environment and health. Scand J Work Environ Health, 2000, 26 (3): 263-

272.

Pelfrene E,et al. Scale reliability and validity of the Karasek 'Job Demand-

Control-Support' model in the Belstress study. Work & Stress, 2001, 15 (4):

297-313.

Kristensen TS, et al. The distinction between work pace and working hours

in the measurement of quantitative demands at work. Work & Stress, 2004,

18 (4): 305-322.

Pejtersen JH, et al. The second version of the Copenhagen Psychosocial

Questionnaire. Scand J Public Health, 2010, 38 (3 Suppl): 8-24.

DIF analysis in research of work stress

questionnaire within one country

Questionnaires

JCQ

ERI

COPSOQ

DIF methods

• Three-way contingency tables

• Item response theory

• Logistic regression

Choi B, et al. A cross-national study on the multidimensional characteristics of the

five-item psychological demands scale of the Job Content Questionnaire. Int J Behav

Med, 2008, 15 (2): 120-132.

Tsutsumi A, et al. Application of item response theory to achieve cross-cultural

comparability of occupational stress measurement. Int J Methods Psychiatr Res,

2009, 18 (1): 58-67.

DIF analysis in research of work stress

questionnaire across countries

10 participating EU countries

United Kingdom

Finland

Norway

The NEXT Study

Germany

Italy

France

The Netherlands

Belgium

Poland

Slovakia

1 participating Asian country

China

COPSOQ used in the NEXT Study

Quantitative Demands (QD)

QD1: Do you have to work very fast?

QD2: Is your workload unevenly distributed so it piles up?

QD3: How often do you not have time to complete all your work tasks?

Hardly/ever (1), Seldom (2), Sometimes (3), Often (4), Always (5)

Possibilities for Development (PD)

Quality of Leadership (QL)

Meaning of Work (MW)

Insecurity at Work (IW)

Independent judgmental review on the

translation comparability of the COPSOQ items

England

English

Germany

German

Italy

Italian

France

French

Netherlan

ds

Dutch

Belgium

French/Flemi

sh

Finland

Finnish

Poland

Polish

Slovakia

Slovak

China

Chinese

QD

QD1 Reference A A A A A A A A A

QD2 Reference A A B A B B A A A

QD3 Reference A A A A A A A A A

Translation equivalence:

A: no difference;

B: negligible difference (no conceptual difference, but some wording is not perfect);

C: moderate difference;

D: large difference.

Characteristics of the NEXT Study subjects

Note: * p < 0.05, difference among countries by Analysis of Variance or Chi-Square Test

England

(N=1694)

Belgium

(N=1954)

Germany

(N=2145)

Finland

(N=1869)

France

(N=3657)

Italy

(N=3452)

Netherlan

ds

(N=2186)

Poland

(N=3171)

Slovakia

(N=2725)

China

(N=3088)

Age

(mean ± SD) *

39.70 ±

9.69

37.48 ±

9.21

37.21 ±

9.28

41.18 ±

9.97

38.81 ±

9.36

37.18 ±

7.50

36.98 ±

9.48

37.63 ±

7.22

38.40 ±

8.65

30.94 ±

8.29

Marital status (N, (%)) *

Single 198

(11.69)

288

(14.74)

512

(23.87)

254

(13.59)

558

(15.26)

447

(12.95)

318

(14.55)

156

(4.92)

165

(6.06)

1193

(38.63)

Cohabitated 1496

(88.31)

1666

(85.26)

1633

(76.13)

1615

(86.41)

3099

(84.74)

3005

(87.05)

1868

(85.45)

3015

(95.08)

2560

(93.94)

1895

(61.37)

Position rank (N, (%)) *

Low 1176

(69.42)

1750

(89.56)

1769

(82.47)

1624

(86.89)

3460

(94.61)

3118

(90.32)

2172

(99.36)

2829

(89.21)

2441

(89.58)

2446

(79.21)

High 518

(30.58)

204

(10.44)

376

(17.53)

245

(13.11)

197

(5.39)

334

(9.68) 14 (0.64)

342

(10.79)

284

(10.42)

642

(20.79)

Shift work (N, (%)) *

Day shift 919

(54.25)

868

(44.42)

835

(38.93)

623

(33.33)

2416

(66.07)

1355

(39.25)

600

(27.45)

614

(19.36)

929

(34.09)

1036

(33.55)

Rotating shift 775

(45.75)

1086

(55.58)

1310

(61.07)

1246

(66.67)

1241

(33.93)

2097

(60.75)

1586

(72.55)

2557

(80.64)

1796

(65.91)

2052

(66.45)

DIF analysis of COPSOQ

Belgium Germany Finland France Italy Netherlan

ds Poland Slovakia China

% of

sample

s with

DIF

Quantitative Demands

QD1 0 0 0 0 0 0 0 0 + 11%

QD2 0 0 + 0 0 0 + - 0 33%

QD3 0 0 - 0 - 0 - + - 56%

Note: England as reference (adjusting for age, marital status, position rank, and shift work)

0: non-DIF item

+: DIF item with higher item score for country than for England at a given score

-: DIF item with lower item score for country than for England at a given score

Measurement of quantitative demands

QD1: Do you have to work very fast?

QD2: Is your workload unevenly distributed so it piles up?

QD3: How often do you not have time to complete all your work tasks?

Kristensen TS, et al. The distinction between work pace and

working hours in the measurement of quantitative demands

at work. Work & Stress, 2004, 18 (4): 305-322.

Work pace

Work time

Means of Quantitative Demands

Note: ANCOVA adjusting for age, marital status, position rank, and shift work

Means of Quantitative Demands

Note: ANCOVA adjusting for age, marital status, position rank, and shift work

Attention should be paid for the international comparison of psychosocial work characteristics.

DIF analysis is useful for such comparison study: more sensitive than routine methods.

Conclusion

Thank you

for your attention!