Mixed-Mode Approaches in the Generations and Gender SurveyPast Experience and Future Expectations

Aat LiefbroerNetherlands Interdisciplinary Demographic

Institute

Overview of the presentation GGP: what is it? Past experience

3rd wave of Dutch GGP 2010-2011 Future expectations

Pilot 4th wave in Slovenia 2011 Blueprint for GGP 2015

Conclusions

Overview

Aim of the GGP To advance our knowledge on intergenerational and

gender relationships, with a focus on understanding gender inequalities and generational differences

Internationally comparable data on relationships between partners and between parents and adult children across Western societies

Main vehicles: Generations and Gender Survey (GGS) and Contextual Database

Overview

Characteristics of the GGS Broad age-range (18-79), because understanding demographic

challenges such as population ageing asks for a focus on both young and old people

Large-scale surveys on population challenges (N≈10,000) to allow in-depth analysis

Panel design to allow for better causal analysis and studying processes of adaptation to change

Cross-national to allow for examining the influence of the social context (including the policy context)

Theory-driven questionnaire design Contextual macro-level database to allow for multi-level analyses

Overview

GGP Countries

Wave 1 18 countries (data for 12 countries currently available)

Wave 2 Bulgaria, Netherlands, Australia, Germany, France, Hungary, Italy, Russian

Federation, Georgia (Bulgaria and Germany to be released this year)

Overview

Challenges for the future of the GGP

Increasing the number of participating countries Extending the data-collection beyond three waves

Reducing fieldwork costs Application of mixed mode design?

Overview

Dutch GGP 1st wave 2004 ≈ 8,200 respondents, CAPI 2nd wave 2007 ≈ 6,200 respondents, CAPI 3rd wave 2010 ≈ 4,300 respondents, mixed mode Response rate in 3rd wave: 72% of wave 2 participants Costs per respondent in 2nd wave: ≈ €195 Costs per respondent in 3rd wave: ≈ €115

3rd wave Dutch GGP

Distribution by mode

18%

27%

55%CAPICATIWEB

3rd wave Dutch GGP

Mode distribution by age

020406080

100

25-44 45-64 65-89

WEBCATICAPI

3rd wave Dutch GGP

Mode distribution by educational attainment

020406080

100

low medium high

WEBCATICAPI

3rd wave Dutch GGP

Missing income info by mode

02468

10

CAPI CATI WEB

3rd wave Dutch GGP

Percentage that does not mention earnings from labour

Disclosure

11.11.21.31.41.5

CAPI CATI WEB

withoutwith

3rd wave Dutch GGP

Mean score on loneliness scale by mode (with and without controls)

Attrition information (I)

50

60

70

80

90

CAPI CATI WEB

3rd wave Dutch GGP

Percentage that gives permission to contact their partner to fill out a questionnaire

Attrition information (II)

5060708090

100

CAPI CATI WEB

3rd wave Dutch GGP

Percentage that gives permission to link data to register data from Statistics Netherlands

Attrition information (III)

020406080

100

CAPI CATI WEB

noyes, decide lateryes, participate

3rd wave Dutch GGP

Percentage that gives permission to be recontacted for the fourth wave

Conclusions3rd wave Dutch GGP

WEB more popular than expected Missing values on WEB limited Small, but systematic mode effects in disclosure WEB respondents are less willing to give permission for

additional or future involvement

Pilot 4th wave Slovenia 2011Pilot Slovenia

Participating institutions University of Ljubljana (Lozar-Manfreda, Petrič) University Utrecht (Hox, De Leeuw) NIDI (Kveder, Liefbroer)

Study 1: comparison of mode effects Study 2: comparison of mixed mode systems

Comparison of mode effects Aim: testing new pilot and examination of potential mode

effects Sample: regular participants in commercial web panel,

randomly assigned to different modes Examination of

Item non-response Scalability of item sets Distributional characteristics Length of interview Evaluation of interview

Pilot Slovenia

Comparison of mode effects Item non-response only slightly higher in web, with one

clear exception….

Pilot Slovenia

Income

0

10

20

30

40

50

CAPI CATI WEB

refusaldon't know

Percentage of refusals and ‘do not know’ on question on household income

Pilot Slovenia

Comparison of mode effects Item non-response only slightly higher in web, with one

clear exception…. Scalability of item sets still has to be analysed First analyses of distributional characteristics suggest

More disclosure on social desirable and difficult questions Slightly higher variance in responses

Pilot Slovenia

Example

010203040

CAPI CATI WEB

Percentage answering ‘yes’ to the question ‘Over the past 12 months, have you thought about breaking up your relationship?’

Pilot Slovenia

Comparison of mode effects Item non-response only slightly higher in web, with one

clear exception…. Scalability of item sets still has to be analysed First analyses of distributional characteristics suggest

More disclosure on social desirable and difficult questions Slightly higher variance in responses

Interview took somewhat longer in CATI mode Evaluation of interview most negative in CATI mode

Pilot Slovenia

Duration of survey

4550556065

CAPI CATI WEB

Pilot Slovenia

Subjective length of survey

0

10

20

30

40

CAPI CATI WEB

Percentage stating ‘yes’ or ‘definitely yes’ to question whether the interview was judged too long

Pilot Slovenia

Comparison of mixed-mode systems (I)

Aim: testing of alternative mixed-mode systems Sample: two-stage random sample from the population

register Examination of

Response rate of different mixed-mode systems Costs of different mixed mode systems Evaluation of modes by a random sample

Pilot Slovenia

Comparison of mixed-mode systems (II)

Comparison of four ‘systems’ CAPI → WEB → CATI (no incentive offered) CATI → WEB → CAPI (no incentive offered) WEB → CATI → CAPI (no incentive offered) WEB → CATI → CAPI (incentive of € 5 offered)

Difference in response rate across systems Difference in costs across systems

Pilot Slovenia

Blueprint GGP 2015Blueprint GGP 2015

Blueprint for GGP 2015 should be ready by the end of 2012 Questionnaire GGP 2015 Sampling design Set of fieldwork rules

Whether to opt for mixed-modes, and if so when? Is it an option in the first wave (or in the first wave with a

refreshment sample), or in later waves only? What kind of mixed-mode system to prescribe?

ConclusionsConclusion

Overall, WEB seems to perform quite well in terms of answering patterns (item nonresponse, scalability and distribution of responses)

WEB leads to very considerable cost reductions The big issue that is not yet clear is whether attrition is

negatively affected Use of WEB in a first wave seems questionable (but pilot will give

partial answers) Use in later waves could lead to higher dropout in future waves as

a result of the legal requirement to ask for permission for re-contacting

Thank you for your attention!

Topics included Provision of care to older adults Successful ageing Economic participation of women and pre-pensioners Realisation of fertility intentions Balancing work and family life Gender equality in household and childcare labour

Overview

Members of Consortium Board Netherlands Interdisciplinary Demographic Institute (NL) Institut National d’Études Démographiques (FR) Bocconi University (IT) Statistics Norway (NO) Demographic Research Institute (HU) Norwegian Social Research (NO) University of Ljubljana (SL) Utrecht University (NL) University of York (GB) Max-Planck Institute for Demographic Research (DE) Erasmus University Rotterdam (NL) Population Unit of the UN Economic Commission for Europe