Covering the general population by Internet

interviewing

Marcel Das

19-04-23

1

MESS Project

• core element: LISS panel

• other key elements:

- core study

- experimental modules

- new forms of data collection

- links with administrative data

- special groups

19-04-23

2

Core element: LISS panel

Online panel of 5,000 households 8,000 individuals (>= 16 years)

Questionnaires each month, 30 min.

Incentive 15 euro an hour (average)

For scientific use, at no cost

19-04-23

3

LISS panel Online interviews as method, but:

Probability sample drawn from address sampling frame of Statistics Netherlands

Includes households without Internet access (less than 15%): CentERdata provides equipment

Contacted by letter, telephone or visit19-04-23

4

simPC

Very small and silent

Only the most frequently used functions

Automatic maintenance, safety

Simple operation and readable screens

Installation and support19-04-23

5

19-04-23

6

Internet interviews

letter and brochure

short interview

simPC and adsl

households with Internet

households without Internet

confirmation

panel question

Recruitment

19-04-23

7

Internet interviews

letter and brochure

short interview

simPC and adsl

confirmation

panel question

Primary response

target: 80%

Secondary response

target: 60%

Tertiary response

target: 53%

Response

Recruitment

experiment

19-04-23

8

Experimental design CATI CAPI N

Special letter

Standard letter

Special letter

Standard letter

Prepaid incentive

Introduce panel in letter 0 euro 10 euro 200

Introduce panel after interview

20 euro 50 euro 300

Promised incentive

Introduce panel in letter

0 euro 10 euro 400

Introduce panel after interview

50 euro 20 euro 300

N 200 200 400 400 1200

CATI = Computer Assisted Telephone InterviewingCAPI = Computer Assisted Personal Interviewing

19-04-23

9

Pilot experimental groups1. CATI: special letter - no incentive - Panel in letter and

brochure

2. CATI: special letter - promised incentive E. 50 - Panel introduced after interview

3. CATI: Standard letter - prepaid incentive E. 10 - Panel in letter and brochure

4. CATI: Standard letter - prepaid incentive E. 20 – Panel introduced after interview

5. CAPI: special letter - prepaid incentive E. 50 – Panel introduced after interview

6. CAPI: special letter - no incentive - Panel in letter and brochure

7. CAPI: Standard letter - promised incentive E. 10 - Panel in letter and brochure

8. CAPI: Standard letter - promised incentive E. 20 - Panel introduced after interview

19-04-23

10

Pilot natural groups

CAPI

1. Unknown or no telephone number: 30%

CATI and CAPI

2. Known (landline) telephone number: 70%

19-04-23

11

Observed response in CATI versus CAPI by subpopulation CATI

households with telnr

%

CAPI Households with telnr

%

CAPI Households

without telnr %

Primary response 65 58 44 Primary refusals 22 25 25 Primary not reached 9 14 25 Primary not usable 3 3 5

Effect of mode (1)

19-04-23

12

Observed response in CATI versus CAPI by subpopulation CATI

households with telnr

%

CAPI Households with telnr

%

CAPI Households

without telnr %

Primary response 65 58 44 Secondary response conditional on primary

67 84 88

Primary * secondary 45 50 39 Tertiary response conditional on secondary

80 72 64

Primary * tertiary 35 35 25

Effect of mode (2)

19-04-23

13

incentives increase response rates

- effect is large from 0 to 10 euro (increase of 15%)

- effect is not significant from 10 to 20, 20 to 50 euro (about 3%)

prepaid works better than promised

Effect of incentives

19-04-23

14

Effect of information

no effect of letter content

no effect of timing panel introduction

effect of seeing/reading brochure

Set up main recruitment

• Combination CATI – CAPI, follow up CAPI

• Prepaid 10 euro incentive

• Promised extra 10 euro incentive for starting

• Attention to design letter and brochure

19-04-23

15

Results

19-04-23

16

Pilot versus main response rates

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

primary secondary tertiary

response

pro

po

rtio

n o

f to

tal

Pilot

Main

Non-response patternsSimilar to those of other leading

scientific panels

Superior to commercial access and volunteer panels:

- no coverage problems

- no selfselection

19-04-23

17

Attention for difficult groups

Correction by oversampling in refreshment of 2009 (with Statistics Netherlands)

New immigrant panel (with Cross-cultural Psychology, Tilburg University and Statistics Netherlands)

19-04-23

18

Correction of bias by stratified refreshment

sample 2009

19-04-23

19

GroupPopulation

2007 %

April 2008Bias

April 2009Bias

Jan 2010 Bias

Age >=70 11.6 -5.4 -5.4 -2.8

Living alone 19.2 -5.4 -5.0 -2.4

Non-western immigrant

9.6 -5.1 -4.8 -3.7

Persons

New forms of data collection

19-04-23

20

Use of graphical or animated presentation

19-04-23

21

Biomarkerspilot study: blood cholesterol, saliva cortisol, and waist

circumference

19-04-23

22

More biomarker experiments...- accelerometers

- advanced scales (based on bioelectrical impedance)

link objective data with self-report data

high frequency measurement

19-04-23

23

Use of the LISS panel

19-04-23

24

Questionnaires

The “household box”

Experimental modules open to academic researchers at no cost

Core study

19-04-23

25

Experimental modules

Who can use the LISS panel:every researcher who wants to collect data for scientific, policy or societal relevant research

Scientific research: Academic researchers, irrespective of nationality, can use the LISS panel at no cost

Proposals can be submitted throughout the year

19-04-23

26

19-04-23

27

Examples: subsample

Effects of illness babies in first year on labour participation parents:

Pregnant women, follow 3 years!

Adolescent/teen identity and the Internet:

12-18 year olds

Random subsample or experimental groups

19-04-23

28

Procedure:

Submit to: a.c.scherpenzeel@uvt.nl

Typically be between 2 and 5 pages

Evaluation: Board of Overseers

Decision: 1-3 months Accept Revise and resubmit Reject

(http://www.lissdata.nl/lissdata/Proposals/How_to_Submit_a_Proposal)

19-04-23

29

Criteria:

Scientific potential: predicated on literature; sound analytical

framework

Fit in LISS panel: value of using an Internet survey

Feasibility: questions doable for respondents?

Burden: length and target sample

(http://www.lissdata.nl/lissdata/Proposals/How_to_Submit_a_Proposal)

19-04-23

30

Requirements:

Data protection statement

Single user access

Publication list

Data available to other researchers!

Proposals: Summary (May 2010)

19-04-23

31

Decision Board of Overseers Count

Undecided (in review or revision) 14

Accepted 49

Rejected 10

Total 73

Proposals: Examples

19-04-23

32

Proposal Affiliation(s) Perceptions of nutrigenomics Twente University (NL)

Mental health Radboud University, Nijmegen (NL); Emory University (USA)

Subjective well-being Max Planck Institute for Human Development, Berlin (Germany)

Computer crime victimization Leiden University (NL)

Volunteer work University of Duisburg / Mannheim (Germany)

Open-ended questions University of Michigan (US)

Identification of guilt sensitivity University of Laval (Canada)/Copenhagen (Denmark)

Available data Who can use the LISS data:every researcher who wants to use data for scientific, policy or societal relevant research

Use of data is free of charge (unless data are used for research funded by means of external sources)

Available data: longitudinal core study proposed studies

19-04-23

33

Longitudinal core study

Household and familyEconomic situation and housingWork and schoolingSocial integration and leisureHealthPersonalityReligion and ethnicityPolitics and values

19-04-23

34

LISS data

Disseminated through website / data archive:

http://www.lissdata.nl

Database is based on an internationally used specification for describing social science data (DDI)

One of the first implementations of DDI version 3

19-04-23

35

8-6-2010

36

Why use DDI? A lot more work?

Simple links to datasets and codebooks: fine for single datasets and once-a-year panel studies

LISS data: complex and extensive

Will the data and documentation still be accessible in 10 years?

8-6-2010

37

Complex data

Q5Panel

members

Q8Q7

Q6

Q1 Q2 Q3 Q4

Core questionnaire

ProposalsQ9

8-6-2010

38

Extensive data Example: “Do you know which panel

members are vegetarian?” Asked somewhere, but in which study?

Example: “Did you ever measure political efficacy?”

Asked sometime, but when exactly?

Example: “Can I estimate the housing expenditure of impulsive persons?”

Which items measured impulsiveness?

8-6-2010

39

DDI 3 & LISS website

Search entry with keyword and on concept, topic, and study (relational)

Can handle longitudinal data (waves)

Exchangeable with other data archives

Flexibility for future applications (Example: iPod version)

October 2010 Publisher: Routledge / Taylor & Francis Group

19-04-23

48