Microsimulation of Survey Collection

Yves BélangerKristen Couture26 January 2010

2

Outline

Motivation Main aspects of microsimulation Overview of the system A short demo A few results Future work

3

Motivation Ultimate goal: make CATI collection more

efficient proactive collection management

Recent initiatives in the field Experimentation with time slices, cap on calls,

calling priorities, Z-groups, ... Takes time, lack of control, costly(?), results not

always easy to interpret

Need for a controlled environment, where the impact of each aspect can be isolated

4

Main Aspects of Microsimulation What is microsimulation?

A modelling technique that operates at the level of individual units, such as persons, households, vehicles, etc.

For us: a "virtual collection" system

What elements are we considering? The cases (sampled units) The servers (interviewers) The call attempts The waiting queue(s) The rules of the call scheduler (flows and priorities)

5

Main Aspects of Microsimulation (cont'd)

What do we want to simulate?1. A random component: the result of each

call attempt Use existing BTH data with appropriate

statistical models

2. A deterministic component: how the cases flow through the system Use a simulation software to replicate

Blaise: SAS Simulation Studio

6

Overview of the System

Pre-existing BTH from Survey

Model Call Outcomes Model Call Duration

Simulation

Collection Parameters

7

Overview of the System (cont'd)

Call outcome Modeled using CSGVP 2004 BTH data Five outcomes derived from BTH

outcome codes Unresolved (eg. Busy signal, wrong #) Out of Scope (eg. Cell phone, Business) Refusal Other Contact (eg. Ans. Machine, appointment) Respondent

8

Overview of the System (cont'd)

Used Multinomial Logistic Regression

7 parameters entered into model: Afternoon – 1 if call made between 12 and 5 Evening – 1 if call made between 5 and 9 Weekend - 1 if call made on weekend Resid – 1 if initial status was residential Unresolved – 1 if call history is only unresolved Refusal – 1 if history shows at least one refusal Contact – 1 if history shows at least one contact

i = 1..nj = 1..k

9

Overview of the System (cont'd)

Calculate probability for each of the five possible outcomes using estimated betas and collection parameters

10

Overview of the System (cont'd)

Call duration Modeled using existing CSGVP 2004 BTH

data Modeled distributions for each of the 5

outcomes

11

Overview of the System (cont'd)

Components of model Input

Allows user to enter parameters via SAS data sets

12

Overview of the System (cont'd)

Clock Creates Time Parameters including

Afternoon, Evening, Weekend, and Time Slice by reading the current simulation time

13

Overview of the System (cont'd)

Queuing System Cases are created and enter a queue

waiting to be interviewed

14

Overview of the System (cont'd)

Determining Call Outcome Uses probability formulas to determine call

outcome: Unresolved, Out of Scope, Other Contact, Refusal, Respondent

15

Overview of the System (cont'd)

Call Center Interview takes place Call duration is simulated Ability to control interviewer schedule

16

Overview of the System (cont'd)

Finalizing Cases Case exits system when…

Outcome code = OOS or Respondent Cap on Calls is reached

Cap of 20 for Residential Status Cap of 5 for Unknown Status

Number of Refusals=3 A BTH file is created as output in terms of a

SAS dataset

17

A Short Demo

18

A Few Results

Simulation with 10,000 cases for 30 days of collection

Interviewer Agenda Shift 1 (9am-12pm): 10 interviewers Shift 2 (12pm-5pm): 10 interviewers Shift 3 (5pm-9pm): 10 interviewers

* Note: No Time Slices in this example

19

A few results (cont'd)

Finalized Cases and Response Rate

Distribution of Outcome Codes

20

A few results (cont'd)

Impact of Changing Parameters Number of Interviewers

Length of Collection Period

21

A few results (cont'd) Changing the Time Per Unit Cap on Calls is in Effect

22

Future Work Continue improvements to system

To outcome model More explanatory variables Distinguish between hhld and person contacts

To simulation system Implement time slices Improve priorities

Presentation to JSM (incl. article) Potential cooperation with Census Other?... will depend on available budget