redesigning french internet business data collection at insee: the coltrane project

13/09/2013

Béguin Jean-MarcBusiness statistics director (Insee)

Redesigning French Internet Business data collection at Insee: the Coltrane project4th International Workshop on Internet survey methods (Daejeon)

09/13/2012

4th International Workshop on Internet survey methods2

Outline of the presentation

Thanks and context

1. The existing system in France1. Surveys2. Tools

2. The Coltrane project (four work packages) 1. Authentication2. Contacts management3. Portal of customized services4. Data collection platform

09/13/2012


1.1 Business surveys and National Statistical Authorities (NSA)

NSA : Insee and 5 or 6 ministerial statistical Offices (MSO) (In 2010 Sessi, the MSO for industry, & Insee have merged)

80 Business surveys (of many kinds)- 30 Insee and 50 MSO (+ others)- About 500,000 enterprises (in over 3,5 millions) receive at

least one questionnaire- The sum of all the samples sizes is 700,000- Taking into account the periodicity of the surveys (12

questionnaires for a monthly survey), 1 800,000 forms are sent each year by the NSA.

"sent" can be - By mail for paper forms- Through a website for dematerialized forms

09/13/2012


1.1 The present situation regarding Internet Usage So far, put surveys on the web has not been a priority 2012 situation:

Insee MSOsNumber of surveys 30 47

Web surveys 22 16Ratio of Internet usage 73.3% 34%Sum of the samples sizes 380,000 305,000

Web surveys 178,000 80,000Ratio of Internet usage 46.8% 26,2%Number of questionnaires (weighted by periodicity) 895,000 875,000

Web surveys 654,000 253,000Ratio of Internet usage 73,1% 28.9%

09/13/2012


1.1 Factors explaning the ratio of Internet answers

Heterogeneous from one office to another Heterogeneous from one survey to another Possible factors

- The age of the process- The type of variables collected- The number of variables and the length of the

questionnaire - The frequency of the survey - The size of the responding firms - The renewal of the samples - The ergonomics of the human interface - the efforts made by Insee staff- ...

09/13/2012


1.2 The tools to implement Internet Surveys

First Internet survey : 2000 (Sessi)- Presently a second-generation application (ASP)- Now Insee took over the portal (since 2010)

Beginning at Insee : 2004 (application called CRPI)- This application is now 8 years old (JAVA)- Designed for repetitive short term surveys

For specific purposes, new ways of collecting dematerialized data have been developed recently:- Upload / download application (eg EXCEL sheets)(2009)- Blaise IS (Dutch software) (2009)- Voozanoo (a free PHP software from Epiconcept) (2008)

Conclusion: many tools ! Complex for us and for enterprises

(e.g. one portal for each tool)

09/13/2012


1.2 An analysis grid to compare the tools (2010)

Originally based on the GSBPM Within the sub-processes we have defined 23 functions

such as :- Build data collection instrument (create the form)

- Create a form on one web page………………………… - Create a form on several web pages or screens …….. - Create simple filters (from one question to another)…. - Create complex filters (from page to page)…………….

- Build or enhance process components - Generate both login & password………………………. - Take into account the campaign dates ……………….. - Customize the questionnaire (e.g. with previous answers of the same

enterprise)

This grid helped us to analyse the pros & the cons of the tools and of the whole system

09/13/2012


1.2 There is a need for a new project

CRPI is the best; but- old (9 years)- based on a proprietary software (to implement java) - its maintenance is costly;- Not designed for big annual surveys

No common directory nor common governance nor common portal

Technically complex moreover: in april 2011, the government requested that

all statistical surveys should be put on the web; existing tools couldn’t allow it (specially for the SBS survey)

09/13/2012


2 The Coltrane project

Platform offering a range of technical & business services

Four independent work packages:- Authentication- Contact management- Portal of customized services (for contacts)- Data collection platform (divided into 4 blocks)

Each WP is composed of several functions accessible through services

A specific batch is planned to transform the SBS-survey into a web-based survey- Innovative because forms should be generated from

their DDI description

09/13/2012


2.1 The Coltrane project: authentication

Independent from the collection itself We do not think about using PKI (Public Key Infrastructure)

but a LDAP directory to authenticate all the contacts Any « contact » in an enterprise should have a unique

login & password We shall offer the possibility of using this service to

all the MSOs- Enterprises do not make any difference whether the

survey is carried out by Insee or a MSO- They receive questionnaires from both

But each ministry has its own Internet policy and have already built something for their own web-sites

we are not sure to be successfull !

09/13/2012


2.2 The Coltrane project: contact management

Independent from the collection itself, but linked to the authentication service (to exchange credentials)

Can be used to manage all kinds of contacts (for example, people buying data)

Insee staff will be able to:- Create or delete a contact and its respective

characteristics- Renew credentials - Display the list of contacts of a given enterprise - Display the list of the surveys of a given contact - Assist the web respondents and inform them about the

coming surveys- Communicate with all the contacts of a given survey- …..

09/13/2012


2.3 The Coltrane project: portal of customized services

A unique Internet portal will present all the surveys (different from today)

Including the MSO surveys (as far as they have accepted to use our portal)

The portal will be customized for each contact so that, once authenticated, he can:- only see the surveys he is interested in- modify his own personal parameters (phone number, e-

mail, address, etc.)- transfer his “answering power” to a colleague- display his previous answers- look at the remaining surveys he still has to answer as

well as their deadlines

09/13/2012


2.4 The Coltrane project: Data collection platform

Main part of the project Block 1: offering different collection modes

- Classical web form- Paper form (exchanged by mail)- Electronic data Interchange (EDI)- Upload / download service

If several collection modes exist, the questionnaires should be generated from the same metadata flow

Block 2: running the collection (eg building management tools)- Opening/closing a campaign - Edit statistics for the collection- Organize reminders- Start litigations with non respondents

09/13/2012


2.4 The Coltrane project: Data collection platform

Block 3: generating questionnaires- Aims at maintaining the coherence between a questionnaire

& its description (metadata)- We will use a DDI model to model the questionnaire- Then the form should be generated from its DDI description - We are testing some softwares helping either the generation

or the DDI description- This could allow us to put the SBS survey on the web

Block 4: managing the collected data- In our view, centralizing all the collected data (whatever the

data collection mode) is of utmost importance- The following functions will be developed : retrieve and

control the data, adapt the formats of data flows, keep an image of the enterprise answer

09/13/2012


2.5 The Coltrane project: Some specific issues

Give the possibility for one enterprise to keep a copy of its answer and for several respondents in the same firm to fill up a single form

Find the best trade-off between- Sending back quickly new credentials to a contact (when he

has lost them)- Keeping a good level of security

Use the dialogue possibilities of Internet - to help firms to answer to questions with hundreds of

possible codes- to implement dynamic controls (eg with previous answers)

or explanations (in case of “peculiar” answers) Make a proof an enterprise has received a web form and

has not replied (for mandatory surveys, to impose fines)- Legal and technical issue

Thanks for your attention !

ContactM. Jean-Marc BéguinTél. : +33 1 41 17 50 41Courriel : [email protected]

Insee18 bd Adolphe-Pinard75675 Paris Cedex 14

www.insee.fr

Informations statistiques :www.insee.fr / Contacter l’Insee09 72 72 4000(coût d’un appel local)du lundi au vendredi de 9h00 à 17h00

The Coltrane project is still in a conceptual phase and should be completed by 2014 or 2015.

http://www.insee.fr/




redesigning french internet business data collection at insee: the coltrane project

Documents