Increasing Efficiency in Data Collection Processes

Arie Aharon, Israel Central Bureau of Statistics

General

In recent years there has been a significant increase in the Central Bureau of Statistics‘ activities An increase in the number of surveys An increase in the number of administrative files received

by the CBS The increase in administrative files has not decreased

the number of interviewers, in fact their number has doubled (450)

General – cont. The large number of interviewers generated the

need for new methods and means to collect data. It was decided to expand use of the Internet and to

use administrative files more efficiently.

The purpose of this presentation is to present the difficulties in implementing this decision.

Conducting Surveys Online

The response rate was low in surveys conducted online.

The question was raised – Currently, is the cost of development justified in light of the low response rate? For this reason the topic was abandoned.

The increase in the number of surveys brought the topic again into focus and it was decided to develop an online response process in any case.

We decided to examine three surveys

Business Trends Survey The questionnaire is sent to company managers.

Higher Education Survey The questionnaire is sent to students in their second year

of studies. Consumer Confidence Survey

The survey is based on a sample from the Social Survey, in which email addresses are collected in the field. The response will be examined in February 2014.

How data was collected online until January 2013: For reasons of information security, online response

was conducted as follows: A letter was sent by mail to each sampled person

with a user name and password. Each sampled person had to enter the CBS

website and download the questionnaire with the password provided in the letter.

The results for the Business Trends Survey – a response rate of 7% – a very low result.

Examining the Response Rate in the Higher Education Survey We expected a higher response rate in this survey.

The survey involves students who use the Internet regularly.

We decided to examine the response rate in the following possibilities: Response by mail only Online response only Option of online or mail response Completion by telephone

Enumeration Results by Experimental System

Total sampled persons

Mail Internet Telephone Total who were enumerated

Number % Number % Number % Number %

Universities – mail only 3100 1525 49 41 1 1101 36 2667 86Universities – Internet only 3100 107 3 1379 45 1206 39 2692 87Universities – mail and Internet 3100 680 22 906 29 1067 35 2653 86Colleges of education – mail only

334 121 37 4 1 141 43 266 82

Colleges of education – Internet only

333 12 4 108 33 146 45 266 82

Colleges of education – mail and Internet

333 77 23 64 19 124 37 265 82

Conclusions from the Response Rate Results in the Business Trends and Higher Education Surveys

As mentioned, the online response rate was low in the Business Trends Survey

In the Higher Education Survey, the response was relatively higher. However, a large percentage responded by mail or CATI.

The question arises: Why is the online response rate low even among a population in which the percentage would be expected to be higher?

Our Estimations for the Reasons for the Low Response Rate

The method of response was not user-friendly - sending a letter with passwords is inconvenient and not easy to use.

Information security concerns. Technical problems on the sampled persons’

computers.

Solutions Proposed

Solution for the problem of user-friendliness: Sending an email with a link to the online questionnaire that can then be filled out online.

The difficulty in implementation – information security How would we know who responded and how would we

know if the respondent was the sampled person? We knew this would require an efficient and elegant

solution.

The Solution Used

A letter with passwords would not be sent. An email would be sent to each sampled person with a link to

the online questionnaire. Every questionnaire opened would have a company number

for the sampled person or business. (The number is used for CBS purposes alone). The person reporting the information does not see it.

This method solves the problem of user-friendliness.

Implementing the Method

We started this method in February 2013 and the number of respondents increased from 7% about 14% of the sample in this month.

Weekly reminders were sent to those who did not respond.

The result of the reminders was an increase in the rate of response by fax.

Reasons for Non-Response in March

By means of CATI, we asked those who did not respond online, why? The answers were: 25% answered that they wish to continue responding as before, by fax

or telephone. 25% answered that they cannot answer online due to issues of

information security. A few percent responded that they have no email address. A few percent responded that they had problems in installation or

operation on the computer. Others gave an evasive answer and said that they would respond next

month.

Conclusion

A process of training the sampled persons how to respond online should be implemented, especially for those who respond by fax.

The importance of a transition to a 'green' paperless environment should be explained.

Support should be provided for technical problems. One should explain that the process involves a high

level of information security.

Summary of Actions Taken

The process of training yielded results: We reached a current 50% response rate among those with an email

address. This demonstrates that:

A user-friendly system is required; The importance of responding online must be explained to the

sampled persons. It was effective to send reminders every four days. In online responses, all the questionnaire items were completed fully,

which indicates a good quality response. The sampled person fills out the questionnaire in his free time.

Summary of Actions Taken (cont.)

The larger the company, the greater the chances for an online response.

The industry with the highest rate of online response was the manufacturing industry, followed by the service industry.

These conclusions led us to take action (through explanation and persuasion) with the large corporations and industries, which have higher response rates, so as to raise their response rate even more.

Summary of Actions Taken (cont.)

There was a concern among interviewers who lost work hours due to the online response. We provided them with work hours as a result of receiving new

surveys. Those who responded online were mainly Deputy CEOs or

CFOs.

The new method has led to an increase in response and from this we can conclude that we need to develop it for additional surveys and thereby economize on interviewers’ hours.

Increased Use of Administrative Files

In recent years, the CBS has received dozens of administrative files in which the following problems were discovered: Each file has a different structure. Each file had a different technology - different

applications were used by each information provider.

Administrative Files (cont.)

Each file would be delivered to a specific person in the office who had responsibility for it; he/she would then use the file according to his/her own understanding

There was no uniformity among the files and there were no shared definitions.

As a result, it was decided to develop one uniform system for all files. We decided to start with files in the field of education.

Definitions for a Uniform System

1. Organization-wide system capable of receiving, improving, maintaining and retrieving data.

2. Metadata definition for the various files.3. Processing files by defined standards and in a

documented manner. 4. Improving information security. 5. The ability to produce integrated products. 6. Continuous absorption of files.

Uniform System (cont.)

The system was begun in education, as mentioned, and has been successful. It has created standardization for all the files obtained.

An expert was defined for each file; his/her role is to know the file well and to treat it according to the defined standards. The expert is responsible for the production of the

statistical product as well. This system will lead to better utilization of existing

information in the CBS. Proper use will lead to a reduction in questions in the classical

surveys.

Summation

The increasing use of the Internet and administrative files will lead to: A reduction in the response burden Improvement in the quality of dataA reduction in the load of managers on various

levelsMore free time to improve the system of

management and supervision of work processes.

Thank You!

aharon@cbs.gov.il