Download - Kathy Downey, Dee McCarthy, William McCarthy, and Brian Meekins U.S. Bureau of Labor Statistics
1
Encouraging the Use of Alternative Modes of Electronic Data Collection: Results of Two Field Studies
Kathy Downey, Dee McCarthy, William McCarthy, and Brian Meekins
U.S. Bureau of Labor Statistics
The opinions expressed here are those of the authors and do not represent official policy of the Bureau of Labor Statistics.
2
BLS Internet Data Collection
First begun in the 2002 survey year Termed the “IDCF” – Internet Data
Collection Facility Uniform and secure structure for
employers to submit data for BLS surveys
Designed to look very similar to the Survey of Occupational Injuries and Illnesses data collection booklet
Savings in mail costs, print costs, data entry costs
3
Survey of Occupational Injuries and Illnesses
Federal/State cooperative survey Sample of 230,000 establishments per year
(private, State and local gov’t.) Data collected based on OSHA logs –
information on illness/injury: Total recordable cases, Cases with days away from work (with more
about demographics and case characteristics), Cases with days of job transfer or restriction
Hardcopy is 12 pages, envelope and insert Web form looks similar to hardcopy form
4
Internet Data Collection: Number of Days-Away-From-Work Cases Submitted
10,595
23,36929,157
51,882
29,551
50,707
0
10,000
20,000
30,000
40,000
50,000
60,000
2002 2003 2004
Estabs Cases
5
Purpose of Tests
Expanding use of IDCF to Reduce processing time Capture narrative text of days-
away-from-work cases Lower printing and postage costs
6
Tests 2005 SOII: January – June 2006
1st mailing: January 2006 1st non-response mailing: Feb 14 2nd non-response mailing: April 5 Handed over to States for calls (end test): May 9 Close-out: June 26
2006 SOII: January – July 2007 1st mailing: January 4 2007 1st non-response mailing: March 2 2nd non-response mailing: April 20 Handed over to States for calls (end test): TBD Close-out: July 17
7
2005 Test: Sample
2,000 units in each of three tests and a control group
6,688 units that used IDCF in 2004 and were in 2005 sample – termed INET units and received the Test 2 booklet
8
2005 Test: Booklets Test 1
4 page booklet with electronic options* and a phone number to request standard booklet – all 3 mailings
Test 2 plus INET 4 page booklet with electronic options* and a phone
number for help – all 3 mailings Test 3
4 page booklet with electronic options and a phone number for help – received standard booklet on second nonresponse mailing
Control group standard booklet
*use IDCF or send an e-mail to get an electronic copy of the survey
9
10
11
Analyses
Performance Response rate
# resp / “viables” (excludes OOB, OOS, duplicates)
Internet response rate # internet resp / “viables”
Burden Percentage of units contacting Help
phone number Added processing time and costs
12
2005 Internet Data Collection: Establishments and Days-Away-From-Work Cases
10,595
23,369
29,157
51,882
29,551
50,70753,575
70,783
010,00020,00030,00040,00050,00060,00070,00080,000
2002 2003 2004 2005
Estabs Cases
13
Survey Year 2005 Tests Results(at end of test – May 9, 2006)
71.1
46.1
71.3
49.5
73.5
47.2
90.3
76.4 78.4
21.7
80.6
25.0
0102030405060708090
100
Test 1 Test 2 Test 3 INET ControlGroup
Allunits
Response rate IDCF response rate
14
87.1
49.2
86.6
53.4
89.8
49.6
96.1
78.5
91.6
23.1
91
25.8
0102030405060708090
100
Test 1 Test 2 Test 3 INET ControlGroup
Allunits
Response rate IDCF response rate
Survey Year 2005 Tests Results(Final)
15
Survey Year 2005 Tests Results(at end of test – May 9, 2006)
14.1
11.7
9.8
5.0
6.4
3.6
10.3
0.6
0
5
10
15
Test 1 Test 2 Test 3 INET
% contacting % wanting booklet
16
Why 2006 Test?
Expand test to larger sample (88,421 units vs. 12,688 units) on a State-level basis
Use States to more fully resemble national implementation Eliminate calls to National Office
(switch to State for help) Estimate data entry burden for States
Improve how respondents access IDCF (move to permanent IDs)
17
2006 Test Test Sample
16 States 8 State partners: CA, DC, FL, IL, MD, NC, NY, VA 8 States by Reg. office: CO, ID, MS, ND, NH, OH,
PA, SD 13,297 INET units Total: 88,421 units received the electronic
options booklet Sampling was modified in the 2006 survey
year to allocate more sample to establishments predicted to have cases
Use only test 2 scenario - 4 page booklet with electronic options* and a phone number for help – all 3 mailings
18
2006 Internet Data Collection: Establishments and Days-Away-From-Work Cases
10,59523,369 29,157
51,882
29,551
50,70753,575
70,783
82,445
157,705
85,974
164,455
0
40,000
80,000
120,000
160,000
200,000
2002
2003
2004
2005
2006
(thru
5/29
)
2006
projected
Estabs Cases
19
74.8
52.4
89.8
77.783.4
28.1
80.6
36
0
10
20
30
40
50
60
70
80
90
Test INET Rest All units
Response rate IDCF response rate
Survey Year 2006 Results(through April 15, 2007)
20
44.4
51.955.655.555
69.4 7378.379.380.8
19.7
25.532.6
3637.427.3
33.140.2
43.945.7
0102030405060708090
Test INET Rest All units
1-10 emp 11-49 emp 50-249 emp250-999 emp 1000+ emp
Survey Year 2006 Results(through April 15, 2007)
21
52.4
8.27.3
3.8 3.1
77.7
5.84.2
0.9 1.2
28.1
51.9
1.1 1.3 1
3637.7
3.1 2.1 1.70
10
20
30
40
50
60
70
80
Test INET Rest All units
IDCF Mail E-mail Telephone FAX
Survey Year 2006 Results(through April 15, 2007)
22
Going back to Purpose of Test…
Reduce processing time Capture narrative text of days-
away-from-work cases Lower printing and postage costs
23
3
6 6
9
13
0
10
20
Phone Mail(Contractor)
Fax E-mail Mail (State)
2006 Test: Lowered Processing Time (IDCF benchmark)
24
6%
32% 32%36%
95% 98%
61%
0%10%20%30%40%50%60%70%80%90%
100%
Mail (State)E-mail Fax Phone Mail(contractor)
IDCF All
2006 Test: Captured More Case Narratives (Percent as of May 3)
25
2006 Test: Lowered Cost (IDCF benchmark)
Per Year - If assumed all cases submitted by IDCF, and sample of 230,000
Booklet Approx mail cost
Approx print cost
Approx total cost
12-page $382,191 $262,808 $644,999
4-page $171,327 $216,142 $387,469
Difference $210,864 $46,666 $257,530
26
Another Option: E-mail Data Collection
First begun in the 2004 survey year Microsoft Word template received from e-
mail request Filled in template transmitted to State
specific e-mail address that routes the form through BLS and to the correct State agency
27
Another Option: E-mail Data Collection
914
2,475
8,809 9,122
0
2,000
4,000
6,000
8,000
10,000
2004 2005 2006 (thru5-29)
2006(projected)
Estabs
28
Conclusions Was successful
53% used IDCF when offered, another 8.5% use E-mail
INET group getting bigger Capturing more case narratives
Explore why not getting higher response rate overall (8 pts) Will run another study of 16 States,
plus 4 more Change instruction booklet to maybe
look less like survey