2015 american community survey mail contact strategy … · 2019. 2. 16. · 2015 american...

This poster is released to inform interested parties of ongoing research and to encourage discussion. Any views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

U.S. Department of CommerceEconomics and Statistics Administration

U.S. Census Bureau census.gov

2015 American Community Survey Mail Contact Strategy Modification TestSandra Clark, Lauren DiFiglia, Jennifer Tancreto, and David Raglin U.S. Census Bureau

Introduction Results

Experimental Design

The Census Bureau conducted the 2015 American Community Survey (ACS) Mail Contact Strategy Test to evaluate several modifications to mail materials and contact strategies in an effort to improve self-response to the Internet and mail modes. In this experiment, we tested the following:

• Eliminating the prenotice letter and sending the initial package four days earlier.

• Strengthening reminder contacts by: making the URL to the survey more prominent; highlighting the User ID; using direct wording; and stating the mandatory nature of the survey.

• Testing the impact of sending the additional reminder postcard to all nonrespondents (including CATI eligible).

The goal of the research was to study the impact of the changes on self-response behavior by comparing return rates between treatments and to conduct a cost analysis to estimate savings.

Calculated total self-response, Internet, and Mail return rates using following formula:

Cost model used estimated workloads and average mailing and operation costs to predict cost differences.

• Eliminating the prenotice:

- No effect on total self-response

- Lower Internet return rates and higher mail return rates

- The prenotice is an additional mailing that prompts earlier Internet response

• Replacing postcard with modified letter for 1st reminder:

- Increased total self-response

- Higher Internet response lower mail response

- Encouraged earlier response to Internet – appears to pick up some would-be mail responders in Internet and additional self-responders

• Eliminating prenotice AND using modified letter reminder:

- Increased total self-response

- Higher Internet response, with no effect on mail response

- Results in cost savings

- Implemented in August 2015 ACS panel

• Sending additional reminder postcard to CATI eligible addresses:

- Increased self-response return rates

- No impact on CATI response

- Further research needed to determine cost effectiveness

Research Methodology

Conclusions

Universes: RQ#1-4: Total # of mailable and deliverable sampled addresses RQ#5: Total # CATI eligible addresses

Source: 2015 Mail Contact Strategy Test, using data from April 2015 ACS panel

#1- Can we eliminate the prenotice letter?

Maybe. Total self-response statistically the same. Higher mail, but lower Internet return rates without the prenotice.

#2- Should we use a postcard or letter reminder?

Reminder Letter. Total self-response 3.8 percentage points higher. Higher Internet return rates, but lower mail return rates.

#4- How do the strategies compare throughout the data collection cycle?

#6- Should we send additional reminder postcards to CATI eligible addresses?

#5- Can we save money by eliminating the prenotice and using the reminder letter?

Maybe. Appears to improve return rates, however more research is necessary to research cost effectiveness.

Letter reminder takes off around day 10 and remains higherthroughout cycle.

For more information see report at: https://www.census.gov/acs/www/Downloads/library/2015_Clark_03.pdf

Yes, estimated savings between $5.7 million and $9.0 million!

#3- Can we increase Internet response for the No Prenotice group by using a reminder letter?

Yes. Total self-response 3.4 percentage points higher with letter.Higher Internet return rates and mail rates statistically the same.

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42

Treatments With PreNotice (C & T3):

Treatments Without PreNotice (Reminder Letter - T1 & T4; Reminder Postcard - T2 & T5):

Pre-N Initial Rem PC Paper Q Rem PC2 Addtl PC

Initial Rem Letter or Rem PC Paper Q Rem PC2 Addtl PC

Days

Sample size: Control ~226,000 addresses and each Test Treatment ~12,000 addresses

Figure 1. List of Mailings by Treatment

Figure 2. Timing and Sequence of Mailings for the Treatments with a Prenotice versus those Without

Return Rate =

# of sample addresses that provided a response *100 Total # of eligible sample addresses

0

10

20

30

40

50

60

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100

103

106

109

112

Figure 5. Total Self-Response, Internet, and Mail Return Rates by Treatment from Panel Start to Finish

Initial Package -PrenoticeGroup

1st Reminder

Paper Questionnarie

2nd Reminder

Additional Postcard

CATI Starts

Total SelfResponse

Internet

Mail

CAPI Starts46.7

29.3

17.4

50.1

32.8

17.4

Total Self-Response Internet Mail

Figure 4. Return Rates by Mode Before Nonresponse Follow-up: Prenotice / Postcard vs No Prenotice / Letter

Prenotice/Postcard No Prenotice/Letter

46.7

29.3

17.4

46.3

27.7

18.6


Figure 2. Return Rates by Mode Before Nonresponse Follow-up: Prenotice vs No Prenotice

Prenotice/Postcard No Prenotice/Postcard

46.3

27.7

18.6

50.1

32.8

17.4


Figure 3. Return Rates by Mode Before Nonresponse Follow-up: No Prenotice Treatments -- Postcard Reminder vs Letter

Reminder

No Prenotice/Postcard No Prenotice/Letter

5.7

11.3

5.9

11.5

4.4

11.2

3.4

9.0

2.9

9.6

3.5

8.5

Internet Mail Internet Mail Internet Mail

Figure 7. Return Rates by Contact Strategy by Mode after CATI Follow-UP (universe: CATI sample addresses)

Additional Postcard Reminder VS No Additional Postcard Reminder

Add. PCNo Add. PC

Add. PCNo Add. PC

Add. PCNo Add. PC

No Prenotice/LetterPrenotice/Post Card No Prenotice/Postcard

Figure 6. Simulated Estimate of Annual Cost Savings-Difference (Prenotice/Postcard-No Prenotice/Letter)

Lower Bound Upper BoundMail savings $1,381,000 $1,357,000CATI/CAPI savings $4,282,000 $7,713,000Total savings $5,663,000 $9,070,000Source: DiFiglia (2015)

Treatment PrenoticeInitial Package Early

Initial Package Normal Time

1st Reminder Letter

1st Reminder Postcard

Additional Addresses Get 3rd

Postcard

Control X X XT1 X XT2 X XT3 X X X XT4 X X XT5 X X X

ACS-20(LX)CST(1-2015)

OFFICE OF THE DIRECTOR

Washington, DC 20233-0001

UNITED STATES DEPARTMENT OF COMMERCE

U.S. Census BureauEconomics and Statistics Administration

census.gov

A message from the Director, U.S. Census Bureau ...

A few days ago, you should have received instructions for completing the American Community Survey online. Local communities depend on information from this survey todecide where schools, highways, hospitals, and other important services are needed. If youhave not already responded, please do so now.

If we do not receive your response online, we will mail a paper questionnaire to your address.

Your response to this survey is required by law. Your response is critically importantto your local community and your country. Responding promptly will prevent your receivingadditional reminder mailings, phone calls, or personal visits from Census Bureau interviewers.

If you need help completing the survey or have questions, please call 1–800–354–7271.

Thank you in advance for your prompt response.

Sincerely,

John H. ThompsonDirector, U.S. Census Bureau

Respond now at https://respond.census.gov/acsLog in using this user ID:

ACS-12(L)S (2013)(6-2012) Washington, DC 20233-0001


U.S. Census BureauEconomics and Statistics Administration

census.gov


In a few days your household will receive instructions in the mail on how to complete a veryimportant national survey, the American Community Survey. Please follow the instructionsto complete the survey promptly. The U.S. Census Bureau is conducting this survey andchose your address, not you personally, as part of a randomly selected sample.

The American Community Survey collects information about various topics like education,housing, and jobs. Information from this survey is used by federal, state, local, and tribalgovernments to meet the needs of communities across America. For example, communityleaders use this information to decide where schools, highways, hospitals, and otherservices are needed. The survey also is used to develop programs to reduce trafficcongestion, provide job training, and plan for the health care needs of the elderly.

If you have access to the Internet and want to learn more about the American CommunitySurvey, please visit the Census Bureau’s Web site: www.census.gov/acs/www.

Thank you in advance for your help.

Enclosure

OFFICE OF THE DIRECTOR


U.S. Census BureauWashington, DC 20233-0001OFFICE OF THE DIRECTOR

Economics and Statistics AdministrationACS-20S(2013) (5-2012)




Economics and Statistics Administration



















DC DC

DC DC

DC DC

A few days ago, you should have received instructions for completing theAmerican Community Survey online. If you have already responded, thankyou. If you have not, please do so as soon as possible athttps://respond.census.gov/acs . If we do not receive your response, wewill mail you a paper questionnaire in a few weeks.

Local and national leaders use the information from this survey for planningschools, hospitals, roads, and other community needs.

If you need help completing the survey or have questions, please call ourtoll-free number (1–800–354–7271).

Thank you.

ACS-20S(2013) (5-2012)





Thank you.





Thank you.





Thank you.





Thank you.





Thank you.





Thank you.





Thank you.

ACS-20S(2013) (5-2012) ACS-20S(2013) (5-2012)

ACS-20S(2013) (5-2012) ACS-20S(2013) (5-2012)

ACS-20S(2013) (5-2012) ACS-20S(2013) (5-2012)

Prenotice Letter

Postcard Reminder

Letter Reminder

AAPOR 71 stAnnual ConferenceMay 12-15 2016Austin, Texas

2015 american community survey mail contact strategy … · 2019. 2. 16. · 2015 american...

Documents