1

Update on The Survey of Income and Program Participation

Presentation to the Association of Public Data Users Annual Conference

September 25, 2009

David JohnsonUS Census Bureau

2

The Evolution of a Phoenix

3

Congresswomen Maloney hails 25th Anniversary of SIPP, Census Bureau’s

Survey of Income and Participation "October, 2008 marks 25 years of SIPP data collection. The vital

data collected by career professionals at the Census Bureau

allows for the evaluation of the efficiency and effectiveness

of government programs and gives us a more robust picture

of how well we are doing as a nation in helping families

progress through tough economic challenges"

“The SIPP allows Congress to allocate scarce government

resources and save tax dollars. It’s fitting that during this

national economic crisis we draw attention to this important

diagnostic tool which helps us understand how we can best

provide assistance to families in need.”

4

The Unique Value of SIPP

• To provide a nationally representative sample for evaluating:– annual and sub-annual dynamics of income

– movements into and out of government transfer programs

– family and social context of individuals and households

– interactions between these items

5

SIPP Basics• National panel survey – Since 1984 with sample

size between about 11,000 to 45,000 interviewed households

• The duration of each panel from 2½ yrs to 4 yrs• The SIPP sample is a multistage-stratified sample

of the U.S. civilian noninstitutionalized population• The survey uses a 4-month recall period – 3

interviews / year• The sample is divided into 4 rotation groups for

monthly interviewing• Interviews are conducted by personal visit and by

decentralized telephone

6

Selected Results from 2004-2007 SIPP

• About 28 percent of households in the bottom quintile in 2004 moved up to a higher quintile in 2007, and 27 percent of households in the top quintile in 2004 moved to a lower quintile in 2007.

• About 31 percent of the population had at least one spell of poverty lasting 2 or more months from 2004 to 2007.

• 1.8 percent of the population living in poverty all 48 months of the period from 2004-2007.

• About 28 percent of non-elderly citizens were uninsured for at least one month between 2004-2007

7

National Academy of Sciences National Research Council

Committee on National Statistics

Panel ReportReengineering the Survey of Income and

Program Participation

Constance F. Citro and John Karl Scholz, Editors

July 2009

8

CNSTAT Report – Importance of SIPP

Conclusion 2-1:The Survey of Income and Program Participation is a

unique source of information for a representative sample of household members on the intrayear dynamics of income, employment, and program eligibility and participation, together with related demographic and socioeconomic characteristics. This information remains as vital today for evaluating and improving government programs addressed to social and economic needs of the U.S. population as it did when the survey began 25 years ago.

9

Celebrating 25 years of SIPPMonthly participation rates, May-Dec 2008

0

5

10

15

20

25

30

May June July August September October November

One or more means-tested noncash benefitsFood stampsMedicaid

10

Celebrating 25 years of SIPPResults of Tax rebate question, 2008

0%

10%

20%

30%

40%

50%

60%

70%

under 25 25 - 44 45 - 64 65 and over Total

Mostly spend Mostly save Mostly pay off debt

11

Celebrating 25 years of SIPP Monthly Uninsured rates and Monthly

Poverty rates, May – Nov 2009

10%

11%

12%

13%

14%

15%

16%

17%

18%

May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09

Percent Uninsured Poverty Rate

12

Our work on SIPP Improvements

• Improve Processing System and Collection Instrument

• Develop Event History Calendar (EHC) Instrument

• Examine use of administrative records data to supplement and evaluate survey data

• Continue meetings with stakeholders, development of survey content, and use of reimbursable supplements

13

Timeline for SIPP Development

2008 paper EHC

2007 2008 2009 2010 2011 2012 Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan 2013

2009 SIPP Re-EngineeringInstrument Dev.

Processing andEvaluation

SIPP 2008 Panel – Waves 1 – 10 collection

Eval. Analysis

Systems Tests -Preparation

2009 Re-engineered SIPP automated

Prototype Reference Period

Field A

ctivities

2012/13 SIPP Re-EngineeringInstrument Refinement

Systems Tests -Preparation

2013 Reengineered SIPP Reference Period

Field A

ctivities

SIPP 2008 Panel – Waves 1 – 13 data release

Data GapSIPP 2004

2nd automated prototype

Reference Period

Field A

ct.

SIPP 2004 Panel data release

Waves 11 – 13

14

Re-engineered Survey Format/Content

• Survey Instrument –– Annual administration (Jan/Feb)

– BLAISE (plus EHC)

– Follow movers

• Content– Responding users indicated a broad

need for most of SIPP core content.

• Calendar –– Improvement on other designs by

integrating more closely with the

instrument programming language.

Front

Coverage

Demographics

Calendar Universe Screeners

Residence History

Marriage/Cohab/Fertility

Labor Force (emp/unemp/leave)

Soc. Welfare and Other Progs.

Health Insurance Coverage

Commuting, Work Schedule, Child Care

Assets / Balances

Annual Programs – Taxes – Lump Sum – Social

Insurance – Child Support

Adult Well-Being, Child Well-Being,Food Sufficiency

Health (Insurance, Expenditures, Disability

Wrap-up and MissingFollow-up visit/call Info

Back

Wkrs Cmp/Disab/Unemp Cmp

Landmark EventsEHC

CALENDAR

Type 2 Roster/Info

Relationship

Enrollment

15

Statements from the CNSTAT Report:

On EHC methodology• As discussed in Belli (1998), in an event history

calendar, “respondents are encouraged to consider various events that constitute their personal pasts as contained within broader thematic streams of events. Not only can respondents not the interrelationship of events within the same themes (top-down and sequential retrieval) but, depending on which themes are represented by the calendar, respondents can also not the interrelationships among events that exist with different themes (parallel retrieval).”

16

Statements from the CNSTAT Report:

The Advantages of the EHC• Another potential advantage of the event history

calendar approach, if it proves capable of generating high-quality monthly data, is that the first year of income data could be collected with no added sample attrition beyond the loss of households that refuse to participate in the survey at all. Under the current design, annual income must be aggregated across four waves in order to have a common 12-month reference period for the four rotation groups

17

CNSTAT Report on Administrative Records

• Conclusion 3-4: Indirect uses of administrative records are those uses, such as evaluation of data quality and improvement of imputation models for missing data, in which the administrative data are never recorded on survey records. They are advantageous for a reengineered SIPP in that they should have little or no adverse effects on timeliness or the needed level of confidentiality protection of SIPP data products.

18

Use of Administrative Records

• National-level data– SIPP linked with administrative records data at the

person level– Examining imputation methods

• Creation of SIPP Synthetic Beta File matched with SSA data

• State-level data– Many social programs administered at the state-

level– SIPP data linked to social program data from states,

e.g., Maryland, Illinois, Texas, New York, Wisconsin

19

CNSTAT Report on Administrative Records

• Recommendation 3-2: The Census Bureau, in close consultation with data users, should develop a strategy for acquiring selected state administrative records, recognizing that it will be costly and probably unfeasible to acquire all relevant records from all or even most states. The bureau’s acquisition strategy should be guided by such criteria as the importance of the income source for lower income households, particularly in times of economic distress, and the relative ease of acquiring the records.

20

FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports

0.0%

2.5%

5.0%

7.5%

10.0%

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

% Y

es

SIPP % yes

EHC % yes

21

FOOD STAMPS (Texas Only) -- % Participation for Each Month of CY2007 According to the SIPP and EHC Reports and Administrative Records data

0.0%

2.5%

5.0%

7.5%

10.0%

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

% Y

es

SIPP % yes

EHC % yes

Admin % yes

22

Statements from Recent CNSTAT Report

On the EHC field test• The development and implementation of the

prototype experiment is a valuable next step in developing the information base needed for the reengineered SIPP.

• If funding during FY2010 and FY2011 is available, this prototype would examine issues that arise with locating movers when interviews are 1 year rather than 4 months apart, as well as the consistency of data reports between interviews that are 1 year apart.

23

Design of the 2010 Test• 8,000 Sample Addresses

– Target 6,500 Complete HH Interviews

• “High Poverty” Sample Stratum

• State-Based Design– With possible access to admin records– TX, IL, MD, NY, WI, CA

• Field Period: Early Jan - mid March 2010– 150-200 Field Reps– FR Training in Dec/Jan

24

New Electronic EHC for 2010 Field Test

25

Statements from Recent CNSTAT Report:

On Outreach• A key element of reengineering SIPP and keeping it

relevant to user needs concerns the survey content. In this regard, we commend the Census Bureau’s efforts to reach out to the user community by asking users to comment on “content matrices” to help identify which portions of the survey are critical to users and to provide input on aspects of SIPP that could be improved. The degree of recent interaction between the Census Bureau and the SIPP user community is exemplary.

26

URL: http://www.sipp.census.gov/sippSource: U.S. Census Bureau, Demographics Survey Division,

Survey of Income and Program Participation branch Census Bureau Links:   Home · Search · Subjects A-Z · FAQs · Data Tools · Catalog · Census 2000 · Quality · Privacy Policy · Contact Us

                                                                                      

Updating SIPP User Guide