foresight, insight. hindsight us statistical observations fritz scheuren norc university of chicago
TRANSCRIPT
Foresight, Insight. HindsightUS Statistical Observations
Fritz Scheuren
NORC University of Chicago
Reminder on Definitions
• Hindsight reflecting on the past –Personally/Collectively
• Insight, where it all comes together, like this Conference
• Foresight future seeing or shaping, also familiar but bears discussion
A Small Statistical Corner
• US Official Statistics• Censuses and Surveys• Administrative Records• Focus on lived experiences,
ala Deming
Times They are a Changing
• Relevance of our Discipline?
• Responsiveness of Statistics?
• Information Age?
• Misinformation Age?
• Service Partnerships?
Children’s Teaching Game?
• High?
• Low?
• You’re Too Slow!
• How Avoid Being Too Slow?
But Change is Accelerating!
• Is our Discipline Keeping Up?
• Certainly Computationally!
• Tool/Theory Building too!
• Practice Slower?
• Organizational Issues?
How To Keep Up/Catch Up?
• Google (of course)
• Metadata Revolution -- Still more Promise than Practice
• Meta-Analytic Reuse -- Still Often Too Hard
High-Clockspeed Trend(use of cell phones, portable devices)
Responsiveness to Trends
• How Well Do We Play?
• High?
• Low?
• You’re Too Slow!
Response Times to Trends(organizational clockspeed = rate at which an organization introduces new products, adopts
new production processes, or reorganizes itself; Sources: Charles H. Fine, 1998; David W. Rejeski, 2003)
Technological Mega-Trends
• Faster and Faster Computing(Slower for Official Statistics)
• Descriptive to Analytic• Randomization-based to Model-
based • Producer Dominated to Customer
Shared
Typical Grief Response to Change Still Often True
• Denial
• Anger
• Bargaining
• Depression
• Acceptance
Examples of Hindsight, Insight, Foresight
• Nonresponse Circa 1980
• US Census Taking Circa 1990
• Paradata Circa 2000
• Visualization Circa 2010
• Next Steps Together?
Nonresponse
Hindsight
Example
40 Year CPS Income Trend • Insignificant Missingness in 1962 • Now Nearly Half of Interviews
have Some Missingness• About a Third of the Amount is
Imputed• But Still using the Same Basic
“Hot Deck” Methods Today
Greater Bias Concerns
• Possibility of Greater Nonresponse Bias
• Potentially More Income Understatement
• Also Characteristics of Poor Blurred
Variance More Understated
• Growing Variance Not Directly Reduced
• Rubin’s Multiple Imputation Solution Still Not Used in CPS
• Remains Descriptive Rather than Analytic
Paradata Modeling
Insight
Example
Meta-Data Revolution
• Applying Computing to Documentation and Training
• Including Measurement Process or Paradata
• Achieving Full Systems Thinking
Unify Meta/Paradata
• Bringing All Survey Aspects together electronically
• Sharing with All Stakeholders
• Breaking Down Barriers between Departments
Unify Meta/Paradata
• Bringing All Survey Meta-Data together electronically
• Sharing with All Stakeholders
• Breaking Down All Barriers Between Departments
Manage System as a Whole
• Not Just Conformance to Requirements Quality
• But Total Fitness for Use Quality• From Sampling/Nonsampling to
Total Survey Inference• Record linkage Example
Total Systems Thinking
• Turning Sample “Models”
• Into Full Survey “Models”
• Using Paradata and Experience
• Politz-Simmons Example
Visualization
Foresight
Example
Complex Survey Graphics
• Clustering and Weighting Distort
• Analytically these can be “solved” Approximately
• Design Effect Example
Restoring Visual Metaphor
• Inverse Sampling Algorithm
• Works for Many Designs
• Satisfactory Analytically
• Works Graphically too but not yet Always
Simulation Alternatives
• Capture Essential (Sufficient?) Conditions
• Simulate Graphics Analytically
• Retain Real Sample?• Apply Empirical Residuals?
Regression Diagnostics
• Design-Based and Analytic Alternatives Being Examined Now
• Both Have Merits• May Imbed both in an Open-
Source, like R
Next Steps?
Further Considerations and Examples
Expected White Swan?
Unexpected Black Swan
Addressing Evolving Trends