Homework SummaryHomework Summary

Impressive job and nice Impressive job and nice opportunity for me to learn about opportunity for me to learn about your researchyour research– Health insurance market reformsHealth insurance market reforms– Payment policy to support best Payment policy to support best practicepractice

– Public health particularly around Public health particularly around reproductive health, obesity and reproductive health, obesity and unhealthy behaviors, workforce unhealthy behaviors, workforce training, and resource allocationtraining, and resource allocation

Public Health PolicyPublic Health Policy

Is there a compelling argument to be Is there a compelling argument to be made that a policy is needed to made that a policy is needed to protect the public’s healthprotect the public’s health

Is it clear that the relevant Is it clear that the relevant industries won’t attempt to correct industries won’t attempt to correct the problemthe problem

Might educating consumers activate Might educating consumers activate market forces to correct a public market forces to correct a public health problem health problem

Is it clear that proposed Is it clear that proposed legislation will correct the problemlegislation will correct the problem

Payment PolicyPayment Policy

Financial incentives and Financial incentives and disincentives to optimize care disincentives to optimize care challenging without evidence base of challenging without evidence base of best practice for specific casesbest practice for specific cases

Internal disagreements within Internal disagreements within profession hard to resolve with profession hard to resolve with legislationlegislation

In general profession doesn’t like In general profession doesn’t like to be told what it can and cannot doto be told what it can and cannot do

Dear Dr. Michaels (Asst Secretary of OSHA):On behalf of the Alliance for Academic Internal Medicine (AAIM)—the consortium of five academically focused specialty organizations representing departments of internal medicine at medical schools and teaching hospitals in the United States and Canada—I write to provide the alliance’s perspective regarding the September 2, 2010, letter to your agency, filed by Public Citizen and other organizations, petitioning for federal regulation and enforcement of resident and fellow physician duty hours. AAIM respectfully urges you to deny this petition based on our 4,000 members’ collective personal and professional knowledge of the Accreditation Council for Graduate Medical Education (ACGME) as an effective oversight organization for graduate medical education (GME).

Insurance Reforms and Insurance Reforms and Other Imminent Policy Other Imminent Policy ChangesChanges Many of you identified specific Many of you identified specific policies that are or are about to policies that are or are about to be implemented that are relevant be implemented that are relevant to your research intereststo your research interests

Opportunity to study this change Opportunity to study this change as a way to inform the policy as a way to inform the policy processprocess

Evaluating Health Policy Evaluating Health Policy through Natural Experimentsthrough Natural Experiments

Andrew B. Bindman, MDAndrew B. Bindman, MD

Professor Medicine, Professor Medicine, Health Policy, Health Policy, Epidemiology & Epidemiology & BiostatisticsBiostatistics

UCSFUCSF

Asking a Policy Research Asking a Policy Research QuestionQuestion

Interesting to youInteresting to you Testable hypothesisTestable hypothesis NoveltyNovelty Value regardless of direction Value regardless of direction of resultof result

EthicalEthical Importance/number impactedImportance/number impacted TimelinessTimeliness

Learning About PoliciesLearning About Policies

Read the newspaperRead the newspaper– NY Times, Washington Post, NY Times, Washington Post, PoliticoPolitico

Health newswire servicesHealth newswire services– California Healthline California Healthline ((www.chcf.orgwww.chcf.org))– Kaiser Health News Kaiser Health News (www.kaiserhealthnews.org)(www.kaiserhealthnews.org)

Academic faculty Academic faculty Community-based organizationsCommunity-based organizations Directly from policy decision-makersDirectly from policy decision-makers

My Research InterestMy Research Interest

Health consequences of Health consequences of public policiespublic policies

Access to and quality of Access to and quality of care for low-income, care for low-income, diverse, and patient diverse, and patient populations vulnerable to populations vulnerable to poor health because of their poor health because of their social circumstancessocial circumstances

Medi-Cal: Medi-Cal: California’s Medicaid California’s Medicaid ProgramProgram 6.8 million beneficiaries6.8 million beneficiaries $47 billion last year$47 billion last year 22ndnd largest use of general fund (17%) largest use of general fund (17%) Pays for 1 in every 2 births in the Pays for 1 in every 2 births in the statestate

Approximately half of beneficiaries Approximately half of beneficiaries are Latinoare Latino

Provides 2/3rds of safety net Provides 2/3rds of safety net fundingfunding

Medicaid Population: Medicaid Population: Clinical QuestionClinical Question

Many Medicaid beneficiaries have Many Medicaid beneficiaries have interruptions in coverage interruptions in coverage (churning)(churning)

Many uninsured gain Medicaid Many uninsured gain Medicaid coverage when hospitalizedcoverage when hospitalized

Does Medicaid coverage provided at Does Medicaid coverage provided at the time of a hospitalization the time of a hospitalization adequate or do interruptions in adequate or do interruptions in Medicaid enrollment have a negative Medicaid enrollment have a negative impact on the health of impact on the health of beneficiaries?beneficiaries?

Designing a Research StudyDesigning a Research Study

Randomized trialRandomized trial– Feasibility?Feasibility?

Designing a Research StudyDesigning a Research Study

Randomized trialRandomized trial– Unethical and impracticalUnethical and impractical

Observational studyObservational study– Compare the experiences of Compare the experiences of beneficiaries who have interruptions beneficiaries who have interruptions in coverage with those who have in coverage with those who have continuous coveragecontinuous coverage

Reverse CausalityReverse Causality

Interruption in coverage might Interruption in coverage might not predict worse health not predict worse health outcome so much as worse health outcome so much as worse health might predict whether or not might predict whether or not have interrupted coveragehave interrupted coverage

Bias of higher admissions among Bias of higher admissions among those with continuous coveragethose with continuous coverage

Designing a Research StudyDesigning a Research Study

Randomized trial- is it Randomized trial- is it feasible?feasible?

Observed variation - is it Observed variation - is it biased?biased?

Natural experiment - does a Natural experiment - does a good one exist?good one exist?

Natural ExperimentsNatural Experiments

A A naturalnatural or q or quasi-experimentuasi-experiment is a is a naturally occurring instance of naturally occurring instance of observable phenomena which observable phenomena which approximate or duplicate the approximate or duplicate the properties of a controlled properties of a controlled experimentexperiment. In contrast to . In contrast to laboratorylaboratory experimentsexperiments, these events , these events aren't created by scientists, but aren't created by scientists, but yield data which nonetheless can be yield data which nonetheless can be used to make causal inferences. used to make causal inferences.

What Are the Elements of a What Are the Elements of a Good Natural Experiment in Good Natural Experiment in Health PolicyHealth Policy

Policy implementation not Policy implementation not biased by patient biased by patient characteristics such as health characteristics such as health statusstatus

Policy can be effectively tied Policy can be effectively tied to a “treatment” exposed groupto a “treatment” exposed group

Access to before/after dataAccess to before/after data

Medicaid Population: Medicaid Population: Policy QuestionPolicy Question

Federal law requires re-determination of Federal law requires re-determination of eligibility for beneficiaries at a eligibility for beneficiaries at a minimum of every 12 months but states minimum of every 12 months but states have option to do more frequentlyhave option to do more frequently

Beneficiaries who do not “re-sign up” Beneficiaries who do not “re-sign up” are dropped from programare dropped from program

Does frequency of a state’s re-Does frequency of a state’s re-enrollment process increase the number enrollment process increase the number of beneficiaries with interruptions in of beneficiaries with interruptions in coverage and if so is this in turn coverage and if so is this in turn associated with patients’ health?associated with patients’ health?

Natural Experiment of Natural Experiment of Interrupted Medicaid Interrupted Medicaid CoverageCoverage

California extended Medicaid re-California extended Medicaid re-enrollment period for enrollment period for allall children in children in California from every 3 to every 12 California from every 3 to every 12 months on January 1, 2001months on January 1, 2001

Extension of eligibility re-Extension of eligibility re-determination period should be determination period should be associated with an increase in associated with an increase in continuity of Medicaid coverage, but continuity of Medicaid coverage, but should not except through its influence should not except through its influence on continuity of coverage be associated on continuity of coverage be associated with the health status of children. with the health status of children.

Challenging Issues in Challenging Issues in Studying Natural Studying Natural ExperimentsExperiments Learning about a policy Learning about a policy change as it is about to change as it is about to happen or after the fact happen or after the fact makes it harder to collect makes it harder to collect baseline databaseline data

Primary Data CollectionPrimary Data Collection

Can be challenging to organize in Can be challenging to organize in time to assess pre-policy time to assess pre-policy condition condition

Lots of work but lots of control Lots of work but lots of control over data collection (eg surveys, over data collection (eg surveys, physiological measures, etc)physiological measures, etc)– Time consumingTime consuming– ExpensiveExpensive

Difficult to maintain over timeDifficult to maintain over time

Secondary DatabasesSecondary Databases

Pre-existing data that are often Pre-existing data that are often collected for an alternative collected for an alternative purposepurpose

Individual level or sometimes Individual level or sometimes aggregate dataaggregate data

Examples:Examples:– National surveys National surveys – RegistriesRegistries– Study cohortsStudy cohorts– Administrative dataAdministrative data

Secondary Data:Secondary Data:Advantages/ChallengesAdvantages/Challenges

Efficient - cheap, fast and often very Efficient - cheap, fast and often very largelarge

Little control on what was collectedLittle control on what was collected Collection is often Collection is often longitudinal/repeated cross-sectional longitudinal/repeated cross-sectional

Potential to analyze temporal changesPotential to analyze temporal changes If source is a payer or a provider may If source is a payer or a provider may have incomplete capturehave incomplete capture

Can be scooped by others with access Can be scooped by others with access to same datato same data

Finding Secondary DataFinding Secondary Data

www.ctsi.ucsf.edu/research/celdwww.ctsi.ucsf.edu/research/celdacac

www.www.phpartnersphpartners..org/health_statsorg/health_stats.html.html

Medicaid Data for Studying Medicaid Data for Studying Interruptions in CoverageInterruptions in Coverage

Comprehensive and detailed Comprehensive and detailed regarding eligibilityregarding eligibility

Fee for service claims complete Fee for service claims complete Missing claims information for Missing claims information for beneficiaries in managed carebeneficiaries in managed care

Won’t reflect experience of Won’t reflect experience of beneficiaries when they aren’t beneficiaries when they aren’t coveredcovered

Statewide Hospital Patient Statewide Hospital Patient Discharge AbstractsDischarge Abstracts

Comprehensive capture of all Comprehensive capture of all hospitalizations in state hospitalizations in state regardless of payerregardless of payer

Includes information on hospital Includes information on hospital admission diagnosesadmission diagnoses

Provides payer source at time of Provides payer source at time of hospitalizationhospitalization

Ambulatory Care Sensitive Conditions:Ambulatory Care Sensitive Conditions:AHRQ Prevention Quality IndicatorsAHRQ Prevention Quality Indicators

1.1. Condition with acute course and window for Condition with acute course and window for interventionintervention

2.2. Condition with chronic course amenable to self-Condition with chronic course amenable to self-managementmanagement

ACS Conditions

Acute Conditions:Acute Conditions:– DehydrationDehydration– Ruptured Appendicitis Ruptured Appendicitis – CellulitisCellulitis– Bacterial PneumoniaBacterial Pneumonia– Urinary Tract Urinary Tract InfectionInfection

Chronic Conditions:Chronic Conditions:– AsthmaAsthma– Hypertension Hypertension – COPDCOPD– Diabetes MellitusDiabetes Mellitus– Heart FailureHeart Failure– AnginaAngina

Statewide Hospital Patient Statewide Hospital Patient Discharge AbstractsDischarge Abstracts

Provides payer source at time of Provides payer source at time of hospitalization but not over timehospitalization but not over time

Critical question for hospitalizations Critical question for hospitalizations for ambulatory care sensitive for ambulatory care sensitive admissions is what the insurance admissions is what the insurance status was prior to the admission status was prior to the admission since many uninsured gain coverage in since many uninsured gain coverage in association with the hospitalizationassociation with the hospitalization

Linked CA Hospital Linked CA Hospital Discharge and Medicaid Discharge and Medicaid Eligibility FilesEligibility Files

OSHPD: Hospital Discharge Data

1998 2003

DHS: Medi-Cal Enrollment Database

1998 2003• Demographics• Monthly enrollment history• Aid Category (e.g. TANF or SSI)• FFS, managed care • Other insurance

• Diagnosis (ICD-9 Code)• Month/Year of admission

Linkage

Pre/Post Study of Re-Pre/Post Study of Re-Enrollment Policy Change for Enrollment Policy Change for ChildrenChildren

Children 1-17 years with a Children 1-17 years with a minimum of 1 month of Medicaid minimum of 1 month of Medicaid coverage in California coverage in California

Outcome = time to a hospital Outcome = time to a hospital admission for an ambulatory care admission for an ambulatory care sensitive condition sensitive condition

Main predictor = time periodMain predictor = time period– Pre policy change = Jan ‘99-Pre policy change = Jan ‘99-December ‘00December ‘00

– Post policy change = Jan ‘01-Post policy change = Jan ‘01-December ‘02December ‘02

Children 1-17 Years in California Medicaid Children 1-17 Years in California Medicaid Before and After Policy to Change EnrollmentBefore and After Policy to Change Enrollment

1999-2000 1999-2000 2001-20022001-2002

NN 3,288,171 3,288,171

3,230,120 3,230,120

Mean Age (yrs)Mean Age (yrs) 99 99

% Female% Female 5050 5151

Ethnicity (%)Ethnicity (%)

HispanicHispanic 5454 5656

Black Black 1313 1212

AsianAsian 88 88

OtherOther 2525 2424

Aid Group (%)Aid Group (%)

TANFTANF 4747 5050

SSISSI 33 33

OtherOther 5050 4747

Managed Care (%)Managed Care (%) 4747 4141

Children with Continuous Children with Continuous Medicaid Enrollment by Time Medicaid Enrollment by Time PeriodPeriod

0%

10%

20%

30%

40%

50%

60%

70%

Pre: 1999-2000 Post: 2001-2002

49

62

Years of Enrollment

Percenta

ge

Probability of a Probability of a Hospitalization for an ACS Hospitalization for an ACS Condition Over TimeCondition Over Time

0.000.050.10

0.150.200.250.30

0.350.40

0 3 6 9 12 15 18 21 24

Before 2001 EnrollmentExtension After 2001 EnrollmentExtension

Children: Adjusted Risk of ACS Children: Adjusted Risk of ACS HospitalizationHospitalization

Relative HazardRelative Hazard P-ValueP-Value

Post policy Post policy 0.740.74

<.0001<.0001

AgeAge 0.880.88

<.0001<.0001

FemaleFemale 0.970.97

0.01750.0175

EthnicityEthnicity

HispanicHispanic

3.263.26

<.0001<.0001

Black Black

4.704.70

<.0001<.0001

AsianAsian

1.101.10

0.09260.0926

OtherOther

2.972.97

<.0001<.0001

Aid GroupAid Group

TANFTANF

1.471.47

<.0001<.0001

SSISSI

24.9024.90

<.0001<.0001

Managed Managed CareCare

0.820.82

<.0001<.0001

Hospital SavingsHospital Savings

Reduction in number of Reduction in number of hospital days hospital days

15,635 15,635

Average daily Medicaid payment Average daily Medicaid payment for a hospitalization for an for a hospitalization for an ACS conditionACS condition

$1,129 $1,129

Total hospital savingsTotal hospital savings $17,651,915 $17,651,915

Medicaid Coverage CostsMedicaid Coverage Costs

Increased number of Increased number of continuously enrolled months continuously enrolled months of Medicaid coverageof Medicaid coverage

TANF/TANF/OtherOther

SSISSI

1,395,2831,395,283

41,52441,524

Average monthly capitation Average monthly capitation raterate

TANF/TANF/Other Other

SSISSI

$96$96

$370$370

Total Medicaid Costs for Total Medicaid Costs for increased coverageincreased coverage

TANF/TANF/Other Other

SSISSI

$134,393,685$134,393,685

$15,350,490$15,350,490

Total Medicaid Coverage CostsTotal Medicaid Coverage Costs $149,744,175$149,744,175

Hospital Savings and Medicaid Coverage Costs Hospital Savings and Medicaid Coverage Costs Associated with Enrollment Extension in 2001 – Associated with Enrollment Extension in 2001 –

20022002

Quasi- (natural) Quasi- (natural) ExperimentsExperiments

"Estimating the internal validity of a "Estimating the internal validity of a relationship is a deductive process in relationship is a deductive process in which the investigator has to which the investigator has to systematically think through how each of systematically think through how each of the internal validity threats may have the internal validity threats may have influenced the data. Then the investigator influenced the data. Then the investigator has to examine the data to test which has to examine the data to test which relevant threats can be ruled out. . . . relevant threats can be ruled out. . . . When all of the threats can plausibly be When all of the threats can plausibly be eliminated it is possible to make eliminated it is possible to make confident conclusions about whether a confident conclusions about whether a relationship is probably causal." relationship is probably causal."

Cook and CampbellCook and Campbell

LimitationsLimitations

Could secular changes other Could secular changes other than the policy change than the policy change explain the observed explain the observed differences? differences?

Strengthening the Design Strengthening the Design of a Natural Experimentof a Natural Experiment

Pre/post changes ideally with a Pre/post changes ideally with a comparison group not exposed to policy comparison group not exposed to policy

““Difference in differences”Difference in differences”

Need to establish conceptual basis for Need to establish conceptual basis for selection of specific comparison group selection of specific comparison group

Potential Comparison Potential Comparison Group: Adults in Medi-CalGroup: Adults in Medi-Cal

Medicaid eligibility re-Medicaid eligibility re-determination period did not determination period did not change during study period for change during study period for adultsadults

Therefore, would not expect a Therefore, would not expect a decrease over time in decrease over time in hospitalizations for ambulatory hospitalizations for ambulatory care sensitive conditions among care sensitive conditions among adultsadults

Comparison Group: Comparison Group: Adults in MedicaidAdults in Medicaid

Adults with Medicaid coverageAdults with Medicaid coverage– 1999-2000 = 62%1999-2000 = 62%– 2001-2002 = 60%2001-2002 = 60%

Adjusted relative hazard of a Adjusted relative hazard of a hospitalization for an ACS hospitalization for an ACS condition for adults in post vs condition for adults in post vs pre period= 1.11pre period= 1.11

Second Comparison Group: Second Comparison Group: Children with Continuous Children with Continuous CoverageCoverage

Comparison of children with Comparison of children with continuous coverage in each time continuous coverage in each time period revealed no significant period revealed no significant difference in hospitalizations for difference in hospitalizations for ambulatory care sensitive ambulatory care sensitive conditionsconditions

Suggests no difference in treatment Suggests no difference in treatment approach to ambulatory care approach to ambulatory care sensitive conditions over time sensitive conditions over time period of studyperiod of study

How Do We Know This is How Do We Know This is About Access to Ambulatory About Access to Ambulatory Care?Care?

Hospitalization rates among Hospitalization rates among children for non ambulatory children for non ambulatory care sensitive conditions care sensitive conditions (appendicitis and (appendicitis and gastrointestinal gastrointestinal obstruction) did not change obstruction) did not change over timeover time

Most with Interruption in Most with Interruption in Medicaid Coverage Do Not Medicaid Coverage Do Not Have Alternative for Have Alternative for Ambulatory CareAmbulatory Care At the time of At the time of hospitalizationhospitalization–59% regain Medi-Cal with 59% regain Medi-Cal with admissionadmission

–7% remain uninsured 7% remain uninsured –33% had another form of 33% had another form of insuranceinsurance

Policy ImplicationsPolicy Implications

States need to become more aware States need to become more aware of the hidden costs in their of the hidden costs in their Medicaid policiesMedicaid policies

Continuity of Medicaid coverage Continuity of Medicaid coverage can support better health and can support better health and decrease wasteful spending on decrease wasteful spending on hospitalizations that could have hospitalizations that could have been avoided with less costly been avoided with less costly outpatient careoutpatient care

Translating Research into Translating Research into PolicyPolicy

Results of study used Results of study used – in testimony to California in testimony to California legislature to prevent more frequent legislature to prevent more frequent eligibility re-determination as part eligibility re-determination as part of budget cut processof budget cut process

– in Congress to support Maintenance of in Congress to support Maintenance of Effort requirements as a part of CHIP Effort requirements as a part of CHIP reauthorizationreauthorization

Published in scientific journals for Published in scientific journals for other states to consider in their other states to consider in their policy makingpolicy making

HomeworkHomework

Identify a plausible natural Identify a plausible natural experiment to evaluate a experiment to evaluate a policy relevant to your area policy relevant to your area of researchof research

Describe data you could use Describe data you could use to study it and possible to study it and possible comparison group(s)comparison group(s)

Examples of Challenging Examples of Challenging Natural Experiments Natural Experiments

Evaluation of voluntary managed Evaluation of voluntary managed carecare–voluntary implementation can voluntary implementation can have health selection biashave health selection bias

Evaluation of the benefits of Evaluation of the benefits of insurance coverageinsurance coverage–uptake by uninsured and “crowd uptake by uninsured and “crowd out” of privately insured can out” of privately insured can make it hard to isolate who make it hard to isolate who got “treatment” of insurancegot “treatment” of insurance

Are There Opportunities Are There Opportunities for Randomized Evaluations for Randomized Evaluations of Health Policies?of Health Policies?

Randomized designs are least Randomized designs are least susceptible to biassusceptible to bias

Political considerations often Political considerations often make this approach impractical make this approach impractical in health policy interventionsin health policy interventions

May be opportunities to use a May be opportunities to use a lottery in implementing lottery in implementing policies that have more demand policies that have more demand than supply (a wait list)than supply (a wait list)