Download - Fundamental issues
Fundamental issues
Alyna T. Chien MD, MSThe University of Chicago
Harvard Quality ColloquiumAugust 20, 2008
Goal
• Illustrate how the first pediatric public reporting effort is facing inherent challenges to pediatric quality measurement
CHARTCalifornia Hospital Assessment and Reporting
Taskforce
Inherent challenges
• Perspective• What to measure• Evidence base• Sample sizes
Inherent challenges
• Perspective– Consumers– Providers
• Free standing children’s hospitals• Community hospitals
– Purchasers
Inherent challenges
• What to measure– Structure, Process, Outcome– Safety, Effectiveness, Efficiency, Equity, Patient-centeredness,
Patient Safety, Timeliness
Inherent challenges
• Evidence base– Proper endpoints?
Inherent challenges
• Sample size– 1/5th of the adult population– Lower disease prevalence– Significant proportion of care provided in adult contexts– Significant proportion of care provided in community contexts
300M adults~50M children (~20% of population)
Max ‘volume’~500,000 newborns in California
271 of 442 hospitals with pediatric services~1800 newborns per hospital
~180 non-newborn admissions per hospital
Strategy
• Perspective Diverse Workgroup• What to measure Diverse portfolio• Patchier evidence base Diverse portfolio• Patchier evidence base Invest in development• Small sample size “Functional” measures?• Small sample size “Structural” measures?• Small sample size Aggregation methods?• Small sample size Invest in development
Tool
Does it apply to children?prevalence
costIs it evidence based?
What dimensions does it measure?Portfolio diversification
Strategy
• Systematic approach
• Portfolio diversification
PMD Tool Quickie
• Does it apply to children? – prevalence
• Which ones?• Impact?
– mortality– disease burden
– cost• Evidence base?• Dimension of quality?
• Portfolio diversification
ALL AGES
Annual discharges U.S. (n)
Annual expenditures (millions)
3,968,346 Liveborn Liveborn $ 10,461
147,782 Pneumonia Appendicitis $ 1,101
139,528 Acute bronchitis Pneumonia $ 1,056
138,574 Asthma Mood disorders $ 974
99,308 Fluid and Electrolyte disorders Fracture of the lower limb $ 965
95,201 Mood disorders Acute bronchitis $ 893
73,455 Appendicitis Asthma $ 796
47,789 Epilepsy, convulsions Epilepsy, convulsions $ 463
38,004 Urinary tract infections Fluid and electrolyte disorders $ 355
34,550 Intestinal infections Maintenance chemotherapy,
radiotherapy $ 306
30,676 Viral infections Urinary tract infections $ 261
29,692 Trauma to the perineum and vulva Viral infections $ 157
26,860Hemolytic jaundice and perinatal jaundice
Trauma to the perineum and vulva $ 154
19,436Normal delivery without complications Intestinal infections $ 151
18,217 Noninfectious gastroenteritis Substance-related and alcohol
disorders $ 148
16,782 Fracture of the upper limb Fracture of the upper limb $ 145
13,884 Fracture of the lower limb ADHD and disruptive behavior
disorders $ 103
13,726Maintenance chemotherapy, radiotherapy Early or threatened labor $ 102
12,897 Early or threatened labor Intracranial injury $ 101
9,522 Diabetes mellitus with complications Hemolytic jaundice and
perinatal jaundice $ 98
8,249Substance-related and alcohol disorders
Normal delivery without complications $ 95
7,255 Poisoning by other medications Complication of device, implant $ 76
6,679ADHD and disruptive behavior disorders
Diabetes mellitus with complications $ 74
6,217Fetal distress and abnormal forces of labor Noninfectious gastroenteritis $ 60
6,152 Abnormal fetal heart rate Fetal distress and abnormal
forces of labor $ 53
5,202 Umbilical cord complication Poisoning by other medications $ 52
4,735 Abdominal pain Abnormal fetal heart rate $ 46
4,384
Hypertension complicating pregnancy, childbirth, and the puerperium
Hypertension complicating pregnancy, childbirth, and the
puerperium $ 38
3,478 Skin infections Excess amniotic fluid and other
problems of amniotic cavity $ 28
3,328 Intracranial injury Umbilical cord complication $ 28
3,325Excess amniotic fluid and other problems of amniotic cavity Abdominal pain $ 25
2,471 Complication of device, implant Skin infections $ 21
2,442 Fever of unknown origin Fever of unknown origin $ 13
811 Sickle cell anemia Sickle cell anemia $ 9
536 Skin disorders Skin disorders $ 3
5,039,493 19,411
CHART Starter SetApplies to children
Population?
Impact
Evidence
Quality dimension
Exclusive breastfeeding rate
VolumeNormal newborns
Sort of Process/OutcomeEffectiveness
NICU nosocomial infection rate
Premies/at-risk newborns
Modifiable by provider
Process/OutcomeSafety
Antibiotics for appendicitis
VolumeAll ages
Not really Process
CHART Future Directions
• Perspective• What to measure• Evidence base• Sample size
Summary
• Pediatric public reporting in its infancy
• Lots of room for improvement
ThanksCHART Core• R. Adams Dudley, MD MBA• Mitzi Dean, PhD• Ted Karrison, MS • James Anderson, PhD
Peds Workgroup• Diana Dooley (CCHA)• Erin Givens (CSCC)• Jeff Gould (Stanford)• Greg Janos (Sutter)• Tom Klitzner (UCLA)• Paul Kurtin (UC San Diego)• Paul Sharef (UCLA)
Extra slides
Public reporting: adult history
• Florence Nightengale• Ernest Codman
• Mortality after Coronary Artery Bypass Grafts
• Medicare
Public reporting: mechanisms?
• Free market mechanism:
Comparative info about healthcare quality
payors (employers, health plans, and patients)
choose higher quality providers
financial rewards will flow to better performers
(and away from poorer ones)
• “Self-improvement” mechanism:
Comparative info about healthcare quality
providers (hospitals, medical groups, individual physicians)
better awareness of quality issues
quality regulation cost containment quality improvement
Do “consumers” act on the information?
• Free market mechanism: Comparative info about healthcare quality payors (employers, health plans, and patients) choose higher quality providers financial rewards will flow to better performers (and away from poorer ones)
• “Self-improvement” mechanism: Comparative info about healthcare quality providers (hospitals, medical groups, individual physicians) better awareness of quality issues quality regulation cost containment quality improvement
YES and NO
Marshall MN et. al, JAMA 2000
Public reporting: mechanisms for exclusive
breastfeeding• Volume• Data availability
• Low prevalence of conditions• Appropriate processes?• Appropriate outcomes?
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?
•Hospital-based supports•A “critical period”?
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?
•Hospital-based supports•A “critical period”?
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers
•Hospital-based supports•A “critical period”?
Performance – three approaches
For CHART’s exclusive breastfeeding measure:
Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)
where:• Ÿh is mean exclusive breastfeeding rate for each hospital• Ýh is mean exclusive breastfeeding rate for all newborns
(statewide average)• No other individual level adjustments• Hospitals admitting <30 newborns annually are excluded
Background
American healthcare lacks quality and equity
Public reporting is one strategy aimed at improving the quality of that care
• “Free market” mechanism– consumers/payors
• “Self-improvement” mechanisms– providers, individuals and organizations
• Evidence equivocal
Public reporting’s role in eliminating (or exacerbating) racial/ethnic disparities is unknown
“Free market” “Self-imprvmnt” Consumers / payors Providers
Consequence:• Improve Want to go somewhere Want to be viewed
as “equitable” “equitable”
• Worsen Promotes segregation
• Status quo
Unique opportunity to explore public reporting’s potential role in presenting racial/ethnic disparities
Exclusive breastfeeding rates in California hospitals:• Enough patients
– 10% of all American newborns born in California (500,000/year)
• Enough hospitals– 283 are licensed to provide pediatric services
• Good racial/ethnic data (as currently available)
– standardized collection– complete– variation expected
This goal of this paper is to explore three approaches to incorporating information on racial/ethnic disparities into hospital public reporting:1. Adjusting expected performance for
race/ethnicity
2. Stratifying performance by race/ethnicity
3. Developing a ‘disparity’ score
Hypotheses:
1. Hospital performance/rankings will change depending on how race/ethnicity is incorporated into performance methodology
2. Each methodology will have ‘pros’ and ‘cons’
Methods
Study Design
Cross-sectional
Primary independent variables: - Different performance measurement methodologies:
1. Proportional2. Stratified3. + ‘Disparity’ score
Primary dependent variables: - Changes in hospital rank
Performance – traditional methodIn general:
Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)Where:• Zh is standardized performance• Ÿh is observed mean performance including adjustments• Ýh is expected mean performance including adjustments• Each adjusted for patient characteristics
Conventionally:• Hospitals with <30 observations are excluded• Performance can be estimated using standard frequentist
approaches• Or using Bayesian ones
Performance – alternative approaches
Traditional ZhT = ŸhT – Ýall / VAR (ŸhT–Ýall)
Proportional ZhP = ŸhP – Ýall / VAR (ŸhP–Ýall)
Stratified ZhWh = ŸhWh – ÝallWh / VAR (ŸhWh–ÝhWh)ZhAA = ŸhAA – ÝallAA / VAR (ŸhAA–ÝhAA)ZhHi = ŸhHi – ÝallHi / VAR (ŸhHi–ÝhHi)ZhAs = ŸhAs – ÝallAs / VAR (ŸhAs–ÝhAs)ZhOt = ŸhOt – ÝallOt / VAR (ŸhOt–ÝhOt)
‘Disparity’ score
=0 if ZhWh = ZhAA = ZhHi = ZhAs = ZhOt
Data Source
California Department of Public Health Center for Family HealthGenetic Disease Screening ProgramNewborn Screening Data 2006• Mandated statewide screening program• Established 1966• Screening rate 99%
Data Collection
As part of the Newborn Screen, all providers are required to answer the following questions:
“All feeding since birth: (check only one box)
[]Breast only []Formula only []Breast & Formula[]TPN/Hyperal []Other. (SPECIFY):__________________________________________________”
“Race/ethnicity: (check all that apply)
[]White []Hispanic []Black []Chinese []Japanese[]Korean []Cambodian []Laotian []Vietnamese []Filipino[]Asian Indian []Middle Eastern []Native American[]Samoan []Other (SPECIFY):________________________________________________________”
Data Quality - Missing
2006
Number of newborns• 492,587 in dataset – cross-checked with Vital Statistics
Breastfeeding status• Indicated 97.2%• Missing 2.8%
Race/ethnicity noted• Indicated 97.4%• Missing 2.6%
Data Quality - Validity
2006
Breastfeeding status GDSP ?NSLAH• Exclusive ~40%• Any ~90%
Race/ethnicity noted GDSP ?Census• White• African American• Hispanic• Asian• Multi• Other
PRELIMINARY RESULTS
271 eligible hospitals174 participating in CHART
PRELIMINARY RESULTS
California 2006
Newborns Exclusive Breastfeeding Rate
n mean sd
Total 492,587 0.427 0.495
White 125,136 0.637 0.481
African America 24,018 0.330 0.470
Hispanic 261,456 0.321 0.467
Asian/PI 45,566 0.440 0.496
Other/Multi 23,353 0.500 0.500
Missing 13,058 0.526 0.499
Additional Analyses
Impact of dropping hospitals with <30 observations, particularly with the Stratified approach
Treatment of missing values - zeros - dropped
Frequentist versus Bayesian approach