fundamental issues
DESCRIPTION
Fundamental issues. Alyna T. Chien MD, MS The University of Chicago Harvard Quality Colloquium August 20, 2008. Goal. Illustrate how the first pediatric public reporting effort is facing inherent challenges to pediatric quality measurement. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/1.jpg)
Fundamental issues
Alyna T. Chien MD, MSThe University of Chicago
Harvard Quality ColloquiumAugust 20, 2008
![Page 2: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/2.jpg)
Goal
• Illustrate how the first pediatric public reporting effort is facing inherent challenges to pediatric quality measurement
![Page 3: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/3.jpg)
CHARTCalifornia Hospital Assessment and Reporting
Taskforce
![Page 4: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/4.jpg)
![Page 5: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/5.jpg)
![Page 6: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/6.jpg)
![Page 7: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/7.jpg)
![Page 8: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/8.jpg)
Inherent challenges
• Perspective• What to measure• Evidence base• Sample sizes
![Page 9: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/9.jpg)
Inherent challenges
• Perspective– Consumers– Providers
• Free standing children’s hospitals• Community hospitals
– Purchasers
![Page 10: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/10.jpg)
Inherent challenges
• What to measure– Structure, Process, Outcome– Safety, Effectiveness, Efficiency, Equity, Patient-centeredness,
Patient Safety, Timeliness
![Page 11: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/11.jpg)
Inherent challenges
• Evidence base– Proper endpoints?
![Page 12: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/12.jpg)
Inherent challenges
• Sample size– 1/5th of the adult population– Lower disease prevalence– Significant proportion of care provided in adult contexts– Significant proportion of care provided in community contexts
![Page 13: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/13.jpg)
![Page 14: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/14.jpg)
300M adults~50M children (~20% of population)
Max ‘volume’~500,000 newborns in California
271 of 442 hospitals with pediatric services~1800 newborns per hospital
~180 non-newborn admissions per hospital
![Page 15: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/15.jpg)
Strategy
• Perspective Diverse Workgroup• What to measure Diverse portfolio• Patchier evidence base Diverse portfolio• Patchier evidence base Invest in development• Small sample size “Functional” measures?• Small sample size “Structural” measures?• Small sample size Aggregation methods?• Small sample size Invest in development
![Page 16: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/16.jpg)
Tool
Does it apply to children?prevalence
costIs it evidence based?
What dimensions does it measure?Portfolio diversification
![Page 17: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/17.jpg)
Strategy
• Systematic approach
• Portfolio diversification
![Page 18: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/18.jpg)
PMD Tool Quickie
• Does it apply to children? – prevalence
• Which ones?• Impact?
– mortality– disease burden
– cost• Evidence base?• Dimension of quality?
• Portfolio diversification
![Page 19: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/19.jpg)
![Page 20: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/20.jpg)
ALL AGES
Annual discharges U.S. (n)
Annual expenditures (millions)
3,968,346 Liveborn Liveborn $ 10,461
147,782 Pneumonia Appendicitis $ 1,101
139,528 Acute bronchitis Pneumonia $ 1,056
138,574 Asthma Mood disorders $ 974
99,308 Fluid and Electrolyte disorders Fracture of the lower limb $ 965
95,201 Mood disorders Acute bronchitis $ 893
73,455 Appendicitis Asthma $ 796
47,789 Epilepsy, convulsions Epilepsy, convulsions $ 463
38,004 Urinary tract infections Fluid and electrolyte disorders $ 355
34,550 Intestinal infections Maintenance chemotherapy,
radiotherapy $ 306
30,676 Viral infections Urinary tract infections $ 261
29,692 Trauma to the perineum and vulva Viral infections $ 157
26,860Hemolytic jaundice and perinatal jaundice
Trauma to the perineum and vulva $ 154
19,436Normal delivery without complications Intestinal infections $ 151
18,217 Noninfectious gastroenteritis Substance-related and alcohol
disorders $ 148
16,782 Fracture of the upper limb Fracture of the upper limb $ 145
13,884 Fracture of the lower limb ADHD and disruptive behavior
disorders $ 103
13,726Maintenance chemotherapy, radiotherapy Early or threatened labor $ 102
12,897 Early or threatened labor Intracranial injury $ 101
9,522 Diabetes mellitus with complications Hemolytic jaundice and
perinatal jaundice $ 98
8,249Substance-related and alcohol disorders
Normal delivery without complications $ 95
7,255 Poisoning by other medications Complication of device, implant $ 76
6,679ADHD and disruptive behavior disorders
Diabetes mellitus with complications $ 74
6,217Fetal distress and abnormal forces of labor Noninfectious gastroenteritis $ 60
6,152 Abnormal fetal heart rate Fetal distress and abnormal
forces of labor $ 53
5,202 Umbilical cord complication Poisoning by other medications $ 52
4,735 Abdominal pain Abnormal fetal heart rate $ 46
4,384
Hypertension complicating pregnancy, childbirth, and the puerperium
Hypertension complicating pregnancy, childbirth, and the
puerperium $ 38
3,478 Skin infections Excess amniotic fluid and other
problems of amniotic cavity $ 28
3,328 Intracranial injury Umbilical cord complication $ 28
3,325Excess amniotic fluid and other problems of amniotic cavity Abdominal pain $ 25
2,471 Complication of device, implant Skin infections $ 21
2,442 Fever of unknown origin Fever of unknown origin $ 13
811 Sickle cell anemia Sickle cell anemia $ 9
536 Skin disorders Skin disorders $ 3
5,039,493 19,411
![Page 21: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/21.jpg)
CHART Starter SetApplies to children
Population?
Impact
Evidence
Quality dimension
Exclusive breastfeeding rate
VolumeNormal newborns
Sort of Process/OutcomeEffectiveness
NICU nosocomial infection rate
Premies/at-risk newborns
Modifiable by provider
Process/OutcomeSafety
Antibiotics for appendicitis
VolumeAll ages
Not really Process
![Page 22: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/22.jpg)
CHART Future Directions
• Perspective• What to measure• Evidence base• Sample size
![Page 23: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/23.jpg)
Summary
• Pediatric public reporting in its infancy
• Lots of room for improvement
![Page 24: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/24.jpg)
ThanksCHART Core• R. Adams Dudley, MD MBA• Mitzi Dean, PhD• Ted Karrison, MS • James Anderson, PhD
Peds Workgroup• Diana Dooley (CCHA)• Erin Givens (CSCC)• Jeff Gould (Stanford)• Greg Janos (Sutter)• Tom Klitzner (UCLA)• Paul Kurtin (UC San Diego)• Paul Sharef (UCLA)
![Page 25: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/25.jpg)
Extra slides
![Page 26: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/26.jpg)
Public reporting: adult history
• Florence Nightengale• Ernest Codman
• Mortality after Coronary Artery Bypass Grafts
• Medicare
![Page 27: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/27.jpg)
Public reporting: mechanisms?
• Free market mechanism:
Comparative info about healthcare quality
payors (employers, health plans, and patients)
choose higher quality providers
financial rewards will flow to better performers
(and away from poorer ones)
• “Self-improvement” mechanism:
Comparative info about healthcare quality
providers (hospitals, medical groups, individual physicians)
better awareness of quality issues
quality regulation cost containment quality improvement
![Page 28: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/28.jpg)
Do “consumers” act on the information?
• Free market mechanism: Comparative info about healthcare quality payors (employers, health plans, and patients) choose higher quality providers financial rewards will flow to better performers (and away from poorer ones)
• “Self-improvement” mechanism: Comparative info about healthcare quality providers (hospitals, medical groups, individual physicians) better awareness of quality issues quality regulation cost containment quality improvement
YES and NO
Marshall MN et. al, JAMA 2000
![Page 29: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/29.jpg)
Public reporting: mechanisms for exclusive
breastfeeding• Volume• Data availability
• Low prevalence of conditions• Appropriate processes?• Appropriate outcomes?
![Page 30: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/30.jpg)
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?
•Hospital-based supports•A “critical period”?
![Page 31: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/31.jpg)
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers Hospital “critical period”?
•Hospital-based supports•A “critical period”?
![Page 32: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/32.jpg)
Public reporting: mechanisms for exclusive
breastfeeding• Prenatal information/decision-making
– Family/Friends– Healthcare providers– Social agencies (e.g. WIC)
• Immediate post-natal period– Family/Friends– Healthcare providers
•Hospital-based supports•A “critical period”?
![Page 33: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/33.jpg)
Performance – three approaches
For CHART’s exclusive breastfeeding measure:
Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)
where:• Ÿh is mean exclusive breastfeeding rate for each hospital• Ýh is mean exclusive breastfeeding rate for all newborns
(statewide average)• No other individual level adjustments• Hospitals admitting <30 newborns annually are excluded
![Page 34: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/34.jpg)
Background
![Page 35: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/35.jpg)
American healthcare lacks quality and equity
![Page 36: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/36.jpg)
Public reporting is one strategy aimed at improving the quality of that care
• “Free market” mechanism– consumers/payors
• “Self-improvement” mechanisms– providers, individuals and organizations
• Evidence equivocal
![Page 37: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/37.jpg)
Public reporting’s role in eliminating (or exacerbating) racial/ethnic disparities is unknown
“Free market” “Self-imprvmnt” Consumers / payors Providers
Consequence:• Improve Want to go somewhere Want to be viewed
as “equitable” “equitable”
• Worsen Promotes segregation
• Status quo
![Page 38: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/38.jpg)
Unique opportunity to explore public reporting’s potential role in presenting racial/ethnic disparities
Exclusive breastfeeding rates in California hospitals:• Enough patients
– 10% of all American newborns born in California (500,000/year)
• Enough hospitals– 283 are licensed to provide pediatric services
• Good racial/ethnic data (as currently available)
– standardized collection– complete– variation expected
![Page 39: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/39.jpg)
This goal of this paper is to explore three approaches to incorporating information on racial/ethnic disparities into hospital public reporting:1. Adjusting expected performance for
race/ethnicity
2. Stratifying performance by race/ethnicity
3. Developing a ‘disparity’ score
![Page 40: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/40.jpg)
Hypotheses:
1. Hospital performance/rankings will change depending on how race/ethnicity is incorporated into performance methodology
2. Each methodology will have ‘pros’ and ‘cons’
![Page 41: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/41.jpg)
Methods
![Page 42: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/42.jpg)
Study Design
Cross-sectional
Primary independent variables: - Different performance measurement methodologies:
1. Proportional2. Stratified3. + ‘Disparity’ score
Primary dependent variables: - Changes in hospital rank
![Page 43: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/43.jpg)
Performance – traditional methodIn general:
Zh = Ÿh – Ýh . VAR (Ÿh – Ýh)Where:• Zh is standardized performance• Ÿh is observed mean performance including adjustments• Ýh is expected mean performance including adjustments• Each adjusted for patient characteristics
Conventionally:• Hospitals with <30 observations are excluded• Performance can be estimated using standard frequentist
approaches• Or using Bayesian ones
![Page 44: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/44.jpg)
Performance – alternative approaches
Traditional ZhT = ŸhT – Ýall / VAR (ŸhT–Ýall)
Proportional ZhP = ŸhP – Ýall / VAR (ŸhP–Ýall)
Stratified ZhWh = ŸhWh – ÝallWh / VAR (ŸhWh–ÝhWh)ZhAA = ŸhAA – ÝallAA / VAR (ŸhAA–ÝhAA)ZhHi = ŸhHi – ÝallHi / VAR (ŸhHi–ÝhHi)ZhAs = ŸhAs – ÝallAs / VAR (ŸhAs–ÝhAs)ZhOt = ŸhOt – ÝallOt / VAR (ŸhOt–ÝhOt)
‘Disparity’ score
=0 if ZhWh = ZhAA = ZhHi = ZhAs = ZhOt
![Page 45: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/45.jpg)
Data Source
California Department of Public Health Center for Family HealthGenetic Disease Screening ProgramNewborn Screening Data 2006• Mandated statewide screening program• Established 1966• Screening rate 99%
![Page 46: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/46.jpg)
Data Collection
As part of the Newborn Screen, all providers are required to answer the following questions:
“All feeding since birth: (check only one box)
[]Breast only []Formula only []Breast & Formula[]TPN/Hyperal []Other. (SPECIFY):__________________________________________________”
“Race/ethnicity: (check all that apply)
[]White []Hispanic []Black []Chinese []Japanese[]Korean []Cambodian []Laotian []Vietnamese []Filipino[]Asian Indian []Middle Eastern []Native American[]Samoan []Other (SPECIFY):________________________________________________________”
![Page 47: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/47.jpg)
Data Quality - Missing
2006
Number of newborns• 492,587 in dataset – cross-checked with Vital Statistics
Breastfeeding status• Indicated 97.2%• Missing 2.8%
Race/ethnicity noted• Indicated 97.4%• Missing 2.6%
![Page 48: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/48.jpg)
Data Quality - Validity
2006
Breastfeeding status GDSP ?NSLAH• Exclusive ~40%• Any ~90%
Race/ethnicity noted GDSP ?Census• White• African American• Hispanic• Asian• Multi• Other
![Page 49: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/49.jpg)
PRELIMINARY RESULTS
271 eligible hospitals174 participating in CHART
![Page 50: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/50.jpg)
PRELIMINARY RESULTS
California 2006
Newborns Exclusive Breastfeeding Rate
n mean sd
Total 492,587 0.427 0.495
White 125,136 0.637 0.481
African America 24,018 0.330 0.470
Hispanic 261,456 0.321 0.467
Asian/PI 45,566 0.440 0.496
Other/Multi 23,353 0.500 0.500
Missing 13,058 0.526 0.499
![Page 51: Fundamental issues](https://reader035.vdocuments.mx/reader035/viewer/2022062304/56813049550346895d95f4bb/html5/thumbnails/51.jpg)
Additional Analyses
Impact of dropping hospitals with <30 observations, particularly with the Stratified approach
Treatment of missing values - zeros - dropped
Frequentist versus Bayesian approach