development of quality measures and data dictionary towards national definitions and agreed...
TRANSCRIPT
DEVELOPMENT OF QUALITY MEASURES AND DATA DICTIONARY
Towards National Definitions and
Agreed StandardsDr Peter Jones MSc EBHC (Oxon) FACEM
With thanks to Dr Alana Harper FACEM and Dr James LeFevre FACEM
MOH Forum, Wellington 5/5/2014
Why Bother?
Each ED could just decide what to measure and how to measure it
Problems Time Resources Skills Duplication (x26) Accuracy Comparisons
‘Admission’=3hrs?
To maximise the return for effort in a resource constrained environment
Aim of this presentation
How to operationalise the MOH suite of quality measures using a real world example Data definitions / development of a ‘Data Dictionary’ Data collection for process and clinical outcomes
My QI Background Auckland City Hospital / ADHB
Morbidity / Mortality 2000-05 Time to Thrombolysis KPI 2000-05 ED Ultrasound Credentialing 2000-03 Procedures Database 2000-05 AHQAS 2001 Cardiac Arrest Documentation 2000-2010
UHCW NHS Trust Audit Lead 2005-06
ACEM Quality Management Subcommittee 2011-current
SOPH PhD Student: Best measure of ED Overcrowding?
Shorter Stays in ED National Research Project
ADHB/ SOPH Auckland University 2009-current Health Policy, Effective Practice, Epidemiology,
Māori Health, Health Economics, Biostatistics, Clinicians
HRC funded 10-588 MREC approved MEC 10/06/060 Multi-stream Mixed-Methods Research
What was done to implement the SSED target? What effect on markers of care? Lessons for future health/public service policy
Kaupapa Māori Research Approach
SSED NRP Stream 2 Quality of Care
Quantitative analysis 13 ‘Quality’ Measures
Process and Outcome Nationwide (routine flow data) 4 Case Study Sites (clinical markers)
Richness of information Target results Q1 2010
Any difference 2006-08 vs 2010-12? Adjust for Ethnicity / Age / Deprivation
SSED NRP Quality
Outcomes Primary (Nationwide)
ED LOS Access Block (wait for admission >8hr in ED) Overcrowding
Hospital LOS Re-attendance rates within 48 hours of discharge Re-admission rates within 28 days of discharge
SSED NRP Quality Outcomes
Secondary Mortality (N)
hospital inpatients for ED attenders at 10, 30 and 90 days
Time to treatment in acute asthma (4 sites) Time to reperfusion for myocardial infarction (1 site) Time to theatre for fractured neck of femur (1 site) Time to appendectomy for acute appendicitis (1 site) Time to antibiotics for severe infections (1 site) Proportion of patients who leave without being seen
(N) ‘Gaming’ the target (N)
a spike of ED discharges at or near the target time (N) digit bias in recording time of ED discharge (N) re-designation of ED patients to ‘stop the clock’ (CS ?N)
SSED NRP Quality Clinical Markers
Clinical Quality Markers Selection Literature review / Evidence Search Reference Group Meeting December 2010
SSED NRP Quality Clinical Markers
Clinical Quality Markers Selection Critical Appraisal of Quality Indicators (QICA)
SSED NRP Quality Data Required
Two Sources, three types of Information NZHIS
Clinical diagnosis data from 4 case sites (ICD) DHBs
Routinely collected process data
ED and acute direct inpatient admissions (PIMS) Hospital Bed Occupancy
Census at night (Bed Management System)
SSED NRP QualityData Collection Plan
7yrs Presentations
Site SpecificClinical Indicators
Process Indicators
SSED NRP Quality Data Dictionary
SSED QualityData Dictionary
Data elements piloted in consultation with analysts from three DHBs
Different names for same process e.g. Episode=Event=Visit=Case
PIMS different
And NZHIS No time stamped process data Highlighted inconsistent submission from DHBs
SSED NRP QualityData Dictionary
Then consulted all 20 DHBs Different PIMS collected data in different ways Not all data elements currently defined Some ED & Inpatient systems not integrated
Level of data capture differs Capture patient event separately Not all Process measures captured by all DHBs
SSED NRPData Dictionary
Dictionary Facilitated Data Cleansing Convert different DHB data format to unified set Identify outliers and validate Link NZHIS data with DHB data
NNPAC/NMDS Identify duplicates from DHB data DHB data has elements missing from NNPAC/NMDS
Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
2006 2007 2008 2009 2010 2011
4
5
6
7
8
9
10
11
12
0
100
200
300
400
500
600
700
800
900
1000
Mean ED LOSData Quality versus true outliers
Mean ED LOS (Hours)
Count of ED LOS > 24 Hours
TIme
LO
S (
Ho
urs
)
Nu
mb
er o
f P
atie
nts
Per
Mo
nth
Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
Jul
Aug
Sep Oct
Nov
Dec Jan
Feb
Mar
Apr
May Jun
2006 2007 2008 2009 2010 2011
4
5
6
7
8
9
10
11
12
0
100
200
300
400
500
600
700
800
900
1000
Mean ED LOSData Quality versus true outliers
Mean ED LOS Outliers Fixed
Count of ED LOS > 24 Hours
TIme
LO
S (
Ho
urs
)
Nu
mb
er o
f P
atie
nts
Per
Mo
nth
SSED Indicator Selection and Data Dictionary Resources Required 24 months work 0.5 FTE Emergency Medicine Specialist 1 FTE Research Fellow 1FTE Data Manager Office / Dedicated desk space PCs (high spec / dual screen) Software (reference management / pdf writer / PIMS) Online Journal Access Monthly team meetings Academic and Administrative support Liaison with NZHIS and all DHB IS departments
SSED NRP Clinical QualityData Collection
Site SpecificClinical Indicators
• List of ICD codes to NZHISJ45 (0,1,8,9); J46
• Events with that ICD code in each time period for each site
Date / NHI / Demographics
• Random sample events
• List of NHIs to the sites• Manual collection of data
Trained senior clinician data collectorsmultiple site visits
• Data accuracy / cleaning
SSED NRP Clinical QualityData Collection Tools
‘Intelligent’ spreadsheets Data validation checks / protected formulas
SSED NRP Clinical Quality Data Cleansing
Data Accuracy ≈15% independently checked
Eligibility 283/297 = 97% Primary outcome 366/387 = 95% Comorbidities 1198/1288 = 93%
Data Cleaned Errors corrected where possible No imputation for missing data
Time Stamp Data Also Required Cleaning
SSED Data Collection Resources Required
12 months work 2-3 sets of records per hour for clinical quality indicators
0.5 FTE Emergency Medicine Specialist 1 FTE Research Fellow 1FTE Data Manager Office / Dedicated desk space PCs (high spec / dual screen) Software (data collection tools) Clinical records departments
Desk / PC / Laptop
Development of Quality Measures and Data Dictionary: What’s Needed? MOH
Data definitions / dictionary IT at each site standardised and audited Standardised ‘intelligent’ Tools for data collection
Web vs local
DHB Dedicated resources for quality
Space / Time / Admin / Clinical Records People (thank you NZMC)
More than lip-service!
Development of Quality Measures and Data Dictionary
Towards National Definitions and Agreed Standards
Questions / Discussion
SSED NRP Stream 2 Quality Method: Clinical Markers
Eight conditions identifiedRepresent whole systemPilot 50 sets notesElectronic DEF with logic (reduce errors)Clinically important difference aprioriSample size 90% power, alpha 0.05 (2 tailed)
SSED NRP Stream 2 Quality Method: Clinical Markers
Sample Size Calculations Not all outcomes could be measured at all sites
10000 sets of notes = not feasible Clinical records departments / our resources
Quality Indicator Critical Appraisal Tool 2 Authors independently appraised the indicators Outcome brought to whole team Different views Compromise
Desire to measure >1 outcome and reflect whole system
1 outcome all sites; 1 other outcome each site
SSED NRP Stream 2 Quality Method: Clinical Markers Asthma (30 min difference time to steroid)
Common / relevant / important All ages / Mäori / Pacific Data available / accessible / sample size manageable all sites
Acute Myocardial Infarction (15 min difference time to lysis) Evidence strong but practice changed over time:1 site
Sepsis (60 min difference time to steroid) Evidence moderate / important / relevant / all ages / Mäori / Pacific Sample size issues + difficult data collection: 1 site
Appendectomy (12hr difference time to theatre) Whole system (balance)
Fracture NOF (6hr difference time to theatre) Whole system (balance)