![Page 1: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/1.jpg)
Prepared by Alfa D’Amato MIPA, MIFA, MPA, MHSM, MPASR
Deputy-Director, ABF Taskforce – NSW Health August 2015
Activity Based Management in NSW
Data Quality
![Page 2: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/2.jpg)
Context NSW Health budget > 25% of NSW’s budget • 3% over budget = DPC budget
1
225 public hospitals (98 ABF) around 106,000 FTE • 5,600 people are admitted to a public
hospital daily • 200 – babies are born
2
National Health Reform Agreement • applicable until 2017 • Commonwealth contributes 45% of
efficient growth • C’th share Uncapped
3
Appropriation 2015/16 $19.6bn Health
![Page 3: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/3.jpg)
“To give the same kind of care to the same kind of patient, some hospitals cost two or three times more than others in the same state.”
“Today, the price paid…includes costs that can and should be avoided.” “Setting the right price is crucial, but it won’t work on its own. Hospitals need to know a lot more about where they stand. They need detailed information about where their avoidable costs are and how they compare to their peers.”
“Activity-based funding is a good pricing system, but cost data can help us improve it.”
![Page 4: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/4.jpg)
Evidence-informed policy-making
“Policy decisions do not wait for excellent information to become available; decisions will be taken even where ‘evidence’ is fragmentary and uncertain”
Brian W. Head (2013) Evidence-Based Policymaking – Speaking Truth to Power? - Australian Journal of Public Administration, vol. 00, no. 0, pp. 1–7
![Page 5: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/5.jpg)
Evidence-informed policy-making
‘evidence [..] can be improved if appropriate standards of transparency and accountability are followed in the process of gathering, analysing, interpreting, and presenting evidence for policy.
Brian W. Head (2013) Evidence-Based Policymaking – Speaking Truth to Power? - Australian Journal of Public Administration, vol. 00, no. 0, pp. 1–7
![Page 6: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/6.jpg)
Know
ledg
e
Und
erst
andi
ng
Info
rmat
ion
Enab
le In
tuiti
on
Dat
a
Clear linkages between dimensions, KPIs, Quality, outcome, patient journey
Fact
s
Business Modelling, forecasting, system optimisation, clinical and financial information
Purchasing, reusable information, utilisation data
Data in context, easily retrievable data, benchmarking
Organised data, standard platform, consistency, business rules , PPM2
Disorganised Data, no timely retrieval, no reconciliation - financial
Ladder of Business Intelligence (LOBI)
ABF
ABM
START
NOW 2012
2015
Foundations – clinical costing data
Clinical Engagement Education
The Journey to ABM Outcomes
![Page 7: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/7.jpg)
FIRST STEPS…(Baby steps)
ORGANISED DISORGANISED
FACT DATA
![Page 8: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/8.jpg)
NSW Health Silos
Clinical Costing • Finance doesn’t help • Adjustments off GL • No accountability • Processing issues • Rules open to
interpretations
Finance • Input focus • Statutory accounts • Bottom line (>50K CC) • Limited awareness of
potential of Clinical Costing • It doesn’t reconcile
It’s in the P&L
Where?
![Page 9: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/9.jpg)
DNR PROCESS
LHD process costing and run Quality Check in PPM2
LHD run PPM2 DNR Module
LHD Review Fatal and Warning Validation Report
Require change
Load DNR File to MoH
Secure File Transfer and Submit to ABF Taskforce
Casemix server Picks up and validate the DNR files
SAS Picks up Activity IP, ED, NAP and Expense - Process Encounter level
quality checks - Generate Quality
report and scores
RQ App Display - Reasonableness report - SAS QA report
DRAFT SUBMISSION FINAL DNR
LHD agree on final submission
LHD Complete Reconciliation
Schedule
CE Signoff Letter Completed
Costing Team review
Reconciliation
LHD review flagged issue
Casemix server Generate aggregate Reasonableness Report
![Page 10: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/10.jpg)
State Price
ABM Portal
PHEC
NHCDC
HEX
ROGS
MHE NMDS
DNR DATA FLOW
PHEC – Public Hospital Establishment Collection NHCDC – National Hospital Cost Data Collection MHE NMDS – Mental Health Establishments National Minimum Dataset HEX – Health Expenditure ROGS – Report on Government Spending
One submission
multiple use
![Page 11: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/11.jpg)
Preliminary results – ↑Reliability
• Increase consistency and reliability of costing results – Finance requirements – Reconciliation to Audited Annual Return
• All cost must be allocated – Documented applicable standards
• Cost Accounting Guidelines addressed peculiar issues pertaining NSW Accounting standards (Funding for Affiliate Health Organisations)
![Page 12: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/12.jpg)
SECOND STEP
BENCHMARKING ORGANISED
DATA information
![Page 13: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/13.jpg)
• Costing Standards User Group (CSUG)
• Cost Accounting Guidelines (CAG)
• Clinical Review
– NSW Workgroups
• CE Sign off of DNR – letter template
Governance
![Page 14: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/14.jpg)
DNR GOVERNANCE
LHD/SHN
Chief Executive
Local Structure
Costing Officer
NSW Health
Director ABF Taskforce
Costing Team
CSUG
CAG
IHPA
JAC
NHCDC AC
AHPCS
CSUG – Costing Standards User Group CAG – Cost Accounting Guidelines IHPA – Independent Hospital Pricing Authority JAC – Jurisdiction Advisory Committee NHCDC AC – National Hospital Cost Data Collection Advisory Committee AHPCS – Australian Hospital Patient Costing Standards
![Page 15: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/15.jpg)
DNR Timeline
![Page 16: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/16.jpg)
The ABM Portal
DESCRIPTION An online patient and activity data interrogation application to compare and benchmark hospital performance and examine real patient journeys across facilities and over time.
PURPOSE To assist clinicians and managers in managing unwarranted cost and clinical variation, evaluating models of care and identifying the impact of funding on patient care delivery.
CURRENT STATUS Available to nominated users in all NSW Local Health Districts (LHDs), Specialty Health Networks (SHNs) and MoH.
![Page 17: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/17.jpg)
The ABM Portal
• Benchmarking hospitals/LHD cost and ALOS performance
• Following a single patient journey • Drill down to individual cost buckets
for a patient/procedure
• Find diagnoses with highest o Ave cost per NWAU o Number of NWAUs o ALOS o Encounter volume
![Page 18: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/18.jpg)
Benchmark
Compares LHDs/SHNs’ or facilities’ costs, number of encounters, average length of stay, and NWAU volume in one financial
year
DRG C16Z Lens Procedures
LHD 1– Cost per NWAU above
State Price
LHD 1 – 425 NWAU
![Page 19: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/19.jpg)
Clinical Variation Major Hospitals Peer Group Facilities
Identifies models of care that need to be addressed
DRG E65B Chronic Obstructive Airways Disease
Is there variation in
average cost per encounter
between facilities?
How does ALOS
compare?
Does any ‘cost bucket’ stand out?
![Page 20: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/20.jpg)
THIRD STEP (Make the most out of it)
USABLE Benchmarking
information knowledge
![Page 21: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/21.jpg)
ENSURING DATA QUALITY
![Page 22: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/22.jpg)
• Standardisation
• QA Reports
• DNR Module and SQL Validations
• Draft Submission
• Reasonableness and Quality (RQ) Application
Ensuring Data Quality
![Page 23: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/23.jpg)
• SQL for PPM2 queries developed centrally and distributed to Costing Teams
• PPM2 Standard QA reports
• Aligned with National QA
• Reviewed yearly
• Other aspects in Reconciliation and sign off
Quality Assurance Reports
![Page 24: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/24.jpg)
Using metrics to score the quality of costing data
NSW Health ABF data processes include a newly developed step which focuses on data quality and issue a scoring system
The Reasonableness and Quality Application allows all LHDs to review how their scored against the metric for patient level costing data before formally submitting the results to the Ministry
The described process has been designed as part of the continuous improvement cycle around patient level costing
![Page 25: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/25.jpg)
![Page 26: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/26.jpg)
Built and published for the use of staff involved in the preparation and review of the District & Network Return (DNR)
A consolidation of the Reasonableness tests and the Data Quality Checks encounter (which are generated in SAS) and distributed via our RQ App.
Only Used in the Draft submission periods to identify encounters that have outlier cost results which may or may not require remediation – Draft DNR submissions made prior to 3.30 pm are included
in the RQ app which is refreshed by 9 am the next business day
The RQ App is…
![Page 27: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/27.jpg)
The RQ App
The Reasonableness section is a series of reports that look at the
average cost results at an aggregate level and will identify
any significant anomalies.
The Quality section is a series of patient level data quality
checks that are run for ABF facilities only.
Data quality checks are assigned a score to provide an indication of the data quality. Quality checks
are either a pass (1) or fail (0) or a score between 0 and 3 depending
on the number of errors.
![Page 28: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/28.jpg)
Patient-level Data Quality Checks
52 validations with a pass/fail or scaled score across all ABF workstreams
Missing data
• Episode is missing costs (eg. No recorded nurse salary and wages cost)
• Prostheses costs missing or outside national limits
• Episode of care type missing or inconsistent with class
Data inconsistent with the average
• Record has costs too high or too low compared to the average
• Record is identified as a possible outlier within the classification
Error codes used
• Record or mode of separation is given an error code (eg. DRG 960Z - Ungroupable)
Data mismatching
• Mismatch of ICU costs and ICU hours
• Mismatch of Surgical DRG recorded and Operating Theatre costs
• Mismatch of EMU and ED costs
• L61Z Haemodialysis average cost and Tier 2 10.10 Renal Dialysis at hospital average cost are within +-10%
![Page 29: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/29.jpg)
Patient-level Data Quality Checks
52 validations with a pass/fail or scaled score across all ABF workstreams
This will inform the ABF Taskforce as to whether some or all of the records that may have a significant or material impact on the average cost, should be included or excluded in the calculation of the State Price for 2016/17.
![Page 30: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/30.jpg)
RQ App: Summary - Reasonableness
![Page 31: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/31.jpg)
RQ App: Top 20 Variance
![Page 32: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/32.jpg)
RQ App: Feeder systems
![Page 33: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/33.jpg)
DNR AUDIT PROGRAM
![Page 34: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/34.jpg)
Current Audit Program
Audit Test DNR 2011/12
DNR 2012/13
DNR 2013/14
Financial Reconciliation Test Yes Yes Yes FMIS adjustment Test Yes Yes Yes Non Patient Product sub program Test Yes Yes Yes Facility level non admitted sub programs expense and activity alignment Test
Yes Yes Yes
Program Fraction Test (especially for T&R) No Yes Yes
Patient Product Cost Test No No Yes
![Page 35: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/35.jpg)
New Audit Program NSW Auditor General recommendation
Annual audit (mandatory)
Consultant engaged to develop program
Rollout 20th July
This is in addition to the Independent Financial Review conducted by IHPA
Planning Current
Inputs Developing
Internal Audit Program
Developing Testing Program
Test audits Implementing roll-out
test audits by June 2015
![Page 36: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/36.jpg)
Audit objective: DNR is fit for purpose and provides consistent and accurate data
Conducted by Internal Audit teams within each LHD/SHN
Attestation will need to be submitted to Secretary
Risk based with three lines of inquiry:
– Is patient data reliable and accurate?
– Are costing methodologies used appropriate and robust?
– Does preparation of DNR comply with NSW CAG?
New Audit Program
![Page 37: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/37.jpg)
Following the patient journey
The ABM Portal allows users to track one patient’s care across all health workstreams – ED, acute, subacute, non-
admitted – at a single facility over the years.
How has Patient A’s treatment progressed since 2011 and what are their hospital costs?
![Page 38: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/38.jpg)
Frequent Patients
Displays frequently admitted patients at an LHD/SHN or
facility to analyse readmission and
representation rates
Patient A: 58 ED encounters 323 days in hospital Total ED cost: $32,291
![Page 39: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/39.jpg)
Patient A 2011-12
ABM PORTAL Single Patient Journey
Chronologically details single patient journey at a facility: treatment
received, number of encounters, cost per encounter, and length of stay.
TOTAL COST: $11,005
![Page 40: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/40.jpg)
Patient A 2012-13
ABM PORTAL Single Patient Journey
TOTAL COST: $65,887
Chronologically details single patient journey at a facility: treatment
received, number of encounters, cost per encounter, and length of stay.
![Page 41: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/41.jpg)
Patient A 2013-14
ABM PORTAL Single Patient Journey
Chronologically details single patient journey at a facility: treatment
received, number of encounters, cost per encounter, and length of stay…
TOTAL COST: $47,722
![Page 42: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/42.jpg)
Patient A 2014-15 (half year)
ABM PORTAL Single Patient Journey
Chronologically details single patient journey at a facility: treatment
received, number of encounters, cost per encounter, and length of stay…
…Across years
TOTAL COST: $10,671
![Page 43: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/43.jpg)
Focusing on measuring improvements year on year
Strengthening Internal Audit skills
Leverage more from the draft submission time
Next Steps
![Page 44: Alfa D’Amato - ABF Taskforce - Health System Support Group - Using metrics to score the quality of costing data](https://reader031.vdocuments.mx/reader031/viewer/2022022412/58f1926c1a28ab52048b4581/html5/thumbnails/44.jpg)
Is it just about the numbers?
“Businesses don’t run on numbers, they run on responses to numbers”
Bri Williams, New Truths About Numbers, In the Black