national cataract audit. data validation... · eliminate poor practice use scale to reduce personal...
TRANSCRIPT
![Page 1: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/1.jpg)
National Cataract Audit
![Page 2: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/2.jpg)
National Cataract Audit
Why?
• Quality assure NHS services Show surgeons / centres their results
Learn from the best
Eliminate poor practice
Use scale to reduce personal burden of audit on individuals
Risk adjust to avoid penalising surgeons who take on difficult cases
![Page 3: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/3.jpg)
Cataract Audit participation
Looking ahead – 114 of 125 NHS Centres signed up
![Page 4: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/4.jpg)
4
![Page 5: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/5.jpg)
![Page 6: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/6.jpg)
2018 Report
• Surgeon and Centre results on NOD website
– 1918 Surgeons invited to check their latest results
• 92 Centres being reported
– Majority Medisoft
– 4 OpenEyes
– 7 Other
• 181,965 Operations
6
![Page 7: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/7.jpg)
Is all this really worth the effort?
PCR % RATE 2010-11* 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 Overall
N = 64,923 N = 82,879 N = 89,602 N = 100,333 N = 107,255 N = 148,073 N = 182,064 N = 775,129
Consultant surgeons
(485,757) 1.55 1.48 1.46 1.33 1.27 1.18 1.10 1.28
Career grade surgeons
(88,272) 1.82 1.64 1.78 1.25 1.69 1.29 1.54 1.55
More experienced
trainee surgeons
(171,458)
2.87 2.65 2.41 2.33 2.21 2.17 1.95 2.29
Less experienced
trainee surgeons
(29,642)
3.66 3.62 3.02 2.35 2.57 2.53 2.33 2.71
Overall 2.00 1.85 1.77 1.56 1.56 1.46 1.38 1.59
Overall 30% reduction in PCR rate from 2010-11 to 2016-17 *Years run from Sept to August
![Page 8: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/8.jpg)
Data Validation and Data Completeness Overview and Data Entry
Professor John Sparrow,
Consultant Ophthalmologist and NOD Audit Clinical Lead
![Page 9: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/9.jpg)
Data Completeness
• Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
• Complications – 100% PCR or some other complication or none
• VA Pre- & Post-op – Variable between centres
• Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 10: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/10.jpg)
Number of Cases and Estimated % Completeness
Data from 2015-2016 Report https://www.nodaudit.org.uk/
![Page 11: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/11.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
• Complications – 100% PCR or some other complication or none
• VA Pre- & Post-op – Variable between centres
• Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 12: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/12.jpg)
![Page 13: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/13.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
Complications – 100% PCR or some other complication or none
• VA Pre- & Post-op – Variable between centres
• Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 14: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/14.jpg)
% Cases with Missing Pre-op VA by Centre
![Page 15: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/15.jpg)
% Cases with Missing Post-op VA by Centre
![Page 16: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/16.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
Complications – 100% PCR or some other complication or none
VA Pre- & Post-op – Variable between centres
• Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 17: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/17.jpg)
![Page 18: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/18.jpg)
![Page 19: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/19.jpg)
![Page 20: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/20.jpg)
![Page 21: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/21.jpg)
![Page 22: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/22.jpg)
![Page 23: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/23.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
Complications – 100% PCR or some other complication or none
VA Pre- & Post-op – Variable between centres
• Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 24: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/24.jpg)
% Cases with Specific Ocular Pathologies
![Page 25: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/25.jpg)
% Cases with Ocular Pathology
![Page 26: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/26.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
Complications – 100% PCR or some other complication or none
VA Pre- & Post-op – Variable between centres
Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 27: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/27.jpg)
Data Validation
• Contributors invited to check
– Number of cases reported
– Adverse outcomes
• PCR
• VA Loss (needs both Pre & Post-op VA)
– Risk indicators data
• For case complexity adjustment
![Page 28: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/28.jpg)
![Page 29: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/29.jpg)
![Page 30: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/30.jpg)
![Page 31: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/31.jpg)
![Page 32: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/32.jpg)
Known
Number cases done % PCRs
Unadjusted data 4503 2.07% 93
Benchmark = 1.10%
Adjusted% = 1.25%
Adjusted% = Benchmark * ObservedN / ExpectedN
ExpectedN (from case complexity) is unknown
ExpectedN = Benchmark * ObservedN / Adjusted% PCRs
ExpectedN = 82
ObservedN is higher than expectedN, therefore
adjusted rate of 1.25% is higher than benchmark of 1.10%
Conclusion:
Centre's recorded case complexity is overall less complex
than the average complexity from all centres
Case Complexity Centre Example
![Page 33: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/33.jpg)
Known
Number cases done % PCRs
Unadjusted data 606 1.65% 10
Benchmark = 1.10%
Adjusted% = 0.92%
Adjusted% = Benchmark * ObservedN / ExpectedN
ExpectedN (from case complexity) is unknown
ExpectedN = Benchmark * ObservedN / Adjusted% PCRs
ExpectedN = 12
ObservedN is lower than expectedN, therefore
adjusted rate of 0.92% is lower than benchmark of 1.10%
Conclusion:
Surgeon's recorded case complexity is overall more complex
than the average complexity from all surgeons
Case Complexity Surgeon Example
![Page 34: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/34.jpg)
Data Completeness
Number of cases reported – Recent joiners acknowledged as such in report (date)
– Comparison between submitted data and NHS digital (pro-rata)
Complications – 100% PCR or some other complication or none
VA Pre- & Post-op – Variable between centres
Risk indicators data – PCR
– VA Loss (doubling or worse of visual angle)
– Important for case complexity adjustment
![Page 35: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/35.jpg)
![Page 36: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/36.jpg)
![Page 37: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/37.jpg)
![Page 38: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/38.jpg)
![Page 39: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/39.jpg)
![Page 40: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/40.jpg)
![Page 41: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/41.jpg)
![Page 42: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/42.jpg)
![Page 43: National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal burden of audit on individuals Risk adjust to avoid penalising surgeons who take](https://reader034.vdocuments.mx/reader034/viewer/2022051910/5fff31018e6a12779515d82f/html5/thumbnails/43.jpg)