local clinical audit, success and failure (how can measuring care lead to improvement ) cormac...
TRANSCRIPT
Local Clinical Audit , Success and Failure
(How can measuring care lead to improvement)
Cormac Kelly, Clinical Lead for AuditClaire Bowler , Senior Quality Outcomes FacilitatorAmanda McFie, Quality Outcomes Facilitator Eric Robinson, Quality Outcomes Manager
Robert Jones and Agnes Hunt HospitalOswestry, Shropshire
RJAH Orthopaedic Hospital
• Leading orthopaedic “centre of excellence”• During 2012 a total of 11960 procedures
took place• Top of the Patient Experience review• Top for Hospital Food• Worst waiting times in 2013
Why I /you bother with CA• I /you want to do a good job• I /you want to know what is working well so I can do
more of it• I /you want to know what is working badly so I can
stop doing it• I /you want to be able to give patients honest answers
to questions such as How often does the operation work in your hands ?What complications have you seen ?• I want to be better than my colleagues at everything• I want my colleagues to think like me
Views of doctors on Clinical Audit (2003)
• “When has CA ever produced anything of Quality or Value ?”
• What is in it for me or my patients ?• Love to, but no time• We are doing OK surely ?• Its all too complicated
Dealing with Difficult Professionals
• I need an audit before the end of the month because…
• Ok its not audit .. Lets call it service evaluation ?• If it is not approved I will do the study anyway…
Freedom of Information Act….• Stop hastleing me about the report.. I presented it
at the BOA last week• Here is the paper.. Do whatever you want with it
• Patient representative 2012• CCG representative 2012• Chaired by Former Medical Director• Usual enthusiastic Clinical audit Facilitators• Structured minutes• Poor attendance of Departmental Leads,
Senior management/Executive• No Non Exec representation
The Clinical Audit Committee
• Half day twice yearly, 3 hours• All elective activity “cancelled”• Invitation to present your audit to Doctors , nurses, Allied Health professional,
managers• Update on CA , Clinical audit lead• Guest speaker• Opportunity to show off good projects and offer constructive criticism of poorer
projects• Alteration of Action plan• Increasing attendance and enthusiasm (average attendance 50 staff)• Cancelled in 2009• Incorporates Mortality /morbidity meeting• We want more………“£100,000.00 of lost revenue to the Trust “
The Multidisciplinary Clinical Audit Meetings (MDCAM) 2004
• Oh, I can….• To share good experience of clinical audit • To enthuse others• To do audit across hospitals• To get clever people to educate us • Supported by Specialist Orthopaedic Alliance
and HQIP• Annual meeting
Its all in the name
• 2012. Amalgamation of Clinical audit and Outcomes Unit
• Is it time to change the emphasis from CA to QI• Quality Outcomes and Improvement . REJECTED
The problem with Junior doctors
• Target is 2 audits in every 6 month cycle• No CA training in postgraduate curriculum• Until recently no scrutiny of CA activity• Still seen as a chore rather than a quality
improvement process • Action plan and improvement rarely witnessed
HQIP knowledge among postgraduate Orthopaedic trainees
• 3 questions• Have you had any formal training in CA
methodology• What is HQIP ?• Where can you access online training ?
Answers
RESULTS
Poor knowledge and little training HQIP……….“High quality Intercourse training”
Poor knowledge among trainees
Our solution
• HQIP Intermediate Clinical audit Course online for doctors
• Clinical audit training is now part of Mandatory training for all staff, e-learning
• Linked up with CASC• 3 successful Training days for staff (at budget
cost) inside and outside Trust• Assign junior doctors to audits at the start of
their attachments
Patient InvolvementThe Out-patient Survey
• Not an audit per se• Designed and conducted with
patient involvement• Soft end point• No hard stats
Results
• Patients love us• Some areas where improvements can be made• Was waiting an hour to be seen but “it was
worth it”
• Illustrates the weakness of using patient opinion only
Fasting for Theatre Audit
• Simple• Easy to understand the benefit• Easy to measure• Not Priority One• Produced a meaningful action plan• Re audit demonstrated a big improvement• RESULT patients are more comfortable
arriving for surgery
Local anaesthetic for Hand surgery
• “A new method of local anaesthesia”• One doctor• Collected, and analysed own results• ? Data collected before audit proposal submitted• Technique Not supported by some colleagues• No patient satisfaction measures• No medium term outcome measures or
complications recorded
Consent Audit
• In association with OICAN• Across 4 sites• Agreed audit tool• Allows comparison of outcome• The medical Director likes it• Keen to try another• Re audit shows improvement in documentation• Revision to include more patient perspectives
So What should I audit ?
• Not enough top down• Not enough linkage to Risk Management• What are the real issues in the Trust• Not enough patient input into choice of topic• Who decides if an audit goes ahead ? • ? Assessing group ?• Does it really need to be audited at all ?• Clinical Audit cannot cure obvious deficiencies• LISTEN TO WHAT PATIENTS ARE TELLING YOU
The problem with Service Evaluation and other projects
• Who is responsible for quality control• Does it need proper number crunching ?• Rarely involve the Clinical Audit department• Often not registered• Often slip through the net and are not registered• Many produce conclusions that are not based on
results, are biased, are presented outside the Trust as “Research”
• BIG PROBLEM
The Action Plan• The Action plan needs to be reasonable , do-able
and supported by the whole team including budget holders
• So where does risk get identified and acted upon• Who is responsible for the action plan happening• Use an Action Planner to facilitate the chase• CA should chase action plan but when does the
chasing stop ?• ? Datix when actions don’t seem to be going
anywhere
Linkage to other committees
• How are concerns based on audit elevated to other committees
• Is there clear guidance and protocols on this ?• Is risk identified and acted upon ?• Is results disseminated to the relevant people
Problems not going away…
• Many units continue to struggle locally in many aspects of Clinical audit
• Increasing pressures on organisations to “get it right” without good knowledge of Quality improvement methodology
• Big increase in National Audits , NICe guidance , NHSLA requirements pushes local interest audits onto backburner
Disillusioned CA Staff
• Poor job structure• Poor training• Too much learning on the job• Much grief from management many of whom don’t understand
CA objectives • Lots of chasing uninterested medical staff• Too much to do eg National Audits “poorly designed” • Feel distant from the Board where the big decisions are made• When something goes wrong …”clinical audit is not working “
C L I N I C A L A U D I T SS E R V I C E
E VA L U AT I O N S
P R O M S O U T C O M E M E A S U R E S
R E VA L I D AT I O N
D ATA P R O V I S I O N
C Q U I N S
P E E R R E V I E W S M O R TA L I T Y M O N I T O R I N G
PAT I E N T E X P E R I E N C E
Q U A L I T Y M E A S U R E S
N H S L A
C O N F I D E N T I A L
E N Q U I R I E SC Q CC O M M I S S I O N E R
R E Q U I R E M E N T S
N I C E G U I D A N C E
Things that annoy Little Miss Audit Facilitator
• I have a name, please use it…… No, its not Audit Girl. • Dear Mr / Mrs highly trained medical professional completing
out a proposal form isn’t brain surgery …….fill it out• No, “ the patient will have an average pain score” is not a
standard• Even if you don’t want my help ….. Still register your audit• Asking me on a Friday afternoon (4 pm) to analyse 125 patients
worth of data by Monday is not ok ………..i do have a life outside work
X
Clinical audit and Quality improvement- time for a rethink ?
Still a tick box exercise for manyContrasting levels of audit knowledge, skills and experienceCA considered time consuming, bureaucratic, managerially driven,…with potential to apportion individual blame and which has no associated personal or professional reward
Bowie et al, Jnl of Eval in Clin Practice. 18(2012) 42-48
….Assurance to the Board..
(assurance to the bored)• What about assurance to the Clinical audit
Committee ?• Is there anybody listening ?• “Chief Quality Officer” Dr Nick Black “The need for the Board’s of Trusts to engage more effectively with the quality of their services, giving as much attention to quality as is customarily given to finance, was recognized by Robert Francis (February 2013) and has recently been endorsed by Jeremy Hunt (King’s Fund, May 2013). “
‘Less pain for more gain…
…..how to survive Clinical Audit’
EDUCATION, EDUCATION, EDUCATION
Educate your staff on quality audit practicePrioritise auditsFewer audits of higher quality with more names as authorsHave a clear policy on non-audit projectsNo
• Revalidation has helped focus the mind OF DOCTORS
• Certificate of attendance at MDCAM• Summary of Involvement in Clinical audit• Enclosed in Clinical Governance package
Linking to Revalidation for Doctors
• Lay Member of CA committee• Outpatient Survey designed by patients and
conducted by patients• Planned Dementia audit• Inspirational……patients do ask the right
questions…
Involving Patients in CA
Local Quality Improvement – Successes and Failures
Katharine (Kat) Young MASenior Quality Improvement Lead, Royal Berkshire NHS FT
Chair, National Quality Improvement and Clinical Audit networkMember, National Advisory Group for Clinical Audit and Enquiries
[email protected] @Clin_Q
www.hqip.org.uk
Royal Berkshire NHS Foundation Trust
• Large District General, Reading • Strategic objectives:
– exceed patient and customer expectations – work together to create a modern and sustainable
healthcare system – deliver the best healthcare in the best possible place for
patients – provide the best place to work, train and learn.
• Re-organisation 2011 • Move to three Care Groups (Urgent, Planned and
Networked) & Clinical Quality Improvement Unit • Need to maximise clinical and financial efficiency
Where we were at
• Quality by name but not in nature
• The candy factory – I love Lucyhttps://www.youtube.com/watch?v=8NPzLBSBzPI
• ‘Insanity: Doing the same thing over and over again and expecting different results’Albert Einstein/Benjamin Franklin/Anon
• Work harder?
Role of the Quality Improvement Team
• Champion and promote culture of continuous improvement
• Leadership, expert advice• Project support• Building Capability and Capacity (facilitated
learning)
RBFT Improve Framework
Shared Vision
The vision should
answer the question
‘where do we want to get to?’ and should be
the inspiration
and framework
for planning
Assurance
CQIU involvement:
-Involvement in completion & challenge of Quality Impact Assessments
- Monitoring of balancing measures
- Research capability
SMART Aims
The aims set should be:
Specific
Measurable
Achievable
Realistic
Timeframe
The Quality Improvement
Approach
Improve it!
Improvement opportunities
Measure
Plan
RoI
Outcomes
Vital behaviours
Evaluate
Vital Behaviours
Desirable – what’s in it for them?
Enable – provide information and skills / deliberate practice
Stakeholders – crucial conversations – team and beyond
Influencers – senior engagement and support; opinion leaders
Rewards – what are the incentives?
Environment –providing the physical means to achieve the outcome
Governance
Project Management Office (PMO) involvement:
-Projects grouped by value (P1-P3)
-Tracking & monitoring
-Project documents
-Risk assessments
-QIPP Reports -Programme Board
Outcomes
What are the Process measures?
What are the Outcome Measures?
Sustainability
To sustain the improvement requires:
-Patient & staff engagement
-Alignment with goals & structures
-Infrastructure
-Credible evidence
-Adaptability
-Continual monitoring of progress
Shared Learning, show casing of examples such as MEMC, clinical leadership programme etcRBFT Quality Improvement Training Programme (training, master classes, visits to centres of excellence)Communication Strategy –sharing plans, outcomes & celebrating success
Quality Improvement Framework: Our journey towards excellence
Shared Vision
The vision should
answer the question
‘where do we want to get to?’ and should be
the inspiration
and framework
for planning
Assurance
CQIU involvement:
-Involvement in completion & challenge of Quality Impact Assessments
- Monitoring of balancing measures
- Research capability
SMART Aims
The aims set should be:
Specific
Measurable
Achievable
Realistic
Timeframe
The Quality Improvement
Approach
Improve it!
Improvement opportunities
Measure
Plan
RoI
Outcomes
Vital behaviours
Evaluate
Vital Behaviours
Desirable – what’s in it for them?
Enable – provide information and skills / deliberate practice
Stakeholders – crucial conversations – team and beyond
Influencers – senior engagement and support; opinion leaders
Rewards – what are the incentives?
Environment –providing the physical means to achieve the outcome
Governance
Project Management Office (PMO) involvement:
-Projects grouped by value (P1-P3)
-Tracking & monitoring
-Project documents
-Risk assessments
-QIPP Reports -Programme Board
Outcomes
What are the Process measures?
What are the Outcome Measures?
Sustainability
To sustain the improvement requires:
-Patient & staff engagement
-Alignment with goals & structures
-Infrastructure
-Credible evidence
-Adaptability
-Continual monitoring of progress
Shared Learning, show casing of examples such as MEMC, clinical leadership programme etcRBFT Quality Improvement Training Programme (training, master classes, visits to centres of excellence)Communication Strategy –sharing plans, outcomes & celebrating success
Quality Improvement Framework: Our journey towards excellence
RBFT Improve: Approach
Improvement opportunity
Measure Plan RoI Outcomes Vital Behaviours
Evaluate
Ho
w…M
eth
odo
logy
… do we want to
improve?
…. do we generate
ideas?
… good are we and how do we know?
… do we make the changes?
… do we prioritise?
…timescale?
… do we demonstrate it’s worth it
…are things different
from before?
… will our behaviours support the
change
… will we know we
have made a difference, and how do
we keep improving?
Brainstorm
P&L
Process Map
Model for Improvement
Spaghetti Diagram
RAG Study
Benchmark
Audit
Historical analysis
Baselines
Lean
Pareto
Staff +Patient
Engagement
5 D’s
PDSA
PMO Docs
Human / Financial cost
and saving
SPC
Business Cases
Project Plan
Owners
Timescales
Rapid Improvemen
t Events
Six Sigma
Open to change
See the benefit or
bigger picture
Engagement
Review performance
Communicate change
Ensure sustainability
Celebrate Success
The Quality Improvement Approach: IMPROVE
Improvement opportunity
Measure Plan RoI Outcomes Vital Behaviours
Evaluate
Ho
w…M
eth
odo
logy
… do we want to
improve?
…. do we generate
ideas?
… good are we and how do we know?
… do we make the changes?
… do we prioritise?
…timescale?
… do we demonstrate it’s worth it
…are things different
from before?
… will our behaviours support the
change
… will we know we
have made a difference, and how do
we keep improving?
Brainstorm
P&L
Process Map
Model for Improvement
Spaghetti Diagram
RAG Study
Benchmark
Audit
Historical analysis
Baselines
Lean
Pareto
Staff +Patient
Engagement
5 D’s
PDSA
PMO Docs
Human / Financial cost
and saving
SPC
Business Cases
Project Plan
Owners
Timescales
Rapid Improvemen
t Events
Six Sigma
Open to change
See the benefit or
bigger picture
Engagement
Review performance
Communicate change
Ensure sustainability
Celebrate Success
The Quality Improvement Approach: IMPROVE
Doing things differently
• Clinical Leadership• Board/Executive support• Reducing bureaucracy• Interaction / workshops• Staff and patient involvement – all levels• Facilitated learning
Behavioural change
• Vision – they why• What do you need
to make it happen?
• How?
• DESIRABLE What’s in it for them
• ENABLE Provide info / skills
• STAKEHOLDERS Team & beyond
• INFLUENCE Supervisor / Seniors
• REWARDS Incentives
• ENVIRONMENT e.g. checklist, rota
Patient Leaders
• Recruitment based on values (12 to date)
• 7 day modular Patient Leader programme includes QI
• Supporting/leading Quality Improvement projects – working in collaboration
Quality Improvement Methodology
• Root Cause Analysis• Pareto• Driver diagrams• Importance of measurement: run charts• Model for Improvement / Clinical Audit• Process Mapping• Lean / Six Sigma
A model for learning and change
When you combine the 3 questions with the…
…the Model for Improvement
PDSA cycle, you get…
49
WHY are We doing it?
HOW we will do it
Move away from traditional clinical audit
Move from traditional audit cycles to real-time, dynamic improvement change audit
Importance of Measurement: Use of run charts
Perla R. BMJ Qual Saf 2011; 20: 46-51
The change seems to be associated with an improvement
The change is not associated with an improvement; if there had been no baseline measurement before making the change, the change mioght have been mistakenly interpreted as making a difference
The change seems to be associated with an improvement initially but the effect does not appear sustained
Remember measure little and often
What are we doing differently: Structure
Quality PerformanceCommittee
Care GroupClinical Governance
SpecialtyClinical Governance
DQORG
Trust ClinicalGovernance
Clinical Outcomes & Effectiveness group
NED: Janet Rutherford
DQORG = Data Quality Outcomes Review Group
What are we doing differently?
• Training/learning - basic• Model for improvement • Rapid audit cycles/PDSA• Importance of measurement
/ run charts• Driver diagrams
• 1:1 Coaching/support• Value of personal interaction
What are we doing differently?
•Junior doctor availability•Opportunities for completion•Time needed for completion
Primary Drivers
Secondary Drivers
Discharge decision
•Ward round times•Ward round structure•Consultant-led decision making•Elective admission- predictable•Emergency admission
•TTO completion•Opening times•Ward pharmacist•portering
•Information to GP•Mandatory•format•coding
EDL write up
Pharmacy
Content
AIM: Increase number of EDLs completed within 24 hours of discharge to 70% within 3 months on ward B to enable effective discharge planning
Topic selection
Successes: improvement in outcomes / sustainability
Week (starting October 2013)
IMPROVE: Hospital Acquired Pneumonia
HAP on 8 wards
0
2
4
6
8
10
12
14
16
18
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
Week
Numb
er of
patie
nts w
ith H
AP
Total on 8 wards Mean
Bed heads raised to 30 degrees at week 9 (17Dec) Oral hygiene
measures Victoria Ward week 25 (8Apr) Oral hygiene
measures across 8 wards week 29 (6May)
Trust w ide roll out w eek 43 (12 Aug)
Successes: improvement in outcomes / sustainability
0123456789
10HAP on 8 wards - progress chart
Num
ber
of p
atien
ts w
ith
HA
P
HAP prevention measures re- launched
Week (starting October 2013)
2012 Prevalence audits on 4 wards
012345
1 2 3 4
2013 HAP prevalence 8 wards
0123456789
10
1 2 3 4 5 6
Week
IMPROVE: Hospital Acquired Pneumonia
Trainees experience
“For the patient a QIP means an improvement in the quality of service they receive from the simplest of things to more complex issues all with the intention of improving patient experience and quality of life.” Anna Brown, CMT1
“This has been a very valuable learning experience into clinical quality improvement as well as being brilliant for my CV. The MEMC team have been supportive and encouraging throughout and there has always been someone available to talk to if I have ever needed any help.” Anna Weil, FY2
“My supervisor was very supportive and motivating throughout…I would definitely do a QIP again.”
Ram Jeeneea CT1
“It was very useful, and its good to know that you can really make a difference - I am now more aware of what we can do, and how to take action.”
Faraz Siddiqui FY1“You hear about projects and they sound really huge, but this has opened my eyes to how you can do little things and make small changes that make a big difference.”
Anna Roche FY2
“I am definitely empowered, and now have the understanding that I can make changes.”
Kunal Kulkarni FY1
“Overall it is very satisfying to engage with trainees and to supervise a QIP as they have an enthusiasm and motivation which is a real joy to work with. I very much enjoyed supervising a QIP project this year”. Maeve McKeogh, Consultant Supervisor
“The DVD produced as part of “Making Every Moment Count” will make an enormous difference to patients on the run up to their surgery, it will ease fears before surgery and calm nerves on the day of surgery”. Olivia Johnson, a patient involved in MEMC
Lessons learnt
• Need to change culture / mindsets• Make it personal • Lots of QI Methods: learning needs• Traditional clinical audit switches a lot of people off
• Use standards where they exist, you can make improvements without them
• Need greater focus on outcomes / return on investment at start of process
• Staff including Junior Doctors and Patient Leaders are real assets
Future
• Roll out Making Every Moment Count (junior doctor) programme to all Trust staff
• Continue focus on outcomes• Increase ability to assess Return on Investment• Sustainability • Increase facilitated learning • Share our learning
Scenarios
What do you need to do more of / less of?
CULTURE • In relation to clinical audit / quality improvement what is the culture in
my organisation?• What behaviours need to change? How to change? • Are we are learning organisation?• Do we have meaningful patient engagement/involvement?
STRUCTURE • Where is quality compromised by silo working?• Who are your clinical leaders?• How engaged are your Board/Executive? • How are we hearing the patient’s voice?
Scenarios
PROCESS • Is process geared around your customers/staff or the
clinical audit team?• Are people put off by your registration process? • How accessible is the clinical audit / QI team?OUTCOMES• Are outcomes measured / shared?• Is the impact of intervention on outcome known?• Is return on investment known/ shared?
Resources
Learning to make a difference : Making Every Moment Count PIlot
• https://www.youtube.com/watch?v=brZv_tftn_M • http://www.clinmed.rcpjournal.org/content/12/6/520.full• https://www.rcplondon.ac.uk/projects/learning-make-diff
erence-ltmd
• https://www.youtube.com/watch?v=sNKXOEPIe2Q&feature=youtu.be
Scenario 1 WHO checklist Audit
• Problem is surgeons are not completing sign out at the end of surgery
• Every theatre doing it differently• PLAN: To audit the WHO checks in order to
make it all better, please help• audit tool includes details of every step in
audit process
• Obvious problem, correct it• Create an SOP• Allow it all to bed in over a year
• Then audit against the SOP
SCENARIO 2Pain Service audit Conclusions/ action plan
• Authors Pain Nurse• 80 % of patients happy with everything• 55% of patients felt psychological needs not
met• ACTION PLAN includes• Appoint a Psychologist to the team part time