Making Audit
“ordinary”
And achieving extraordinary results!
Clinical Audit
“Clinical audit is a process that has been defined as "a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change".
(Wikipedia)
“Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria…Where indicated, changes are implemented…and further monitoring is used to confirm improvement in healthcare delivery.”
(Principles for Best Practice in Clinical Audit ,2002, NICE/CHI )
“Clinical audit is an independent review of the quality of clinical care against explicit criteria or recognised standards”
Audit means many things…
• Unit audit:
• A chat amongst friends
• Accreditation:
• A chat with new friends?
• Tax audit:
• Perhaps less friendly
The extremes of audit.
• Bedside audit
• Began in Queensland
• Now enshrined in the mechanisms of accreditation
• At its best it identifies opportunities for improvement
• Fits all the definitions that talk about a “process”
• “Chaps” audit
• Been around since Hippocrates
• Enshrined in the medical psyche
• At it’s best is a fantastic forum for peer learning
• And at its worst…
Clinical audit: a comprehensive review of the
literature
(The Centre for Clinical Governance Research in Health )
Limitations of clinical audit
•clarity and measurability of the criteria and standards chosen,
•quality of the data available,
•engagement of clinicians,
•involvement of consumers’
•skills and training of participants,
•time involved to undertake an audit,
•use of information technology,
•feedback provided,
•if and how the findings are translated into quality improvement strategies,
•evaluation of improvement strategies (closing the loop).
So what if audit was …
• Evidence based
• Targeted
• Designed & agreed by the clinicians responsible
• Automatic & IT-enabled
• Embedded in the organisation
• Unavoidable
• Designed by clinicians to drive improvement
• Designed to be self-sustaining
And here’s one I prepared earlier..
Target Zero
• Essentially a targeted improvement program
• Risk based
• Clinician driven
• Clinical audit embedded
• Specific
• Improvement is mandatory
• And innovation is inevitable
• AND a TRICK
A Trick?
• Our performance
• Not benchmarked – just not good enough!
• Of course Zero harm is unachievable
• But a 50% reduction in harm each year for just 5 years can’t be that hard can it?
• After all 96.875 % isn't zero!
• And of course we wont get there but imagine if we get 3 years of improvement!
Target Zero
• Extreme clinical risks only
• Executive responsible for the risk
• Performance reported monthly & directly to
• Executive Management Team
• Chief Executive
• Board of Directors
• Plan to reduce harm by 50% each year for 5 years
• A Risk Management Committee formed
Target Zero Risk Management
Committee.
• Executive chair
• Multidisciplinary & expert
• Evidence based:
• Performance measures – how we measured achievement
• Performance- what we should achieve
• Activities
Target Zero Risk Management
Committee.
• Program logic mapping exercise
• Identify the performance gap
• Identify inputs
• Describe what the end point looks like
• Describe how the end point will be “measured”
• Describe the activities and the short term outcomes along the way
PLM for deteriorating patient
The Evidence.
• What should our performance be as a minimum
• What should we be measuring
• Sensitivity / Specificity
• Process / Outcome
• Easily collected
• Reproducible
• What should we be doing
• Best practice
How did we make it work?
• Cornerstone of organisational strategic plan
• Therefore talked about by:
• Board of Directors
• Chief Executive
• Executive Management Team
• Monash Innovation & Quality Team
• Medical Directors
• Nursing & Site Directors
Talks Cheap
– and its not clinical audit
• Monthly reporting of performance measures
• Against a target for that year
• Either:
• Green = target achieved
• Amber = better than last year’s target
• Red = worse than last year’s target
• Reported to:
• Organisation
• Risk Management Committee
• Executive Management Committee
• Board Quality Committee
• Board of Directors
How did we embed Target Zero
• Quarterly report by Risk Management Committee (RMC) to executive
• Risk rating reviewed
• Actions taken
• Barriers identified by RMC
• Strength of controls
• Each risk reported activities ,achievements & barriers to Board Quality Committee yearly
How did we embed Target Zero -
downwards
• Critical part of Executive performance plan
• Therefore:
• Critical part of Medical & Nursing Directors performance plan
• Part of Unit Head & Nurse Unit Managers performance plan
• At least part of the conversation with clinicians
• Hawthorne effect
How did we embed Target Zero -
downwards
Unit Head M&M report
•Listing of all deaths and adverse events:
• Deaths
• MET calls, delayed METs, unplanned ICU admissions
• ISR 1 & 2 incidents incl. falls and medication errors
• Staph Aureus bacteraemia
• DVT/PE
•Taken from Riskman and clinical coding data
•Sent monthly via email direct to Unit Head
•Informs “Chaps” style of clinical audit
What did the reports look like?
Instant hits and perennial misses!
•DVT/ PE: an instant hit
•Deteriorating patients: an eventual hit
•Falls: perennial miss but actually a success story- but don’t tell anyone
What did the reports look like?
0.00
0.50
1.00
1.50
2.00
2.50
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
2009-10 2010-11 2011-12 2012-13 2013-14
Rate of preventable hospital acquired DVT/PE per 1000
separations
What did the reports look like?
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
Jan
Fe
b
Mar
Ap
r
May
Jun
Jul
Au
g
Se
p
Oct
No
v
De
c
2010 2011 2012 2013 2014
Rate of In-patient Cardiac Arrests
What did the reports look like?
Rate of falls resulting in serious injury (e.g. death, fracture
or head trauma) per 1000 bed days
What did we learn?
• We are not different from other health care organisations:
• We have the same risks
• These risks have the same causes
• These risks have the same solutions
• We are different from other health care organisations:
• The way a risk develops in each organisation varies
• The way solutions develop in each organisation varies
• The barriers to improvement vary between and within organsiations
Learning & learning organisations
and improvement
• Learning:
• the acquisition of knowledge or skills through study, experience, or being taught
• Learning organizations are skilled at five main activities:
• systematic problem solving,
• experimentation with new approaches,
• learning from their own experience and past history,
• learning from the experiences and best practices of others,
• transferring knowledge quickly and efficiently throughout the organization.
Triple Loop Learning: Chris Argyris
• What are we doing now
• Anything?
• Something?
• Can we do it better- Improvement
• Are we actually doing what we think we are- AUDIT
• How well are we doing what we do- AUDIT
• What else can we do- Innovation
• What haven’t we tried
• What hasn’t anyone tried
Another depiction attributed to
Argyris
• http://www.thorsten.org/wiki/index.php?title=Triple_Loop_Learning
Have we got a
problem?
Are we fixing it as well as
possible?
Is this the best solution?
What haven’t we thought of?
A learning view of Target Zero.
0
20
40
60
80
100
120
Year Zero
Year One
Year Two
Year Three
Year Four
Single Loop Learning
Double Loop Learning
(Anything is better)
( More is better)
No idea at all!
( Better is More )
Triple Loop Learning
Consider DVT/ PE(A single loop learning success story)
0
20
40
60
80
100
120
Year Zero
Year One
Year Two
Year ThreeYear Four
The Problem.
1. Ongoing measurement
2. No new actions or interventions
1. Org wide measurement of rate DVT/PE
& audit DVT prophylaxis
2. Standardised Risk assessment &
prophylaxis
3. Audits compliance
4. Case reviews of all in hospital DVT/PE
1. Continued measurement & auditing
compliance
2. National Medication Chart
3. SHIPP includes DVT/PE
4. Case reviews of all in hospital DVT/PE
Now Consider Detiorating patients
0
20
40
60
80
100
120
Year Zero
Year One
Year Two
Year ThreeYear Four
The Problem. Performance always amber or greenDouble loop learning from the start:
Missed MET callsLocal responsesEducation of ward staff
Single Loop Learning:MET introducedCall criteria agreed
World Best Practice!
What’s happening ?
Now consider FallsA double loop learning success/failure
0
20
40
60
80
100
120
Year Zero
Year One
Year Two
Year ThreeYear Four
The Problem. Steady decline in rate of falls but never of planned magnitude.Significant fluctuation in rate but difficult to attribute to any action of oursFocus on risk assessment, prevention strategies & assessment after fall has occurred.
Introduction of best practice prevention strategies& post falls Mx procedure.
Medical Engagement, Volunteer Observers
Approaching world
best rates
What did we do about falls?
• A process of constant revisiting:
• Why do people fall in hospital?
• What did we plan to do about that?
• Was this the best plan?
• How well was it implemented?
• How can we improve implementation
• What else can we do?
Making audit ordinary.
Why Target Zero worked
• Important
• Comparisons are odious & not allowed
• Performance measures
• Evidence based
• Clinician agreed
• Minimal effort
• Improvement unavoidable
• And there’s a trick within the trick!
• As it got harder everyone tried harder
• Then they tried different
Making Audit Ordinary
• Purposeful & worthy
• Clinician designed evidence based measures
• Easy to collect and reproducible data
• Questioning the data required not resented
• Simple improvement methodology:
• begins with PLM
• Clinicians chose what to improve
• IT enabled &distributed
• Process & outcome measures
• Owned by everryone