Assessing Performance and Assessing Performance and

Human Factors in Paediatric Human Factors in Paediatric

Cardiac SurgeryCardiac Surgery

Dr Ken CatchpoleDr Ken Catchpole

Quality, Reliability, Safety and Teamwork UnitQuality, Reliability, Safety and Teamwork Unit

Nuffield Department of Surgery, University of OxfordNuffield Department of Surgery, University of Oxford

Safety in National SystemsSafety in National Systems

No Harm, 9531,

75%

Severe, 181, 1% Death, 88, 1%

Moderate, 872, 7%

Low , 1977, 16%

Process, 1067, 8%

Clinical

assessment, 524,

4%

Documentation /

Consent, 2273,

18%

Medication, 1120,

9%

Other, 1018, 8%

Infrastructure /

Equipment, 2791,

22%

Treatement /

Procedure, 3856,

31%

12,649 NRLS incidents associated

with anaesthesia Jan 2004 – Feb 2006

“…“…..none of the reviewers none of the reviewers

have ever filled in such a have ever filled in such a

report, or knew of anyone report, or knew of anyone

who had.who had.””

Catchpole, Bell & Johnson. (2008). Anaesthesia 63, 340-346.

Safety at the Safety at the ““Coal FaceCoal Face””

Initiation of bypass without sufficient heparin is catastrophicInitiation of bypass without sufficient heparin is catastrophic

Hospital AHospital A–– Surgeon: Heparin pleaseSurgeon: Heparin please

–– Anaesthetist: Okay, heparinAnaesthetist: Okay, heparin

–– Anaesthetist: Heparin going inAnaesthetist: Heparin going in

–– Surgeon: Are we ready to go on bypass?Surgeon: Are we ready to go on bypass?

–– Anaesthetist: Yes, readyAnaesthetist: Yes, ready

–– Perfusionist: Yes, IPerfusionist: Yes, I’’m readym ready

Hospital B:Hospital B:–– Surgeon: Okay?Surgeon: Okay?

–– Anaesthetist: YesAnaesthetist: Yes

–– Surgeon: Alright thenSurgeon: Alright then

““ItIt’’s fine if you know how s fine if you know how wewe do it do it herehere..””

““About 6 months ago when we had About 6 months ago when we had

a bit of an incident with someone a bit of an incident with someone

new, but they werennew, but they weren’’t here long.t here long.”

No recent heparin incidents

System ThreatsOrganisation Environment Task Patient

Major Problem

Adverse Event

Minor Problem

Human Errors

Technical Non-Technical

What did we do?What did we do?

Two surgical environmentsTwo surgical environments–– Paediatric cardiac surgeryPaediatric cardiac surgery 24 cases24 cases

–– Orthopaedic surgeryOrthopaedic surgery 20 cases20 cases

InIn--theatre observations made in realtheatre observations made in real--timetime

Video recording made of each operationVideo recording made of each operation

MeasuresMeasures–– Minor failures Minor failures –– small process problemssmall process problems

–– Major failures Major failures –– events that came close to accidentsevents that came close to accidents

–– NonNon--technical skills technical skills –– cognitive and teamcognitive and team--working skillsworking skills

0 0.5 1 1.5 2 2.5 3 3.5 4

Decision-related surgical errorKnown problem

Fatigue

External pressuresPsychomotor-related perfusion error

Resource management

Fault resolutionPsychomotor Error (general)

Pre-operative diagnosis failureTemperature control difficulties

Planning failure

Expertise / skill failurePerfusion difficulties: technical

Team Conflict

Vigilance / awarenessProcedure-related Error

Cannulation difficultiesExternal resource failure

Psychomotor-related surgical error

Perfusion difficultiesDistraction

Equipment / Workspace managementUnintended effects on patient

Safety consciousness

Patient-sourced procedural difficultiesEquipment Configuration failure

Equipment failure

AbsenceCo-ordination / communication

Mean number per operation

24 Operations24 Operations

366 minor problems366 minor problems

29 different types29 different types

Minor Problem Types

PAEDIATRIC

CARDIAC

SURGERY

Catchpole, K, Giddings, A, De Leval,

M, Peek, G, Godden,

P, Utley, M, Gallivan,

S, Hirst, G, Dale, T (2006). Identification of

systems failures in

successful paediatric

cardiac surgery. Ergonomics 49(5-6),

pp.567-588.

Number of Minor Problems

Catchpole, K, Giddings, A, Wilkinson, M, Hirst, G, Dale, T, De Leval, M. (2007) Improving patient

safety by identifying latent failures in successful operations. Surgery 142(1), pp.102-110.

Sources of Minor FailureSources of Minor Failure

THREATS ERRORS

Catchpole, K, Giddings, A, Hirst, G, Dale, T, Peek, G, De Leval, M. (2008). A Method for Measuring Threats and Errors in Surgery. Cognition, Technology and Work 10(4), pp. 295-304.

0

5

10

15

20

25

30

35

40

Operations by Risk Level

Min

or

Fa

ilu

res

Ob

se

rve

d

Level 2 Level 3 Level 4 Level 6Level

1

"Low" Risk

Operations

"High" Risk

Operations

Minor Failures Per Operation (Paediatric Cardiac Surgery)

Significant difference between failures in “low” and “high” [u = 29.5, p<0.05]

0

5

10

15

20

25

30

35

40

Operations by Risk Level

Min

or

Fa

ilu

res

Ob

se

rve

d

Level 2 Level 3 Level 4 Level 6Level

1

"Low" Risk

Operations

"High" Risk

Operations

Major Failures

Minor Failures Per Operation (Paediatric Cardiac Surgery)

Major FailuresMajor Failures

Paediatric CardiacPaediatric Cardiac–– Swab causes compression of right coronary arterySwab causes compression of right coronary artery

–– ExEx--sanguination during postsanguination during post--bypass heamofilteringbypass heamofiltering

–– Omission of key surgical step Omission of key surgical step

–– Premature separation from bypass due to breakdown in teamworkPremature separation from bypass due to breakdown in teamwork

–– Aortic homograft ruptured during sternotomyAortic homograft ruptured during sternotomy

–– Incorrectly labeled homograftIncorrectly labeled homograft

–– Difficult management of activated clotting timeDifficult management of activated clotting time

OrthopaedicsOrthopaedics–– Multiple uncertainty leads to teamwork and task breakdown.Multiple uncertainty leads to teamwork and task breakdown.

Examples of minor failures implicated in major failure sequencesExamples of minor failures implicated in major failure sequences::

Communication/coCommunication/co--ordinationordination failures in 5 out of 8 major failuresfailures in 5 out of 8 major failures

AbsencesAbsences in 4 out of 8 major failuresin 4 out of 8 major failures

Equipment failuresEquipment failures in 4 out of 8 major failuresin 4 out of 8 major failures

Vigilance/awareness failuresVigilance/awareness failures in 3 out of 8 major failuresin 3 out of 8 major failures

ExEx--sanguination fortuitously sanguination fortuitously

mitigatedmitigatedContextContext

–– 3hrs 46minutes into a Norwood stage3hrs 46minutes into a Norwood stage--1 operation1 operation

–– reasonably eventreasonably event--free surgical progressfree surgical progress

–– just come off bypass with low oxygen saturations (approx 70%)just come off bypass with low oxygen saturations (approx 70%)

The BuildThe Build--UpUp

13:20 AC: Happy to come off whenever. S: Would you like an LA li13:20 AC: Happy to come off whenever. S: Would you like an LA line? AC: helpful, ne? AC: helpful, butbut……..

13:22 S: Are you happy or not? AC: yes, I13:22 S: Are you happy or not? AC: yes, I’’m happym happy

13:25 S: Would you like to come off bypass. AC: Yes. Perf: Yes 13:25 S: Would you like to come off bypass. AC: Yes. Perf: Yes –– just a minute (P just a minute (P coaches trainee P).coaches trainee P).

13:26 P: Off. S: trickle on hotline. R: Just a second13:26 P: Off. S: trickle on hotline. R: Just a second…….hotline trickling. S: Give 10. .hotline trickling. S: Give 10. P:yep. Can we Teg venous, please? S: Give 10. AC: are you givingP:yep. Can we Teg venous, please? S: Give 10. AC: are you giving 10? P: Yep, 10? P: Yep, going in. S: Another 10 (perf reads back).going in. S: Another 10 (perf reads back).

13:28 S: Another 10. S: Give 10. P:yep13:28 S: Another 10. S: Give 10. P:yep

13:29 S: Give 10. P:10 S:Give 10. P:1013:29 S: Give 10. P:10 S:Give 10. P:10

13:30 Ventilation starts. S: Give 10. P: 1013:30 Ventilation starts. S: Give 10. P: 10

13:32 S: Give 10. P: 10.13:32 S: Give 10. P: 10.

13:33 S: where is arterial line? AC: femoral13:33 S: where is arterial line? AC: femoral

13:34 AC operates anaesthetic workstation. S: You13:34 AC operates anaesthetic workstation. S: You’’re not giving. P: Should be re not giving. P: Should be improved. AC: we changed the scale.improved. AC: we changed the scale.

13:35 Discussion of desired haematocrit levels13:35 Discussion of desired haematocrit levels

13:36 AC (to perf): don13:36 AC (to perf): don’’t let the green RAP go too high. Probably want it at 14.t let the green RAP go too high. Probably want it at 14.

13:37 S: Give 10.13:37 S: Give 10.

Unclear communication

Trainee upsets ‘surgical rhythm’

Perfusionist & surgeon out

of step = poor co-ordination

Misleading equipment leads

to poor co-ordination

6 different types of equipment

Equipment / team / process coupling

The EventThe Event

13:39 P: Filtration stopped. AC: What13:39 P: Filtration stopped. AC: What’’s the crit? P: 40. AC (to P): I s the crit? P: 40. AC (to P): I think we ought to continue + discussion of new plan. Meanwhile, think we ought to continue + discussion of new plan. Meanwhile, surgeon takes the MUF line out. 1A is involved in planning, but surgeon takes the MUF line out. 1A is involved in planning, but thinks the agreement is to concentrate the blood in the pump.thinks the agreement is to concentrate the blood in the pump.

13:40 1A: got a gas? AC: reads out bloodgas13:40 1A: got a gas? AC: reads out bloodgas

13:41 Surgeon asks for more calcium.13:41 Surgeon asks for more calcium.

S: I took out the MUF line. P: WeS: I took out the MUF line. P: We’’ve started filtering again. S: Ive started filtering again. S: I’’m m glad I said something. How much volume did you take out? P: Not glad I said something. How much volume did you take out? P: Not a a lot.lot.

13:42 MUF line replaced. P: MUFfing again. S: Give 10. S: Give 13:42 MUF line replaced. P: MUFfing again. S: Give 10. S: Give another 10.another 10.

13:43 P & AC make new filtering plan.13:43 P & AC make new filtering plan.

New plan not clearly

communicatedTask conflict; attention elsewhere

Early Mitigation

New plan co-ordinated

Fortuitous co-ordination

Error goes unnoticed for >120s

Major Failure 2: Surgical OmissionMajor Failure 2: Surgical Omission

CONTEXTCONTEXT

Trainee perfusionist overseen by senior perfusionistTrainee perfusionist overseen by senior perfusionist

Light handle cover omittedLight handle cover omitted

Infusion pumps not plugged in until 80 mins after 1Infusion pumps not plugged in until 80 mins after 1stst incisionincision

Anaesthetic consultant periodically absent, leading to:Anaesthetic consultant periodically absent, leading to:–– Inappropriate support to registrarInappropriate support to registrar

–– 11:5411:54--12:31 Poor starting and heparin communication protocols12:31 Poor starting and heparin communication protocols

–– 12:36 Registrar can't operate ACT machine properly (thinks it is12:36 Registrar can't operate ACT machine properly (thinks it isbroken), leading to 3 minute delay in obtaining ACTbroken), leading to 3 minute delay in obtaining ACT

12:54 Undiagnosed intramural coronary pattern found12:54 Undiagnosed intramural coronary pattern found

13:23 Perfusion air lock13:23 Perfusion air lock–– Sucking on red vent allows air to travels across septal defect; Sucking on red vent allows air to travels across septal defect; surgical surgical

field fills up; Senior perfusionist sorts out problemfield fills up; Senior perfusionist sorts out problem

14:27 Rewarming starts14:27 Rewarming starts

14:31 Surgeon finishes construction of the neo14:31 Surgeon finishes construction of the neo--aorta. x clamp is aorta. x clamp is removed unannounced.removed unannounced.

14:34 External registrar talks to surgeon about the case14:34 External registrar talks to surgeon about the case

14:36 Anaesthetic consultant returns after absence.14:36 Anaesthetic consultant returns after absence.

14:37 Heart begins to beat14:37 Heart begins to beat

14:38 Surgeon realises he14:38 Surgeon realises he’’s forgotten to close the septostomy.s forgotten to close the septostomy.

14:39 Rewarm halted14:39 Rewarm halted

14:40 Venous return re14:40 Venous return re--configuredconfigured

14:48 Septostomy closed14:48 Septostomy closed

The EventThe EventViolation of normal procedure

Distraction

Unaware septostomy is not closed

Mitigation Plan

Error identified

Major Failure 3: Compression of Major Failure 3: Compression of

Coronary Artery in ASOCoronary Artery in ASOCONTEXT (POST BYPASS)CONTEXT (POST BYPASS)

LeCompte manoeuvre fails due to patient anatomy, requiring alterLeCompte manoeuvre fails due to patient anatomy, requiring alternative native strategystrategy

–– MPA transected at bifurcationMPA transected at bifurcation

Continued postContinued post--bypass bleeding due to surgerybypass bleeding due to surgery–– Compounded by lack of blood productsCompounded by lack of blood products

–– SubSub--optimal ACT protocolsoptimal ACT protocolsMissed communications between AC & perfusionist mean sample is lMissed communications between AC & perfusionist mean sample is left in uncovered eft in uncovered syringe for 3 minutes before being placed in machinesyringe for 3 minutes before being placed in machine

Lungs wet postLungs wet post--bypassbypass–– AC balances SATs & lung stiffness by alternating between automatAC balances SATs & lung stiffness by alternating between automatic & bag ic & bag

ventilationventilation

Difficulty visualising repair with echocardiogramDifficulty visualising repair with echocardiogram–– ToE not possible; control of ventilation is difficultToE not possible; control of ventilation is difficult

Temperature overshootTemperature overshoot

Next patient is preNext patient is pre--medded, then has to be remedded, then has to be re--scheduledscheduled–– Anaesthetist asks ward to preAnaesthetist asks ward to pre--med the next patient before bleeding is apparentmed the next patient before bleeding is apparent

Frequent changes in surgeons at the tableFrequent changes in surgeons at the table–– 11stst Assistant surgeon leaves table at 13:33 & returns at 14:09Assistant surgeon leaves table at 13:33 & returns at 14:09

–– Consultant Surgeon leaves table at 14:06 & returns at 15:07Consultant Surgeon leaves table at 14:06 & returns at 15:07

–– 22ndnd Assistant Surgeon leaves table at 14:35 & returns at 14:49Assistant Surgeon leaves table at 14:35 & returns at 14:49

The EventsThe Events14:0014:00--15:00 Considerable number of swabs/clotting aids used15:00 Considerable number of swabs/clotting aids used

15:0315:03 Heart has swelled and is beginning to look very purple. SATS atHeart has swelled and is beginning to look very purple. SATS at 78; ETCO at 3.978; ETCO at 3.9

1A discusses hypoxia1A discusses hypoxia

S (not scrubbed): maybe we should check another echoS (not scrubbed): maybe we should check another echo

1A: that's happened in the last 3 or so minutes1A: that's happened in the last 3 or so minutes

S: lost sinus rhythm as well.S: lost sinus rhythm as well.

15:0415:04 S: back in sinus rhythm. What's happening?S: back in sinus rhythm. What's happening?

1A: the right ventricle isn't moving1A: the right ventricle isn't moving

S: what's happening?S: what's happening?

S goes to scrub up again.S goes to scrub up again.

AC: we need some pacing. Pacing box is connectedAC: we need some pacing. Pacing box is connected

15:0515:05 AC: Atrial flutter. Let's try a transAC: Atrial flutter. Let's try a trans--atrial shockatrial shock

1A: Okay, lets try to defibrilate1A: Okay, lets try to defibrilate

AC prepares the deAC prepares the de--fib;fib;

1A reaches for the paddles; the headlight generator is in the wa1A reaches for the paddles; the headlight generator is in the way, and there is a sheet y, and there is a sheet covering the paddles. 1A: "Is that clean?" SN nods.covering the paddles. 1A: "Is that clean?" SN nods.

Heart is shockedHeart is shocked

15:0615:06 1A inspects surgical site1A inspects surgical site

S: there's got to be a reason for it. We've got to find out.S: there's got to be a reason for it. We've got to find out.

AC: Has somebody called for an echo? Will somebody call?. The ruAC: Has somebody called for an echo? Will somebody call?. The runner goes.nner goes.

15:0715:07 1A still inspecting surgical sites1A still inspecting surgical sites

S arrives at table S arrives at table -- takes the 1A positiontakes the 1A position

15:07 15:07 -- 15:1015:10 1A and AC in diagnostic discussions & measurements1A and AC in diagnostic discussions & measurements

Triggering event unclearTriggering event unclear

Obvious problemObvious problem

Immediate responsesImmediate responses

Lowered SALowered SA

Workspace management failureWorkspace management failure

Fails to see problemFails to see problem

DiagnosticsDiagnostics

Further initial responsesFurther initial responses

15:1115:11 AC: if we get the sats up, everything will be better.AC: if we get the sats up, everything will be better.

1A: ETCO's way off1A: ETCO's way off

AC: now blood flowAC: now blood flow

S: we've had no problems with the right coronaryS: we've had no problems with the right coronary

1A: I put a stitch on the <unintelligible>1A: I put a stitch on the <unintelligible>

S: that could be it.S: that could be it.

15:1215:12 1A & S discuss possibility that it's 1A's stitch1A & S discuss possibility that it's 1A's stitch

Echo machine arrivesEcho machine arrives

1A investigates area of R coronary with suction1A investigates area of R coronary with suction

AC: have you got some heparin ready? (To AR): can you get some hAC: have you got some heparin ready? (To AR): can you get some heparin eparin ready?ready?

15:1215:12 S removes small section of mastoid strip from area of R coronaryS removes small section of mastoid strip from area of R coronary

Sats & ETCO riseSats & ETCO rise

AC: that looks betterAC: that looks better

15:1315:13 Realisation & reliefRealisation & relief

15:2015:20 Discussion of the errorDiscussion of the error

Fails to see problem againFails to see problem again

Diagnosis moving closerDiagnosis moving closer

Problem identifiedProblem identified

Confirms the solutionConfirms the solution

Review events Review events but not how it but not how it

could have been avoidedcould have been avoided

The EventsThe Events

Teamwork in the Cardiac Operating Teamwork in the Cardiac Operating

TheatreTheatre

S

1A

SN

P

ACR

Perfusion

HLM

Anaesthetic

Workstation

2A

AR

Pumps & Drips

Surgical NOTECHSDimensions Elements

Leadership & Management LeadershipMaintenance of StandardsPlanning & PreparationWorkload ManagementAuthority & Assertiveness

Teamwork & Co-operation Team building & MaintainingSupport of othersUnderstanding team needsConflict solving

Problem Solving & Decision Making Definition & DiagnosisOption GenerationRisk AssessmentOutcome Review

Situation Awareness NoticeUnderstandThink Ahead

Below Standard (1) Basic Standard (2) Standard (3) Exceed(4)

Behaviour directly compromises patient safety and effective teamwork.

Behaviour in other conditions could directly compromise patient safety and effective teamwork.

Behaviour maintains an effective level of patient safety and teamwork.

Behaviour enhances patient safety and teamwork. A model for all other teams.

Failures and NonFailures and Non--Technical SkillsTechnical Skills

(Paediatric Cardiac Surgery)(Paediatric Cardiac Surgery)

Most Effective Least Effective

Spearman’s Rho = 0.738, n=24, p<0.001

Non-technical Errors

““EscalationEscalation”” MechanismMechanism

No

n-T

ech

nic

al S

kills

Minor Problems

Operating

Errors

Major

Problems

Risk

0.29 0.04

Paediatric

Cardiac SurgeryOrthopaedic

Surgery

9.5 13.1

5.75 4.89

Mean Per Operation

N=24 N=18

Type Risk

Risk

• Leadership &

Management

• Teamwork &

Co-ordination

• Problem

Solving and

Decision

Making

• Situation

Awareness

Catchpole, K, Giddings, A, Wilkinson, M, Hirst, G, Dale, T, De Leval, M. (2007) Improving patient

safety by identifying latent failures in successful operations. Surgery 142(1), pp.102-110.