how should we establish the clinical case numbers required to achieve proficiency in flexible...

How should we establish the clinical case numbers required to achieve proficiency in flexible endoscopy?

Melina C. Vassiliou, MD, M.Ed, FRCSCBenjamin K Poulose MD, Pepa A Kaneva MSc, Brian J Dunkin MD, Jeffrey M Marks MD, Riadh Sadik MD, Gideon Sroka MD, Stephen D Pooler MD, Klaus Thaler MD, Gina L Adrales MD, Jeffrey W Hazey MD, Jenifer R Lightdale MD, Vic Velanovich MD ,Lee L. Swanstrom MD, John D Mellinger MD, Gerald M Fried MD

Flexible endoscopy is a necessary part of the general surgery curriculum

• Flexible endoscopy: important skill for GI & community surgeons

• Retrospective review of 5 surgeons: 54% of procedures were flex endo

• Survey of PD in 2000: 60% of programs have formal endoscopy rotations, only 33.3% by fellowship trained instructors

• Increased requirements for surgical trainees (35 EGDs and 50 colos)

1- Nimeri AA, Hussein SA, Panzeter E, et al. The economic impact of incorporating flexible endoscopy into a community general surgery practice. Surg Endosc 2005; 19(5):702-4.2- Marks JM, Nussbaum MS, Pritts TA, et al. Evaluation of endoscopic and laparoscopic training practices in surgical residency programs. Surg Endosc 2001; 15(9):1011-5

How many cases are needed to achieve proficiency?

• Case #’s as a surrogate for proficiency• ASGE - 130 EGDs & 140 colos (90% esophageal &

pyloric/splenic flex &cecum)• Surgical study: no correlation between #’s and

completion/complications• Another study – only 50 colonoscopies needed for

90% completion rate1. Cass OW, Freeman ML, Cohen J, et al. Acquisition of competency in endoscopic skills (ACES) during training: a multicenter study

[abstract]. Gastrointest Endosc 1995;41:3172. Reed WP, Kilkenny JW, Dias CE, Wexner SD. A prospective analysis of 3525 esophagogastroduodenoscopies performed by surgeons.

Surg Endosc 2004;18:11-21.3. Wexner SD, Garbus JE, Singh JJ. A prospective analysis of 13,580 colonoscopies. Reevaluation of credentialing guidelines. Surg Endosc

2001;15:251-61.

GAGESGlobal Assessment of Gastrointestinal

Endoscopic Skills• Created by expert

endoscopists• Multicenter study

demonstrated interrater reliability, internal consistency and construct validity

1- Intubation of the esophagus

2- Scope Navigation

3- Ability to keep a clear endoscopic field

4-Instrumentation

5- Quality of the Examination

1- Intubation of the esophagus

2- Scope Navigation

3- Ability to keep a clear endoscopic field

4-Instrumentation

5- Quality of the Examination

GAGES- Upper endoscopy

consists of 5 items scored on a

Likert scale

Interrater Reliability: 0.96 (0.90-0.99)

Internal Consistency: 0.89 (n=82)

1- Scope Navigation

2- Use of Strategies

3- Ability to keep a clear endoscopic field

4-Instrumentation

5- Quality of the Examination

1- Scope Navigation

2- Use of Strategies

3- Ability to keep a clear endoscopic field

4-Instrumentation

5- Quality of the Examination

GAGES- Colonoscopy consists of 5

items scored on Likert scale

Interrater Reliability: 0.97 (0.92-0.99)

Internal Consistency: 0.95 (n=57)

The purpose of this study was to:

• Challenge the current case number recommendations and methods by which proficiency in flexible endoscopy is determined

• Use GAGES to help define proficiency in flexible endoscopy

Methods

• IRB approved 11 institutions in Europe and NA• Demographic information• Participants from surgery and

gastroenterology• Scored by attending during routine upper

endoscopy and/or colonoscopy

Data Analysis

• For Upper endoscopy: 3 groups compared using ANOVA (Tukey post-hoc analysis) <35, >35<130, >130

• GAGES –C scores compared for different case cut-offs (T-test): >50 versus >140

• Scores plotted against case numbers to identify plateau

Results: The participants

139 evaluations, 11 centers

Demographic Percentage of total cohort

Dominant Hand 96% RightDiscipline 62 % surgeons; 38% GISex 79% male

GAGES upper endoscopy

GAGES Upper group Mean score ±SD

<35 n=35 14.4 ±3.7 NS

>35 & <130 n=22 17.8 ±1.8 P<0.05

>130 n=29 19.1 ±1.1 P<0.05

There is no difference between groups 2 and 3Both groups 2 and 3 are significantly different compared to group 1

1

2

3

Both groupings show statistically significant differences between novice and experienced

colonoscopists

GAGES -C Novice (95%CI) Experienced (95%CI) p-value

Novice <50 n=29 11.8 (10.3-13.2)

n=28 18.8 (18.3-19.3) p<0.001

Novice <140 n=32 12.4(10.9-14.0)

n=25 18.8 (18.8-19.3) p<0.001

NS NS

Scores plateau at ~ 50 cases for upper endoscopy

Upper Endoscopy Case numbers

Tota

l GAG

ES-U

pper

Sco

re

Scores seem to plateau at ~ 100 cases for colonoscopy

Colonoscopy Case numbers

Tota

l GAG

ES C

olon

osco

py S

core

Summary- Upper endoscopy

• For upper endoscopy, participants with 35-130 previous cases perform similarly to those with >130 cases

• Both of these groups perform better than those with less than 35 cases

• Performance as measured by GAGES seems to plateau at the 50 case level for upper endoscopy

Summary- Colonoscopy

• There was no difference in performance when the cut-off was set a 50 cases or at 140 cases

• We do not have enough data for the “intermediate” group

• Performance measured by GAGES plateaus at ~ 100 cases

Discussion & Limitations

• Still not enough data in the intermediate group

• We have not yet determined what the “passing score” for GAGES should be

• ROC – sensitivity and specificity• Ceiling effect

In Conclusion

• Current case recommendations may not represent what is needed for proficiency

• GAGES scores may help to define proficiency in basic flexible endoscopy

• Clinical numbers needed to achieve proficiency may vary from one learner to another

• GAGES may be a valuable tool to measure outcomes of training strategies and to provide feedback to learners

Acknowledgements:Members of the FES committeeLisa Jukelevics, Carla Bryant & Sarah ColonParticipants and contributors from all of the institutions