pilot validation study of the european association of ... · of trainees, shorten their learning...
TRANSCRIPT
Platinum Priority – EducationEditorial by Jens-Uwe Stolzenburg, Hasan A.R. Qazi and Bhavan Prasad Rai on pp. 300–301 of this issue
Pilot Validation Study of the European Association of Urology
Robotic Training Curriculum
Alessandro Volpe a,b, Kamran Ahmed c, Prokar Dasgupta c, Vincenzo Ficarra a,d,Giacomo Novara a,e, Henk van der Poel f, Alexandre Mottrie a,g,*
a OLV Vattikuti Robotic Surgery Institute, Aalst, Belgium; b Division of Urology, University of Eastern Piedmont, Novara, Italy; c MRC Centre for
Transplantation, Kings College London, Guy’s Hospital, London, UK; d Department of Experimental and Clinical Medical Sciences, Division of Urology,
University of Udine, Udine, Italy; e Department of Surgical, Oncological and Gastrointestinal Sciences, Division of Urology, University of Padua, Padua, Italy;f Division of Urology, The Netherlands Cancer Institute, Amsterdam, The Netherlands; g Division of Urology, Onze-Lieve-Vrouw Hospital, Aalst, Belgium
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9
avai lable at www.sciencedirect .com
journal homepage: www.europeanurology.com
Article info
Article history:
Accepted October 14, 2014
Keywords:
Curriculum
Radical prostatectomy
Robotics
Simulation
Training
Abstract
Background: The development of structured and validated training curricula is one of the current
priorities in robot-assisted urological surgery.Objective: To establish the feasibility, acceptability, face validity, and educational impact of a
structured training curriculum for robot-assisted radical prostatectomy (RARP), and to assess
improvements in performance and ability to perform RARP after completion of the curriculum.Design, setting, and participants: A 12-wk training curriculum was developed based on an expert
panel discussion and used to train ten fellows from major European teaching institutions. The
curriculum included: (1) e-learning, (2) 1 wk of structured simulation-based training (virtual reality
synthetic, animal, and cadaveric platforms), and (3) supervised modular training for RARP.Outcome measurements and statistical analysis: The feasibility, acceptability, face validity, and
educational impact were assessed using quantitative surveys. Improvement in the technical skills of
participants over the training period was evaluated using the inbuilt validated assessment metrics on
the da Vinci surgical simulator (dVSS). A final RARP performed by fellows on completion of their
training was assessed using the Global Evaluative Assessment of Robotic Skills (GEARS) score and
generic and procedure-specific scoring criteria.Results and limitations: The median baseline experience of participants as console surgeon was 4 mo
(interquartile range [IQR] 0–6.5 mo). All participants completed the curriculum and were involved in a
median of 18 RARPs (IQR 14–36) during modular training. The overall score for dVSS tasks significantly
increased over the training period ( p < 0.001-0.005). At the end of the curriculum, eight fellows (80%)
were deemed able by their mentors to perform a RARP independently, safely, and effectively. At
assessment of the final RARP, the participants achieved an average score�4 (scale 1–5) for all domains
using the GEARS scale and an average score >10 (scale 4–16) for all procedural steps using a generic
dedicated scoring tool. In performance comparison using this scoring tool, the experts significantly
outperformed the fellows (mean score for all steps 13.6 vs 11).Conclusions: The European robot-assisted urologic training curriculum is acceptable, valid, and
effective for training in RARP.Patient summary: This study shows that a 12-wk structured training program including simulation-
based training and mentored training in the operating room allows surgeons with limited robotic
experience to increase their robotic skills and their ability to perform the surgical steps of robot-
assisted radical prostatectomy.
# 2014 European Association of Urology. Published by Elsevier B.V. All rights reserved.
* Corresponding author. OLV Vattikuti Robotic Surgery Institute, Department of Urology,Onze-Lieve-Vrouw Hospital, Moorselbaan 164, 9300 Aalst, Belgium. Tel. +32 53 724378;Fax: +32 53 724411.E-mail address: [email protected] (A. Mottrie).
http://dx.doi.org/10.1016/j.eururo.2014.10.0250302-2838/# 2014 European Association of Urology. Published by Elsevier B.V. All rights reserved.
1. Introduction
The concept of surgical training has been evolving in the last
decade from the traditional concept of ‘‘see one, do one, teach
one’’ towards better defined and standardized methodolo-
gies for surgical education based on the development of skill-
based curricula [1–5]. Furthermore, the development and
diffusion of surgical robotic platforms are increasingly
supporting the development, use, and validation of simula-
tion-based training methods ranging from bench-top syn-
thetic models, animal, and cadavers to high-fidelity virtual
training platforms [6–8]. Simulation-based training should
be an essential part of surgical training programs to
significantly improve the technical and nontechnical skills
of trainees, shorten their learning curves for different
procedures, and improve surgical safety [9,10].
Nevertheless, training for robotic techniques remains
mainly unstructured. There has been a recent call by various
training bodies for the development of well-organized
educational curricula to increase preclinical exposure and of
validated assessment tools that allow constructive feedback
for performance improvement. These curricula, as well as
proficiency-based credential processes, are important for
improving patient safety and surgical outcomes in urologi-
cal surgery [5,11].
On the basis of these considerations, the European
Association of Urology (EAU) Robotic Urologic Section
(ERUS) has designed and developed a structured training
program and curriculum in urology that focuses on robot-
assisted radical prostatectomy (RARP). The aim of the
present study was to assess the feasibility, acceptability,
face validity, and educational impact of this curriculum, and
to assess improvements in performance and ability to
perform RARP after completion of the curriculum.
2. Materials and methods
2.1. Study design and participants
This was a longitudinal prospective study using quantitative observa-
tional measures. The participants were ten international fellows training
in robotic surgery provided by major teaching European institutions
under the recommendation of an expert mentor.
2.2. Curriculum
The curriculum was developed based on an expert panel discussion [12]
and was used for training of fellows. The key components of the curriculum
include: (1) e-learning, (2) an intensive week of structured, simulation-
based training (virtual reality synthetic, animal, and cadaveric platforms),
and (3) supervised modular training in RARP (Fig. 1).
2.3. Process
The overall study duration was 12 wk. After evaluation of baseline
experience, the fellows underwent e-learning using the e-module
developed by the ERUS expert panel [13] and observed and assisted in
live surgery for 3 wk. The participants then underwent an intensive week
of structured, simulation-based laboratory training including virtual
reality simulation (da Vinci surgical simulator, dVSS) and dry and wet
laboratory simulation platforms (synthetic, animal, and cadavers)
(Supplementary Table 1). Following this, the fellows participated into
a modular training program under supervision, which involved
progressive, proficiency-based training through surgical steps with
increasing levels of complexity (Supplementary Table 2) [14]. The
fellows continued the training until they carried out a full RARP
procedure, which was assessed by their mentors and video-recorded for
review and evaluation of performance by blind assessors.
2.4. Study outcomes
The outcomes of interest were (1) the feasibility, acceptability, face
validity, and educational impact of the curriculum [15] and (2)
improvements in performance and ability to perform RARP following
completion of the curriculum. Face validity is the extent to which the
learning and assessment environment resembles the situation in the real
world [15].
2.5. Evaluation of outcome parameters
Feasibility, acceptability, face validity, and educational impact were
assessed using quantitative surveys that were developed and validated
according to the expert opinions of robotic surgeons.
The technical skills of the participants were assessed via inbuilt
validated assessment metrics on the dVSS. The specific skills included[(Fig._1)TD$FIG]
BASELINE EVALUATION
F
E
D
C
B
A
E-learning module Operating room observation(bedside-console)
SIMULATION-BASED TRAINING(1-wk intensive course)
Virtual realitysimulation
Dry lab Wet lab
MODULAR CONSOLE TRAINING
TRANSITION TO FULL PROCEDURAL TRAINING(Video recording of a full case of RARP)
FINAL EVALUATION
Fig. 1 – Structure of the European Association of Urology Robotic Training Curriculum.
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9 293
moving the camera and clutching, manipulating the endowrist, use of
energy and dissection, and needle driving. The score at baseline and on
final assessment were compared to determine the improvement in basic
robotic skills.
Following successful completion of the modular training, the
mentors evaluated the procedural skills of the fellows in performing
RARP using the validated Global Evaluative Assessment of Robotic Skills
(GEARS) score (Supplementary Table 3) [16]. The mentors also assessed
the quality of the surgical skills for each surgical step using a RARP
procedure-specific scoring scale (Supplementary Table 4) ranging from
1 to 5, for which �3 was considered safe.
Videos of the final RARP procedures performed by each fellow were
further assessed by blinded, expert robotic surgeons using a generic
dedicated scoring criterion for each procedural step (Supplementary
Table 5). This score ranged from 4 to 16, and�10 was considered safe. The
videos of each surgical step were assessed by the same two independent
reviewers for all participants. The scores obtained by the fellows were
compared with those assigned by the same blinded reviewers to the
performance of two expert robotic surgeons to establish the construct
validity of the assessment. Construct validity is the extent to which a test is
able to discriminate between various levels of expertise [15].
2.6. Statistical analysis
Descriptive statistics were performed for the available variables.
Categorical variables are reported as frequency and percentage, and
continuous variables as median and interquartile range (IQR) or mean and
standard deviation, as appropriate. Mean values among groups were
compared using the Student t test and analysis of variance, as appropriate.
Statistical significance was set at p � 0.05. The statistical analysis was
carried out using SPSS version 20 (IBM Corp., Armonk, NY, USA).
3. Results
The characteristics and previous robotic experience of the
participants are reported in Table 1. Most participants had
minimal or no previous experience of simulation-based
training. The median times of involvement as a table
assistant and a console surgeon at baseline were 9.5 mo
(IQR 5.75–16 mo) and 4 mo (IQR 0–6.5 mo), respectively.
All participants completed the required e-learning
module and passed the final test for assessment of
theoretical knowledge. All fellows observed and assisted
in the recommended number of procedures (>12 cases)
during the first 3 wk of the curriculum, participated
successfully in all activities during the intensive week of
laboratory training, and completed the 8 wk of supervised
modular training. The median number of RARPs they were
involved in as console surgeons during modular training
was 18 (IQR 14–36).
The overall scores obtained by participants for perfor-
mance of four different dVSS tasks at baseline, during
the training program (weeks 4 and 5), and at the end of the
curriculum are reported in Figure 2. For all exercises the[(Fig._2)TD$FIG]
0 4 5 12
0
20
40
60
80
100
Camera and Clutching - Ring walk 3
100
90
80
70
60
0 4 5 12Training week
Training week
Training week
Ove
rall
scor
e (%
)
Ove
rall
scor
e (%
)
Ove
rall
scor
e (%
)
Ove
rall
scor
e (%
)
Training week
100
90
80
70
60
100
90
80
70
60
0 4 5 12 0 4 5 12
Needle driving - Suture sponge 2Energy and dissection - Energy switch 2
Endowrist manipulation - Match board 2
Fig. 2 – Progressive improvement in overall scores for different tasks on the da Vinci surgical simulator before, during (weeks 4 and 5), and aftercompletion of the curriculum. * Significant difference compared to overall score before the curriculum ( p < 0.05).8 Significant difference compared tooverall score in week 4 ( p < 0.05).
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9294
overall score significantly increased over the training period
( p < 0.001–0.005).
The GEARS scale was used by mentors to evaluate the final
RARP performance of fellows (Fig. 3A). Good to excellent
scores were obtained by 80–100% of trainees for their
depth perception (median 5, IQR 4–5), bimanual dexterity
(median 4.5, IQR 4–5), efficiency (median 4, IQR 3.75–5),
force sensitivity (median 4, IQR 4–5), autonomy (median 4,
IQR 3.75–5), and robotic control (median 5, IQR 3.75–5) skills.
A procedure-specific scale was used to score the performance
of fellows for each RARP surgical step (Fig. 3B).
Eight trainees (80%) were considered by their mentors
able to perform a RARP independently, safely, and efficiently
on completion of the curriculum. Three (30%) were consid-
ered able to perform a complex RARP independently, safely,
and effectively.
The blinded video-based assessment results for RARP
steps performed by each fellow and two robotic experts
are shown in Table 2. The fellows achieved an average score
that was considered safe (�10) for all surgical steps. The
highest average scores were obtained for bladder detach-
ment and urethrovesical anastomosis. When the scores for
all procedural steps were assessed, eight fellows (80%)
reached an average score�10. The two participants who did
not reach an average sufficient score were residents. The
robotic experts significantly outperformed the fellows
overall (mean score 13.6 vs 11) and for all RARP steps
except bladder detachment and endopelvic fascia incision,
confirming the construct validity of the assessment (Fig. 4).
Table 2 – Results for blinded video-based assessment of individual procedural steps in robot-assisted radical prostatectomy performed byfellows and robotic experts according to a generic dedicated scoring scale ranging from 4 to 16, with I10 considered safe
Bladderdetachment
Endopelvicfascia incision
Ligation ofdorsal vein
complex
Bladder neckincision
Dissectionof vasa and
seminalvesicles
Preparationand sectionof prostatic
pedicles
Dissection ofneurovascular
bundles
Apicaldissection
Urethrovesicalanastomosis
Mean
Fellow A 11.5 11 10.5 10.5 9.5 8.5 9.5 7 9.5 9.7
Fellow B 11 11 12 12 9.5 12 10.5 12 10 11.1
Fellow C 12 12.5 NP 12 10 12 11 11 13.5 11.8
Fellow D 12 12.5 8.5 11 8 8 NP 10.5 14.5 10.6
Fellow E 12 10 11 10 11 12 12 10 12.5 11.2
Fellow F 12 12 10 10 13.5 12 12.5 10 14 11.8
Fellow G NP 10 NP 9.5 6 8 6.5 6 14.5 8.6
Fellow H 11.5 12 12 10.5 10.5 10 9.5 9 12 10.8
Fellow I 12.5 12.5 10.5 12 13.5 12.5 13 14 11 12.3
Fellow L 12 12.5 NP 12 12.5 10 14 10.5 12 11.9
Expert A 13.5 12.5 NP 13 12 13.5 13.5 13 14.5 13.2
Expert B NP 12 15 14 14 13.5 13.5 14 15.5 13.9
NP = not performed.
Table 1 – Baseline characteristics and robotic experience ofparticipants
Median age, yr (IQR) 35 (31–36)
Training stage, n (%)
Resident 3 (30)
Fellow 5 (50)
Staff 2 (20)
Involvement in robotic surgery as table assistant
Median time, mo (IQR) 9.5 (5.75–16)
Median cases, n (IQR) 50 (29.5–175)
Involvement in robotic surgery as console surgeon
Median time, mo (IQR) 4 (0–6.5)
Median cases, n (IQR) 12 (0–24)
Median experience of virtual reality simulation, h (IQR) 0.5 (0–6.5)
Median experience of dry laboratory, h (IQR) 0 (0–8)
Median experience of wet laboratory, h (IQR) 0 –
IQR = interquartile range.
[(Fig._3)TD$FIG]
Fig. 3 – Mentor assessment of robot-assisted radical prostatectomy performance at the end of the European robot-assisted urologic training curriculumusing (A) the GEARS scale and (B) a procedure-specific scale.
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9 295
6.0
Fellows Fellows
Fellows Fellows
FellowsFellowsFellows
6.0
8.0
9.0
10.0
11.0
12.0
13.0
14.0
Bladder neck incision Dissection of vasa / seminal vesicles
10.0
12.0
14.0
Sco
re
Sco
reS
core
Sco
re
Sco
re
10.0
10.5
11.0
11.5
12.0
12.5
8.0
9.0
10.0
11.0
12.0
13.0
14.0
Overall procedure Ligation of dorsal vein complex
Sco
re
Sco
re
Sco
re
Sco
reFellows
6.0
8.0
10.0
12.0
14.0
Apical dissection
10.0
12.0
14.0
16.0
Urethrovesical anastomosis
8.0
9.0
10.0
11.0
12.0
13.0
14.0
Preparation and section of prostatic pedicles
Fellows
Experts Experts Experts
ExpertsExpertsExperts
ExpertsExpertsExperts
Dissection of neurovascular bundles
8.0
10.0
12.0
14.0
Endopelvic fascia incision
8.0
10.0
12.0
14.0
Fig. 4 – Comparison of robot-assisted radical prostatectomy performance between fellows and experts using a generic dedicated scoring scale (range 4–16).
EU
RO
PE
AN
UR
OL
OG
Y6
8(
20
15
)2
92
–2
99
29
6
The results for quantitative surveys completed at the end
of curriculum are reported in Figure 5. All fellows gave an
excellent overall evaluation of the training program and felt
that the curriculum was very effective in improving their
console exposure, their basic robotic skills, and their ability
to perform RARP.
4. Discussion
This is the first study that incorporates and validates
different components of a training curriculum for robot-
assisted surgery at a multi-institutional level. The study
demonstrates that a 12-wk structured training program
including theoretical e-learning, laboratory training, and
modular training in the operating room is feasible, accept-
able, and effective in improving the technical robotic skills
and ability of young surgeons with limited previous robotic
experience to perform the surgical steps of RARP. The study
also demonstrates the face validity of this curriculum.
In the last few years there has been growing interest in
the field of surgical education, especially in minimally
invasive laparoscopic and robotic surgery [3,4,9,10]. How-
ever, curricula for training in robotic urologic procedures
have not yet been standardized and the optimal integration
of simulation-based training in surgical training programs
is not clearly defined or evidence-based. The definition of
curricula for each surgical procedure and their validation
and progressive implementation are important for accredi-
tation of surgeons and teams for each specific intervention,
with the ultimate aim of improving surgical safety and
patient outcomes [5,11].
To date, three basic curricula for training and assessment
of robotic surgeons have been developed and reported in
the literature [3,4,17]. It has been shown that they are valid,
feasible, and effective in significantly improving basic
robotic surgery skills. However, these curricula do not
include modular training in the operating room and they
have not been validated for specific procedures.
On the basis of these considerations, the ERUS board
members designed and proposed a curriculum for RARP
including simulation-based and modular console training
with an expert mentor. The curriculum content was finalized
in accordance with the expert opinion of robotic surgeons
across various regions to ensure content validation [12].
The program is a step forward from existing approaches
because it includes all the simulation training modalities
available (virtual reality simulation, bench-top models, live
animal surgery, and cadaveric procedures) combined with
clinical training in the form of a mini-fellowship. Fellowships
are actually considered a key component of training for
complex urologic procedures such as RARP [18].
An ideal training program should be feasible, acceptable,
valid, and economically sustainable, with an effective
educational impact [15]. Our results show that the EAU
robotic training curriculum is feasible and acceptable, as all
participants were very satisfied with the program and would
highly recommend it to other colleagues. All modules of the
curriculum were found to be useful, although the fellows
subjectively found the virtual reality simulation training, the
dry and wet laboratories, and the cadaver training particu-
larly valuable. These data suggest that all these elements
should ideally be integrated in training programs, although
there are cost, ethical, and regulatory issues to take into
account. The centralization of simulation training in an
intensive week at a single, fully equipped training centre was
well perceived and proved to be effective, as previously
demonstrated in other studies [19].
An effective simulation-based training program repre-
sents the ideal background for clinical training in the
operating room. The concept of modular training proposed
by Stolzenburg et al [14] for laparoscopic radical prostatec-
tomy was adopted in the curriculum for progressive,
proficiency-based training of a fellow through steps of
increasing levels of difficulty in a surgical procedure. This
strategy has the potential to overcome the problems of
teaching complex surgical procedures, allowing safe train-
ing of surgeons with limited expertise and potentially
accelerating their learning curve. The importance of proper
mentoring for effective modular training has recently been
highlighted [18].
This pilot study showed that the EAU robotic training
curriculum can effectively improve the basic robotic skills
[(Fig._5)TD$FIG]
Fig. 5 – Assessment of the educational impact, face validity, feasibility, and acceptability of the European Association of Urology Robotic TrainingCurriculum via quantitative surveys.
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9 297
of participants, as demonstrated by the significant improve-
ment in overall scores achieved by fellows at the end of the
curriculum for all dVSS tasks. The first weeks of simulation
training were found to be particularly effective in this
respect, potentially allowing participants to optimize the
results of subsequent training in the operating room.
Importantly, at the end of the curriculum the majority of
fellows were able to perform RARP with good or acceptable
technical quality, as reflected by scores from mentors and
blinded reviews of video-recorded surgeries. The two
participants who were not deemed able to perform RARP
safely and efficiently by the end of the training program
were residents. This may indicate that the curriculum is
more likely to be effective for urologists who have already
completed their basic training in residency programs.
The curriculum also provided constructive feedback on
the performance of fellows for individual procedural steps.
Overall, the fellows were able to reach similar performance
levels compared to expert surgeons for the easiest RARP
steps, such as bladder detachment and endopelvic fascia
incision. However, the experts achieved significantly better
scores for challenging parts of the procedure, confirming
the construct validity of the assessment. Fellows who did
not reach sufficient scores for specific surgical steps will
need further training before being deemed able to safely
perform RARP independently.
Finally, the results indicate that the curriculum has a
good educational impact, as well as face validity, defined as
the extent to which the program is subjectively viewed as
delivering the desired goal. In fact, the majority of the
participants felt that the curriculum significantly improved
their robotic skills and their ability to perform different
RARP steps. However, further comparative studies and,
ideally, randomized controlled trials will be needed to
assess whether this curriculum is superior to traditional
nonstructured training for RARP.
This study has some limitations. First, the number of
participants was limited. Second, although the participants
had limited previous robotic experience, they were not all
completely novice to console surgery, so the results for the
training program may be overestimated. The experience
level of mentors was not factored in the analysis, but they
were all high-volume robotic surgeons at teaching institu-
tions. Third, modular training was not standardized and the
console exposure of the participants was variable. Fourth,
assessment of the technical skills of the fellows using GEARS
and the procedure-specific scoring scale at the end of the
training program was performed by their mentors. Howev-
er, to standardize the assessment process and minimize bias
due to nonblinded assessment, the mentors were informed
about and educated in the use of the scoring systems. Fifth,
the performance of fellows in RARP video clips was assessed
using a new scoring tool. The contents of this scoring
criterion were developed and validated on the basis of
expert opinion. Sixth, a 3-mo training program may be too
short for some fellows to achieve sufficient skills to
adequately perform complex parts of the procedure. Finally,
there was no specific training or proper pre- and post-
training assessment of nontechnical skills.
This study represents the first step towards the
definition of an ideal training program for RARP and of
criteria for accreditation of surgeons for this procedure.
Further studies need to be carried out to address the issues
associated with certification and recertification using
training curricula.
5. Conclusions
This study establishes the effectiveness of the first
structured training curriculum for robot-assisted surgery
that integrates simulation-based training in dry and wet
laboratories, and modular training in the operating room
with expert mentorship. The study shows that the 12-wk
curriculum is valid, feasible, and acceptable, and has a good
educational impact, allowing participants to improve their
basic robotic skills and their ability to perform the surgical
steps of RARP. Further studies are needed to better define
the ideal length and structure of these training programs.
Author contributions: Alexandre Mottrie had full access to all the data in
the study and takes responsibility for the integrity of the data and the
accuracy of the data analysis.
Study concept and design: Mottrie, Dasgupta, van der Poel, Ficarra, Volpe.
Acquisition of data: Volpe.
Analysis and interpretation of data: Volpe, Ahmed.
Drafting of the manuscript: Volpe, Ahmed.
Critical revision of the manuscript for important intellectual content:
Mottrie, Dasgupta, van der Poel, Novara, Ficarra.
Statistical analysis: Volpe.
Obtaining funding: None.
Administrative, technical, or material support: None.
Supervision: Mottrie.
Other (specify): None.
Financial disclosures: Alexandre Mottrie certifies that all conflicts of
interest, including specific financial interests and relationships and
affiliations relevant to the subject matter or materials discussed in the
manuscript (eg, employment/affiliation, grants or funding, consultan-
cies, honoraria, stock ownership or options, expert testimony, royalties,
or patents filed, received, or pending), are the following: None.
Funding/Support and role of the sponsor: None.
Acknowledgments: We would like to thank the ten participants
(F. Audenet, A. Briganti, M. Brown, V. De Marco, M. Gan, M. Janssen,
M. Oderda, R. Navarro, R. Sanchez Salas, and E. Wit) and their respective
mentors and teaching institutions (R. Sanchez Salas, Institut Montsouris,
Paris, France; F. Montorsi, San Raffaele Hospital, Vita Salute University,
Milan, Italy; P. Dasgupta, Guy’s Hospital, Kings College, London, UK;
W. Artibani, University of Verona, Verona, Italy; G. De Naeyer, OLV
Hospital, Aalst, Belgium; M. Stockle, University of Saarland, Homburg/
Saar, Germany; T. Piechaud, Clinique Saint Augustin, Bordeaux, France;
A. Ruffion, Centre Hospitalier Lyon Sud, Hospices Civils de Lyon, Lyon,
France; P. Wiklund, Karolinska University Hospital, Stockholm, Sweden;
and H. van der Poel, The Netherlands Cancer Institute, Amsterdam, The
Netherlands) for their contribution, enthusiasm, and dedication to this
project. We would like also to acknowledge the contributions of the
ERUS board members (http://www.uroweb.org/sections/robotic-
urology-erus/?no_cache=1) and the reviewers of the surgical videos
(C. Assenmacher, N. Buffi, G. D’Elia, P. Dekuiper, N. Doumerc, V. Ficarra,
S. Klaver, D. Murphy, K. Palmer, C.H. Rochat, C. Vaessen, B. van
Cleynenbruegel, C. Wagner, J. Walz, C. Wijburg, and J. Witt).
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9298
Appendix A. Supplementary data
Supplementary data associated with this article can be
found, in the online version, at http://dx.doi.org/10.1016/j.
eururo.2014.10.025.
References
[1] Kneebone RL. Practice, rehearsal, and performance: an approach for
simulation-based surgical and procedure training. JAMA 2009;302:
1336–8.
[2] Scott DJ. Patient safety, competency, and the future of surgical
simulation. Simul Healthc 2006;1:164–70.
[3] Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic
surgery: a multi-institutional randomized controlled trial for valida-
tion of a simulation-based curriculum. Urology 2013;81:767–74.
[4] Smith R, Patel V, Satava R. Fundamentals of robotic surgery: a
course of basic robotic surgery skills based upon a 14-society
consensus template of outcomes measures and curriculum devel-
opment. Int J Med Robot 2014;10:379–84.
[5] Lee JY, Mucksavage P, Sundaram CP, McDougall EM. Best practices
for robotic surgery training and credentialing. J Urol 2011;185:
1191–7.
[6] Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation:
a systematic review. Ann Surg 2006;243:291–300.
[7] Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training
improves operating room performance: results of a randomized,
double-blinded study. Ann Surg 2002;236:458–63.
[8] Ahmed K, Jawad M, Abboudi M, et al. Effectiveness of procedural
simulation in urology: a systematic review. J Urol 2011;186:26–34.
[9] Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ.
Features and uses of high-fidelity medical simulations that lead to
effective learning: a BEME systematic review. Med Teach 2005;27:
10–28.
[10] Brewin J, Ahmed K, Challacombe B. An update and review of
simulation in urological training. Int J Surg 2014;12:103–8.
[11] Zorn KC, Gautam G, Shalhav AL, et al. Training, credentialing,
proctoring and medicolegal risks of robotic urological surgery:
recommendations of the Society of Urologic Robotic Surgeons.
J Urol 2009;182:1126–32.
[12] Khan RS, Ahmed K, Mottrie A, et al. Towards a standardised training
curriculum for robotic surgery: a consensus of an international
multidisciplinary group of experts. Proceedings EAU Robotic Urol-
ogy Section Congress. Stockholm 2013.
[13] http://www.uroweb.org/index.php?id=466&cid=3&auth=1.
[14] Stolzenburg JU, Schwaibold H, Bhanot SM, et al. Modular surgical
training for endoscopic extraperitoneal radical prostatectomy. BJU
Int 2005;96:1022–7.
[15] Ahmed K, Jawad M, Dasgupta P, Darzi A, Athanasiou T, Khan MS.
Assessment and maintenance of competence in urology. Nat Rev
Urol 2010;7:403–13.
[16] Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Global evalua-
tive assessment of robotic skills: validation of a clinical assessment
tool to measure robotic surgical skills. J Urol 2012;187:247–52.
[17] Foell K, Finelli A, Yasufuku K, et al. Robotic surgery basic skills
training: evaluation of a pilot multidisciplinary simulation-based
curriculum. Can Urol Assoc J 2013;7:430–4.
[18] Hay D, Khan MS, Van Poppel H, et al. Current status and effective-
ness of mentorship programmes in urology—a systematic review.
BJU Int 2014.
[19] Shamim Khan M, Ahmed K, Gavazzi A, et al. Development and
implementation of centralized simulation training: evaluation of
feasibility, acceptability and construct validity. BJU Int 2013;111:
518–23.
E U R O P E A N U R O L O G Y 6 8 ( 2 0 1 5 ) 2 9 2 – 2 9 9 299