6 mitchell 2012 apdvs milestones
TRANSCRIPT
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
1/27
Erica L. Mitchell, M.D.
Associate Professor Surgery
Howto
Determine
Appropriate
Milestones:EducationalPerspective
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
2/27
Overview
AimsoftheACGMEmilestoneproject
Competencieswe
need
to
assess
Toolscurrentlyusedtomeasurecompetence
Howwe
can
develop
milestones
for
Vascular
Surgery
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
3/27
ACGMEMilestone
Project
Processof
educationNext
Accreditation
Cycle(NAS) Educationaloutcomes
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
4/27
Whatdo
we
need
to
assess?
medicalknowledge intellectual
practical
patientcare
professionalism communicationandinterpersonalskills practice
based
learning
and
improvement
systemsbasedpractice
Knowledge,skills&attitudes
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
5/27
Endproduct
of
Vascular
Surgical
Training
Workingknowledgeofvasculardiseases Coordinatesthecareofpatients
Performs
appropriate
procedures
safely Trustworthyandempathetic Leadershipanddecisionmakingability
Communication
skills Maintainsscholarship Informedonevidencebasedpractices
CompetentVascularSurgeon
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
6/27
Howcanweassessthedomainsof
competence?
Theassessmentofclinicalskills/competence/performance
Professionalauthentici
ty
behavior
cognition
novice
expertPerformanceintegrated
intopractice
Demonstrationof
learning
Interpretation/
application
of
knowledge
Factgathering
Knowledge,skills&attitude MillerGE.AcadMed1990
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
7/27
Howcanweassessthedomainsof
competence?
Writtenassessments
Oralexaminations
Simulation
Workplacebasedassessments
Multisource(360
degree)
feedback
(MSF)
Typesofassessmentcommonlyused
RMEpstein
Assessmentin
Medical
Education
NewEnglJMed2007356:387396
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
8/27
Assessmentof
competence
VSITE&ABSqualifyingexam
Multichoicequestions Singlebestanswer
Knowledge&
reasoning
ability
Writtenassessments
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X
Limitations:
cannot
measure
behavioral
and
non
technical
skills
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
9/27
Assessmentof
competence
Mockorals&ABScertifyingexam
Casespecific Unstructuredfacetofacesession
Knowledge,skills
and
judgment
Oralexaminations
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Limitations:
cannot
reliably
predict
performance
across
range
of
cases
andcannotassesstechnicalskills
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
10/27
Assessmentof
competence
Miniclinicalevaluationexercise(miniCEX)
Directobservationofproceduralskills(DOPS)
Procedurebasedassessment(PBA)
Nontechnical
skills
for
surgeons
(NOTSS
Multisource(360degree)feedback(MSF)
Workplacebasedassessments
Formativeassessment
s Assesswhatthetraineeactuallydoesintheworkplace
Feedback Limitations:supervisedsetting
NorciniJ
Workplace
based
assessment
as
an
educational
toolMedTeacher200729:855871
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
11/27
Assessmentof
competence
Directobservationofatraineeinteractionwithapatientintheclinicalsetting
Casespecific
Miniclinicalevaluationexercise(miniCEX)
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Limitations: maynotreliablypredictperformanceacrossrangeofcases
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
12/27
Effective2012theABSisrequiring
2clinicalassessments/resident
Canbeusedinanumberof
differentclinicalsettings.
Potentiallyused
to
evaluate
residentpatientinteractions:
Dialysisaccess
Claudication
Criticallimb
ischemia
Carotidstenosis
Aneurysmaldisease
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
13/27
Howcanweassessthedomainsof
operativecompetence?
Surrogatemeasures
Simulation
Workplace
based
assessments
Objectivestructuredassessmentoftechnicalskills(OSATS)
Procedurebasedassessment(PBA)
Nontechnicalskillsforsurgeons(NOTSS)
Typesofassessmentcommonlyused
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
14/27
Assessmentof
technical
competence
Operativespeed
Correlatespoorlywithpatientoutcome
Casenumbers
lacksvalidityandreliabilityasamethodofassessingtraineessurgicalskill
Surrogatemeasures
Assessmentof
operative
competence
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
15/27
Assessmentoftechnicalskillinaskillslab:
Procedurespecific
Proceduralskillorsequencing
Simulation
Limitations:needtosamplewidelyandconsiderbothcontextandconstruct,
cangame
the
simulator,
ceiling
effect
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X
Assessmentof
operative
competence
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
16/27
Assessmentof
operative
competence
Checklists
Procedurespecific
Captureoccurrenceofparticularbehavior
Perceivedtobemoreobjective
Scoresslightly
more
reliable
Goodforassessingtrainees
Globalratingscores
Itemscommon
to
any
procedure
Capturethequalityofaperformance
Perceivedtobemoresubjective
Scoresslightly
more
valid
Goodforassessingexperts
Objectivestructuredassessmentoftechnicalskills
(OSATS)
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
17/27
Traineesobservedperformingaprocedureonarealpatient
Procedurespecific
Directobservationofproceduralskills(DOPS)
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Limitations: maynotreliablypredictperformanceacrossrangeofcases
Assessmentof
operative
competence
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
18/27
Effective2012theABSisrequiring
2operativeassessments/resident
Potentiallybeusedtoevaluate
residentsperformingroutine
coresurgical orminorvascular
procedures:
I&Dabscess
Radialarterialline
Centralvenouscatheter
Temporarydialysis
catheter
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
19/27
AssessmentoftechnicalskillintheOR:
Procedurespecific
OSATS
Knowledge,skills
&
attitudes
Procedurebasedassessment(PBA)
Limitations:needtosamplewidelyandconsiderbothcontextandconstruct
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Assessmentof
operative
competence
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
20/27
Assessmentof
operative
competence
AssessmentofnontechnicalskillintheOR:
Situationawareness
Decisionmaking
Communicationand
teamwork
Leadership
Nontechnicalskillsforsurgeons(NOTSS)
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Limitations:needtosamplewidelyandconsiderbothcontextandconstruct
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
21/27
Assessmentof
competence
Peerassessment(untrainedraters) Teammembers,physicianextenders
Referringservices,anesthesiastaff
Clinic
staff,
ward
clerks Self
Multisource(360)feedback(MSF)
Medicalknowledge Patientcare Professionalism Communication&
interpersonal
skills
Practicebased
learning&
improvement
Systemsbased
practiceIntellectual Practical
X X X X X X X
Limitations:needmultipleassessments
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
22/27
Makingjudgmentsaboutthe
performanceofpeershasformedthe
basisof
the
referral
process
in
medicine
andtheotherprofessionsforcenturies.
NorciniJJ
Peerassessmentofcompetence
MedEducation200337:539543
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
23/27
Developingmilestones
for
vascular
surgery
Definemarkersofachievementacrossthecontinuumofvascularsurgicaleducation
Blueprintthese
to
the
entire
curriculum
Knowledge,skills&attitudes
6corecompetencies
Coresurgical&specialtyspecificskills
Consensus:Delphiprocessusingspecialtyexperts
Systematicapproach
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
24/27
Whatwe
need
to
do
as
aspecialty
Identifyorcreatetoolsthatcanbeembraced asmeaningfulmeasuresofperformance Acrossthecontinuumofvascularsurgicaltraining
Assessacross
multiple
contexts:
Clinical,simulatedandORsetting Clinical&operativecompetencies
Assessmentcontent:taskanalysisandDelphiprocessesusingspecialtyexperts
Systematicapproach
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
25/27
Whatwe
need
to
do
as
aspecialty
Needtoensureassessmentsare: Reliable Valid
Acceptable Costeffective Educationalimpact
Vander
Vleutens
utility
index
Utility=Validityw xreliabilityw xacceptabilityw xeducationalimpactw xcostw W=weighted
Systematicapproach
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
26/27
Assessment
Standardizedtesting VSITE,ABSQE
Operativeandcaselogdata
Assessmentof
patient
care
Clinicalsetting(miniCEX)
ClinicorOR(DOPS)
Operativesetting(PBA)
360 evaluations(MSF)
Compliancewithqualityperformanceindicators
Competencyassessed
medicalknowledge intellectual
practical
patientcare
professionalism
communicationandinterpersonalskills
practicebasedlearningandimprovement
systemsbasedpractice
Evidencedocumenting
trainees
learning
Learningportfolio
-
7/28/2019 6 Mitchell 2012 APDVS Milestones
27/27
Endproduct
of
Vascular
Surgical
Training
Workingknowledge
of
vascular
diseases
Coordinatesthecareofpatients Performsappropriateproceduressafely Trustworthyandempathetic Leadershipanddecisionmakingability Communicationskills Maintainsscholarship Informedonevidencebasedpractices
CompetentVascularSurgeon