michael jibson , m.d., ph.d. david kaye , m.d. sandra sexson , m.d
DESCRIPTION
Clinical Skills Verification Rater Training MODULE 4 Strategies for Clinical Skills Assessment: Models and Best Practices. Michael Jibson , M.D., Ph.D. David Kaye , M.D. Sandra Sexson , M.D. Module 4 Pre-Test. - PowerPoint PPT PresentationTRANSCRIPT
Clinical Skills Verification Rater TrainingMODULE 4
Strategies for Clinical Skills Assessment:Models and Best Practices
Michael Jibson, M.D., Ph.D.David Kaye , M.D.
Sandra Sexson , M.D.
Module 4 Pre-Test
1. The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except:
A. A high stakes examination
B. direct observation of clinical skillsC. provision of immediate feedback on performanceD. repeated administrations during residency
2. Which of the following models for CSV exam is the least standardized?
A. Workshop Model
B. Mock Board Model
C. Clinical Evaluation Model
Module 4 Pre-Test
Module 4 Pre-Test
3. Which of the following models for the CSV exam makes the greatest use of resident peer feedback?
A. Workshop Model
B. Mock Board ModelC. Clinical Evaluation Model
Module 4 Pre-Test
4. Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by:
A. All raters using the same evaluation model
B. Limiting assessment to the three competency areas outlined by the ABPN
C. Using standardized patientsD. Monitoring inter-rater reliability
Clinical Skill Verification (CSV)
Strengths of Clinical Skills Verification• Direct observation of clinical skill• Multiple observations and observers• Defined areas for evaluation• Specific anchors for performance• Immediate feedback• Ample opportunity for remediation
Evaluation Model
We recommend that the evaluation:• Be done early in residency and repeated
frequently• Include immediate feedback • Avoid the “high-stakes exam” model
Evaluation Model
Mock Board Model• Use the CSV for the current practice of
“Mock Board” exams• Prearranged patients • Scheduled interviews
Mock Board Model
Advantages• Familiar format• Highly standardized
Disadvantages• Retains high-stakes exam format• Narrow patient selection• Difficult to schedule
Evaluation Model
Workshop Model• Use the CSV form for a workshop or class
on interviewing• Involve peer resident observers • Videotape the encounter
Workshop Model
Advantages• Multiple residents
involved• Highly
standardized• Excellent
feedback (if videotaped)
Disadvantages• Few opportunities
for each resident to interview
• Narrow patient selection
Evaluation Model
Clinical Model• Use the CSV form for any initial clinical
encounter with supervising faculty• Inpatient admission• Emergency room evaluation• Outpatient assessment
Clinical Model
Advantages• Broad range of
patients• Many
opportunities to interview
• Easy to schedule
Disadvantages• Moderately
standardized• Faculty buy-in
essential
Evaluation Model
We recommend that the evaluation:• Occur during regularly scheduled clinical
rotations• Involve the standard interview used in that
clinical setting• Involve randomly selected patients• Be done as part of the routines of clinical
care
Additional Issues
Programs may want to consider:• Using the exam to test additional
competencies for their own purposes• Requiring residents to pass the examination
before advancing through training
Additional Competencies
CSV requirements may be combined with other clinical competencies:• Case formulation• Diagnostic assessment• Treatment planning
Faculty Training
We recommend that programs:• Offer training to faculty in how to conduct
and score the examination• Make an effort to standardize scoring on the
examination• Monitor inter-rater reliability internally• Consider credentialing faculty as evaluators
Post-Test Module 4
1. The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except:
A. A high stakes examination
B. direct observation of clinical skillsC. provision of immediate feedback on performanceD. repeated administrations during residency
Post-Test Module 4
1. The strength of a Clinical Skills Verification Examination stems from its ability to serve as a forum for all of the following except:
A. A high stakes examination
The in –residency CSV exam allows programs to administer repeatedly and avoid the high-stakes aspect.
Post-Test Module 4
2. Which of the following models for CSV exam is the least standardized?
A. Workshop Model
B. Mock Board ModelC. Clinical Evaluation Model
Post-Test Module 4
2. Which of the following models for CSV exam is the least standardized?
C. Clinical Model
The clinical model utilizes patients seen on regular clinical services without pre-selection of patients
Post-Test Module 4
3. Which of the following models for the CSV exam makes the greatest use of resident peer feedback?
A. Workshop Model
B. Mock Board ModelC. Clinical Evaluation Model
Post-Test Module 4
3. Which of the following models for the CSV exam makes the greatest use of resident peer feedback?
A. Workshop Model
Post-Test Module 4
4. Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by:
A. All raters using the same evaluation model
B. Limiting assessment to the three competency areas outlined by the ABPN
C. Using standardized patientsD. Monitoring inter-rater reliability
Post-Test Module 4
4. Within a program, consistency of faculty expectations and rating of residents on the CSV exam can be maximized by:
D. Monitoring inter-rater reliability
Along with faculty training on how to conduct and score the examination, tracking inter-rater reliability among faculty after observing the same performance and giving feedback to faculty regarding their ratings compared to others is a very effective way of standardizing ratings.