applying best practices in residency selection€¦ · applying best practices in residency...
TRANSCRIPT
Applying Best Practices in Residency Selection :
A Quality Improvement Project for the Dalhousie Internal Medicine Program
Authors: Alison Rodger, Marko Balan, Ian Epstein
Date: September 30, 2016
Delete this box before presenting: Four title slide options have been provided with different feature
images, select one for use in your presentation.
I do not have an affiliation (financial or otherwise) with a
pharmaceutical, medical device or communications organization.
Je n’ai aucune affiliation (financière ou autre) avec une
entreprise pharmaceutique, un fabricant d’appareils
médicaux ou un cabinet de communication.
2
Delete this box before presenting: Please keep only the applicable statement, shown above is Option 1
with Option 2 to follow on next slide.
Note: Speakers who have no involvement with industry should inform the audience that they cannot identify any conflict of interest.
Supprimez cette boîte avant de faire votre présentation : Veuillez conserver uniquement l’énoncé pertinent; ci-dessus se trouve l’option 1, l’option 2 paraissant à la diapo suivante. Note: Les conférenciers qui n’ont aucun lien avec l’industrie doivent aviser leur auditoire qu’ils n’ont aucun conflit d’intérêts à déclarer.
Summary
• Transparency, knowledge translation and applicant ranking were the
main areas of weakness identified in this review.
• This project lead to the development of program-specific documents
and protocols for the applicant selection process.
• Controversy exists regarding how to best manage conflict of interests
and whether to use information other than that contained in application
files.
3
Background
• Significant variation exists between programs for applicant selection
for postgraduate residency programs
• Ensuring efficacy and standardization in resident selection is
challenging
• Increasing emphasis has been placed on the responsibility of medical
faculties in training a diverse group of physicians, with the right
composition of generalists and specialists who are willing to serve in
underserviced areas1,2
• Best practice recommendations have been developed by a working
group at the University of Toronto and have been proposed to be used
by programs across Canada1
4
Purpose and Methods
• The purpose of this study was to evaluate the residency selection
process based on the twenty-four best practice recommendations as
outlined by the Best Practices in Application and Selection Final
Report (BPAS) 1
• Two of the study authors conducted document review and interviews
with the program administrators and the Program Director
• Adherence to each best practice recommendation was determined by
consensus agreement between both authors
• The authors of the study identified and completed action items specific
to the program’s deficiencies
5
Dalhousie Internal Medicine Application and Selection Process
6
Results
7
Transparency Adherence
1. Define selection goals Partially
2. Define where relevant attributes assessed No
3. State processes and metrics to candidates Partially
4. Maintain records of adherence Yes
5. If using information outside application, must be consistent, fair, transparent
No
6. If using such information, must have process to investigate and validate
No
7. Specific practice for retention and protection of records Yes *Table developed using Best Practices in Applications and Selection Final Report 1
Results
8
Fairness Adherence
8. Assess each component independently Partially
9. Abide by guidelines for management of conflict of interest
Partially
Selection Criteria
10. Program specific criteria for assessment Yes
11. Criteria specific and validated for specialty Partially
Process
12. Assessment systems must be standardized Partially
13. Assessments based on skills or previous behaviours Yes
14. Assessments based on multiple independent samples Yes
15. Regularly assess outcomes of process Partially *Table developed using Best Practices in Applications and Selection Final Report 1
Results
9
Assessors Adherence
16. Selection team with breadth of perspectives Yes
17. Assessors trained in all aspects of the process Partially
Assessment Instruments
18. Strive to incorporate objective assessments Yes
Knowledge Translation
19. Share best practices among programs No
20. Innovations in process done in scholarly manner No
Ranking
21. Process to receive information alleging improper behaviours
Partially
22. Clear criteria for ‘do not rank’ No
23. Rank candidates based on assessment Partially
24. Rank using pre-defined, transparent process Partially *Table developed using Best Practices in Applications and Selection Final Report 1
Discussion
• Areas of weakness were in transparency, applicant ranking and
knowledge translation
• Applicant selection practices specific for the Dalhousie Internal
Medicine Program were updated and documented reflecting best
practices as per BPAS 1
Program goals were revised and selection process goals were developed
File review and interview templates were updated to better reflect desired
candidate attributes and selection goals
File reviewer and interviewer guides created
• Changes to the current interviewer structure were not deemed
necessary
10
Discussion
• Goals and templates approved by Residency Program Committee
• Dalhousie Legal concerns regarding wording of proposed conflict of
interest policy
• Dalhousie Legal suggests omitting information obtained outside of an
applicant’s file
• CaRMS website for 2017 application cycle updated to reflect new
Dalhousie IM selection practices as per BPAS1
11
Summary
• Transparency, knowledge translation and applicant ranking were the
main areas of weakness identified in this review.
• Undergoing this project led to the development of program-specific
documents and protocols for the applicant selection process that will
improve the overall process.
• Controversy exists regarding how to best manage conflict of interests
and whether to use information other than that contained in application
files.
12
References
• 1. Bandiera et al. 2013. Postgraduate Medical Education University of
Toronto, 2013.
• 2. The Future of Medical Education in Canada: A Collective Vision for
Postgraduate Medical Education in Canada. 2012.
• 3. Maksidi G et al. 2011. Journal of Surgical Education 2011; 68(1):
67-72.
• 4. Provan JL et al 1995. Canadian Medical Association Journal. 1995;
153(7): 919-923.
• 5. Stephenson-Famy A et al 2015. Journal of Graduate Medical
Education. 2015; 7(4): 539-48.
13
• Download the ICRE App,
• Visit the evaluation area in the Main Lobby, near Registration, or
• Go to: http://www.royalcollege.ca/icre-
evaluations to complete the session evaluation.
Help us improve. Your input matters.
• Téléchargez l’application de la CIFR
• Visitez la zone d’évaluation dans le hall principal, près du comptoir d’inscription, ou
• Visitez le http://www.collegeroyal.ca/evaluations-
cifr afin de remplir une évaluation de la séance.
Aidez-nous à nous améliorer. Votre opinion compte!
You could be entered to win 1 of 3 $100 gift cards.
Vous courrez la chance de gagner l’un des trois chèques-cadeaux d’une valeur de 100.
14