results of the 2018 nrmp program director survey · introduction nrmp program director survey...
TRANSCRIPT
Results of the 2018 NRMP Program Director Survey
www.nrmp.org
June 2018
Requests for permission to use these data as well as questions about the content of this publication or the National Resident Matching Program data and reports may be directed to
Mei Liang, Director of Research, NRMP, at [email protected].
Questions about the NRMP should be directed to Mona M. Signer, President and CEO, NRMP, at [email protected].
Suggested CitationNational Resident Matching Program, Data Release and Research Committee: Results of the
2018 NRMP Program Director Survey. National Resident Matching Program, Washington, DC. 2018.
Copyright © 2018 National Resident Matching Program. All rights reserved. Permission to use, copy and/or distribute any documentation and/or related images from this publication shall be
expressly obtained from the NRMP.
Introduction .................................................................................................................................................................. 1 All Specialties............................................................................................................................................................... 2 Charts for Individual Specialties
Anesthesiology ....................................................................................................................................................... 9 Child Neurology ................................................................................................................................................... 16 Dermatology ......................................................................................................................................................... 23 Emergency Medicine ........................................................................................................................................... 30 Family Medicine ................................................................................................................................................... 37 Internal Medicine .................................................................................................................................................. 44 Internal Medicine/Pediatrics ................................................................................................................................. 51 Interventional Radiology ...................................................................................................................................... 58 Neurological Surgery ............................................................................................................................................ 65 Neurology ............................................................................................................................................................. 72 Obstetrics and Gynecology .................................................................................................................................. 79 Orthopaedic Surgery ............................................................................................................................................. 86 Otolaryngology ..................................................................................................................................................... 93 Pathology ............................................................................................................................................................ 100 Pediatrics ............................................................................................................................................................ 107 Physical Medicine and Rehabilitation ................................................................................................................ 114 Plastic Surgery .................................................................................................................................................... 121 Psychiatry ........................................................................................................................................................... 128 Radiation Oncology ............................................................................................................................................ 135 Radiology-Diagnostic ......................................................................................................................................... 142 Surgery ............................................................................................................................................................... 149 Thoracic Surgery ................................................................................................................................................ 156 Transitional Year ................................................................................................................................................ 163 Vascular Surgery ................................................................................................................................................ 170
Table of Contents
Introduction
1NRMP Program Director Survey Results, 2018
In March 2018, the National Resident Matching Program (NRMP) conducted its biennial survey of the directors of all programs participating in the Main Residency Match®. The primary purpose of the survey was to shed light on the factors that program directors use to (1) select applicants to interview and (2) rank applicants for the Match.
The survey was fielded during the 18 days between the Rank Order List Deadline and Match Week so that program match outcomes would not influence respondents' answers.
The survey solicited information on: the number of applications received, screened, and
reviewed, as well as the number of applicants interviewed and ranked
whether the program typically interviews and ranks specific applicant groups
use of test scores in considering which applicants to interview, and
the factors used for both interview selection and for ranking applicants.
The overall response rate for all specialties in this report was 29.2 percent. Transitional Year programs and programs in 23 specialties with 10 or more responses are included in this report. Response rates are listed in the table below.
Beginning with the 2016 survey, program directors were asked to indicate the percentage of interview invitations sent and
interviews conducted in certain time periods. They also were asked how often they interviewed and ranked candidates from each applicant group. In addition, program directors rated factors used in assessing residents' success. In surveys conducted prior to 2014, program directors were asked to indicate factors they used in selecting applicants to interview and to rate the importance of each factor in ranking. Beginning with the 2014 survey, program directors have been asked to indicate factors they used in selecting applicants to interview and rank and rate their importance on a scale of 1 to 5.
Numbers of responses are presented in most of the graphs, and some graphs use data from multiple survey questions. In those cases, different N's are listed. Numbers of applicants ranked and positions in the Match are extracted from the NRMP database. Graphs are suppressed for questions with fewer than five responses.
This report presents results by specialty on selected items from the survey. The NRMP hopes that program directors, medical school officials, and applicants find these data useful as they prepare for and participate in the Main Residency Match._______________
The NRMP's data reporting and research activities are guided by its Data Release and Research Committee. NRMP data and reports can be found at: www.nrmp.org/main-residency-match-data/.
Specialty Surveys Sent Number Responding Response Rate
Anesthesiology 234 62 26.5%
Child Neurology 94 36 38.3%
Dermatology 124 21 16.9%
Emergency Medicine 190 92 48.4%
Family Medicine 487 152 31.2%
Internal Medicine 741 181 24.4%
Internal Medicine/Pediatrics 72 27 37.5%
Interventional Radiology 70 18 25.7%
Neurological Surgery 104 28 26.9%
Neurology 151 41 27.2%
Obstetrics and Gynecology 263 85 32.3%
Orthopaedic Surgery 142 49 34.5%
Otolaryngology 97 35 36.1%
Pathology 128 43 33.6%
Pediatrics 209 78 37.3%
Physical Medicine and Rehabilitation 99 28 28.3%
Plastic Surgery 77 22 28.6%
Psychiatry 205 61 29.8%
Radiation Oncology 90 27 30.0%
Radiology‐Diagnostic 203 48 23.6%
Surgery‐General 492 107 21.7%
Thoracic Surgery 24 11 45.8%
Transitional Year 103 30 29.1%
Vascular Surgery 52 16 30.8%
All Others 95 35 36.8%
Total 4,546 1,333 29.3%
All Specialties Combined(N=1,333)
100% 50% 0%
94%
86%
81%
80%
78%
76%
70%
70%
69%
68%
67%
65%
65%
61%
60%
59%
58%
56%
54%
54%
53%
52%
50%
47%
46%
41%
40%
37%
26%
24%
24%
22%
16%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
1 2 3 4 5
4.1
4.2
4.0
4.0
3.7
4.1
4.5
3.9
4.3
4.2
4.3
4.2
4.5
4.1
3.9
4.1
3.8
4.2
3.8
4.0
4.0
3.6
3.8
3.8
3.8
3.7
4.1
4.8
3.8
3.8
3.7
3.3
3.4
Figure 1All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=1,233)
Percent Citing Factor Average Rating
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
3NRMP Program Director Survey Results, 2018
100% 50% 0%
96%95%91%
86%78%
72%70%65%65%64%63%60%59%58%58%56%54%51%47%46%45%44%41%40%38%36%36%32%30%30%26%25%23%22%19%17%16%14%14%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=1,208)
Figure 2
1 2 3 4 5
4.84.94.84.7
4.14.14.14.1
4.64.4
4.24.2
4.04.34.3
3.64.14.2
4.54.2
3.83.94.0
3.84.0
3.83.93.84.0
3.74.7
4.03.73.84.03.9
3.63.6
3.4
Average RatingPercent Citing Factor
4NRMP Program Director Survey Results, 2018
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
21%
39% 40%
Exam Required?
All SpecialtiesPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure 3
Percentage of Programs That Use USMLE Step 1 Scores
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
34%
64%
2%
Exam Required?
0%
20%
40%
60%
80%
100%
Never Seldom Often
30%
58%
12%
Would your program consider applicants who fail theexam on the first attempt?
0%
20%
40%
60%
80%
100%
Never Seldom often
35%
57%
8%
Would your program consider applicants who fail theexam on the first attempt?
N=1,232N=1,215
N=1,207 N=1,171
N=342N=623N=736
USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
N=394
170
180
190
200
210
220
230
240
250
260
Scores Below Which Programs Generally Do Not Grant Interviews
170
180
190
200
210
220
230
240
250
260
Scores Above Which Programs Almost Always Grant Interviews
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol in the box represents the mean.
USMLE Step 2 CKUSMLE Step 1USMLE Step 2 CKUSMLE Step 1
5NRMP Program Director Survey Results, 2018
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 3%
93%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%
39%34%
COMLEX-USA Level 1*Exam Required?
N=590%
20%
40%
60%
80%
100%
Yes No
54%46%
USMLE Step 2 Clinical Skills (CS)Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%
28%
54%
COMLEX-USA Level 2-CE*Exam Required?
N=470%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5% 3%
92%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%
13%
58%
COMLEX-USA Level 2-PE*Exam Required?
N=965 N=946
N=939 N=842
* Osteopathic applicants only
Figure 4 All SpecialtiesPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA
Scores* When Considering Which Applicants to Interview
N=1,195 N=1,024
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
6NRMP Program Director Survey Results, 2018
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12017
October 2017 November 2017 December 2017 January 2018 orlater
18%
47%
22%
9%4%
Percentage of Program's Interviews Extended During Each TimePeriod
All SpecialtiesProgram's Interview Activities Figure 5
0
100
200
300
400
500
600
700
800
900
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants ranked
7
904
121 95 82
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applications rejectedbased on a standardized screening
process
Percentage of applicants receiving anin-depth review
48% 47%
Average Percentage of Applicants Rejected and Reviewed
N=1,223 N=1,254N=1,288 N=1,284 N=1,301 N=1,303*N=1,333*
7NRMP Program Director Survey Results, 2018
* Data from the NRMP database. All Responding programs included
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12017
October 2017 November 2017 December 2017 January 2018 orlater
1%
12%
31% 33%
23%
Percentage of Program's Interviews Conducted During Each TimePeriod
N=1,209 N=1,210
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
U.S. Senior U.S. Graduate Osteopathic Physician Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
97%
65%71%
39%
16%
59%
42%
Percentage of Programs that Typically Interview and Rank Each Applicant Type
N=1,242
13% of all programs consider all applicant groups
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev. Grad Osteo Pathway Canadian US IMG Non-US IMG1% 6%
12%
70%
32%
14%
32%
4%
62%
31%
27%
55%
54%
46%95%
32%
57%
3%12%
32%21%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev. Grad Osteo Pathway Canadian US IMG Non-US IMG1%
7%14%
72%
34%
16%
34%
3%
60%
29%
25%
52%
52%
45%96%
33%
57%
3%14%
32%20%
Often Seldom Never
Ranking
Frequency of Programs Interviewing and Ranking Candidates
n=1,228 n=1,225
All SpecialtiesOther Topics Figure 6
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
We want to receive onlythe ERAS application
We will initiate contactwith references and/orthe medical school, but
we do not wish to becontacted
We want to be contactedby the applicant’s medical
school after we haveinitiated contact with the
applicant
Other
48%
22%18%
12%
Preferred Type of Communication About Applicants During SOAP
N=1,075
8NRMP Program Director Survey Results, 2018
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.9
4.8
4.7
4.3
4.3
3.9
3.8
3.3
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
n=1,165
Anesthesiology(N=62)
NRMP Program Director Survey Results, 2018 9
Figure AN-1AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=58)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.4
4.0
4.2
4.3
3.5
4.2
4.7
4.1
4.1
4.2
4.2
4.2
4.5
4.2
3.9
3.9
3.8
4.3
3.7
4.2
4.4
3.7
3.9
3.8
4.0
3.6
4.3
4.7
3.2
2.9
3.6
3.7
3.0
100% 50% 0%
97%
88%
84%
79%
79%
84%
79%
83%
71%
78%
50%
76%
69%
71%
66%
79%
55%
43%
50%
74%
50%
59%
57%
55%
34%
29%
31%
57%
16%
16%
26%
26%5%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 10
AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=58)
Figure AN-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.9
4.84.7
4.33.7
4.24.2
4.74.34.34.24.24.34.4
3.74.44.3
4.74.0
3.84.0
4.43.7
4.24.14.04.14.1
3.84.7
3.93.9
3.33.8
3.43.7
3.33.9
100% 50% 0%
97%97%
91%88%
81%62%
71%62%
67%64%66%
60%66%66%
59%47%
52%41%43%
36%53%
57%53%
31%36%
52%47%
34%29%
24%33%
21%40%
19%14%14%19%
24%26%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 11
170
180
190
200
210
220
230
240
250
260
AnesthesiologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure AN-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
34%
66%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
40%
58%
2%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
86%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
47%
39%
Exam Required?
N=59
N=58
N=59N=57
N=27N=50 N=45 N=22
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 12
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%
45% 45%
COMLEX-USA Level 1*Exam Required?
N=59
0%
20%
40%
60%
80%
100%
Yes No
53%47%
USMLE Step 2 CSExam Required?
AnesthesiologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure AN-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
7%
31%
62%
COMLEX-USA Level 2-CE*Exam Required?
N=47
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%2%
89%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
8%3%
89%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
21%
10%
69%
COMLEX-USA Level 2-PE*Exam Required?
N=59N=47
N=42 N=42
N=42 N=36
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 13
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 95%86%
78%
54%
15%
73%
51%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure AN-5AnesthesiologyProgram's Interview Activities
0
100
200
300
400
500
600
700
800
900
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
7
805
164131 111
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
47% 47%
Average Percentage of Applicants Rejected and Reviewed
N=60N=59N=62* N=61*N=61N=61
N=59
10% of Anesthesiology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=61
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2%
72%
14%7%
14%3%
32% 34%
26%
63%
59%
68%95%
68% 66%
2%
23%34%
18%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 5%
73%
20%9%
18%4%
32%29%
27%
57%
51%
61%95%
68% 65%
23%
40%
21%
Often Seldom Never
Ranking
n=59n=59
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 14
Figure AN-6AnesthesiologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=56
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
23%
44%
21%
8% 5%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
14%
35% 33%
18%
Percentage of Program's InterviewsConducted During Each Time Period
N=59 N=59
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
4.9
4.9
4.8
4.9
4.5
4.4
3.7
4.3
3.2
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 15
Child Neurology(N=36)
NRMP Program Director Survey Results, 2018 16
Figure CN-1Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=33)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
3.9
4.1
4.2
3.9
3.5
3.8
4.5
3.7
4.5
4.2
4.3
4.0
4.6
4.0
3.7
4.0
3.4
4.4
3.5
3.9
4.1
3.6
3.9
3.5
4.0
3.6
3.5
4.8
3.2
3.9
3.7
3.2
3.6
100% 50% 0%
100%
91%
91%
88%
79%
85%
67%
76%
88%
58%
82%
70%
70%
52%
58%
61%
67%
67%
55%
58%
52%
73%
79%
45%
67%
55%
33%
39%
33%
30%
9%
39%24%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 17
Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=33)
Figure CN-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.84.8
4.64.1
3.84.04.1
4.64.5
4.34.2
3.94.3
3.93.6
4.04.54.4
4.23.4
4.13.6
3.43.93.8
3.63.63.93.9
4.52.6
3.13.94.0
2.63.4
4.03.0
100% 50% 0%
97%97%97%97%
82%88%
79%76%
79%88%
79%58%
85%52%
61%76%
64%67%
39%70%
42%58%
48%36%
48%58%
52%61%61%
45%24%
27%21%
45%12%15%
27%21%
36%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 18
170
180
190
200
210
220
230
240
250
260
Child NeurologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure CN-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
27%
70%
3%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
39%
58%
3%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
45%
55%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
38%31% 31%
Exam Required?
N=33
N=33
N=32N=33
N=8N=18 N=17 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 19
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
31%23%
46%
COMLEX-USA Level 1*Exam Required?
N=32
0%
20%
40%
60%
80%
100%
Yes No
66%
34%
USMLE Step 2 CSExam Required?
Child NeurologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure CN-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
28%
8%
64%
COMLEX-USA Level 2-CE*Exam Required?
N=26
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
19%
0%
81%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
16%
0%
84%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
31%
8%
62%
COMLEX-USA Level 2-PE*Exam Required?
N=32N=26
N=26 N=25
N=26 N=25
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 20
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 94%
64%73%
45%
18%
61%52%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure CN-5Child NeurologyProgram's Interview Activities
0
20
40
60
80
100
120
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
2
101
2922 20
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
39%
60%
Average Percentage of Applicants Rejected and Reviewed
N=35N=34N=36* N=36*N=36N=35
N=33
12% of Child Neurology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=36
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
6%15%
72%
15%6%
18%
52%
52%
28%
76%
73%64%94%
33%48%
9%21% 18%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
3%13%
6%
77%
15%9%
16%
47%45%
23%
73%76% 69%97%
41%48%
12% 15% 16%
Often Seldom Never
Ranking
n=33n=33
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 21
Figure CN-6Child NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=32
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
21%
48%
19%
8%4%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
10%
30%
41%
19%
Percentage of Program's InterviewsConducted During Each Time Period
N=33 N=33
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.8
4.8
4.8
4.8
4.5
4.1
3.9
3.5
3.3
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 22
Dermatology(N=21)
NRMP Program Director Survey Results, 2018 23
Figure D-1DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=18)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.3
4.1
4.4
3.6
3.8
4.3
4.4
4.1
4.1
4.7
4.0
4.5
4.3
4.0
3.8
3.9
3.8
3.8
3.9
4.0
3.5
3.4
3.8
3.8
3.7
4.0
4.7
4.7
3.6
4.2
3.3
3.5
3.4
100% 50% 0%
100%
100%
83%
83%
89%
89%
56%
72%
94%
83%
50%
83%
94%
72%
83%
89%
78%
33%
83%
61%
17%
67%
33%
56%
33%
61%
33%17%
56%
61%
17%
33%28%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 24
DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=18)
Figure D-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.64.6
4.24.0
3.83.7
3.94.1
3.74.1
4.44.64.84.7
3.54.3
4.53.3
4.74.1
4.44.0
3.8
4.04.24.34.5
4.2
4.04.6
4.05.0
4.0100% 50% 0%
94%100%
83%94%
83%67%
33%78%
61%50%
61%61%
39%56%
61%56%
44%11%17%
17%44%
39%28%
33%0%
17%28%
17%
11%33%
0%0%
11%39%
0%17%6%0%
11%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 25
170
180
190
200
210
220
230
240
250
260
DermatologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure D-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
50% 50%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
60%
40%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
18%
82%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12%18%
71%
Exam Required?
N=17
N=18
N=17N=15
N=13 N=10
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2018 26
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
10%
90%
COMLEX-USA Level 1*Exam Required?
N=17
0%
20%
40%
60%
80%
100%
Yes No
35%
65%
USMLE Step 2 CSExam Required?
DermatologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure D-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 2-CE*Exam Required?
N=16
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%6%
81%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 2-PE*Exam Required?
N=17N=16
N=10 N=10
N=10 N=10
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 27
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
68%
32%
21%
5%
26%21%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure D-5DermatologyProgram's Interview Activities
0
100
200
300
400
500
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
4
485
38 36 34
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
34%
58%
Average Percentage of Applicants Rejected and Reviewed
N=21N=21N=21* N=21*N=21N=21
N=19
5% of Dermatology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
6%
33%
88%
47%
19%
59%67%
61%
13%
47%
81%
41%
94%
33%
6% 6%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
6%
33%
88%
47%
25%
59%53%
67%
13%
47%
75%
41%
94%
47%
6%
Often Seldom Never
Ranking
n=18n=18
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 28
Figure D-6DermatologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=18
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
3% 5%
70%
15%7%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
4% 0%5%
39%
53%
Percentage of Program's InterviewsConducted During Each Time Period
N=19 N=19
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
4.9
4.9
4.9
5.0
4.7
4.6
3.9
4.1
3.9
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 29
Emergency Medicine(N=92)
NRMP Program Director Survey Results, 2018 30
Figure EM-1Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=87)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
3.8
4.8
3.3
3.9
3.2
3.9
4.6
3.5
4.2
4.3
4.5
4.6
4.5
4.1
3.5
3.7
3.7
4.0
3.6
3.8
3.3
3.0
3.2
3.5
3.9
3.0
4.0
4.6
4.6
3.3
3.3
2.4
3.4
100% 50% 0%
97%
97%
83%
86%
62%
75%
83%
72%
76%
70%
85%
82%
61%
69%
59%
61%
64%
64%
64%
43%
51%
47%
45%
54%
55%
33%
36%
24%
75%20%
18%
11%
16%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 31
Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=87)
Figure EM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.94.8
4.53.7
4.73.9
3.54.6
4.14.04.1
3.44.3
4.73.1
3.84.0
4.64.3
3.73.43.7
3.53.8
3.13.6
3.23.9
3.14.34.2
3.73.3
4.04.5
3.33.7
2.5100% 50% 0%
94%98%94%
88%78%
89%76%
57%67%65%61%
67%56%
65%78%
50%59%55%56%
63%43%
46%30%
48%29%27%
41%30%33%
20%26%
16%23%17%
9%66%
23%17%
7%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 32
170
180
190
200
210
220
230
240
250
260
Emergency MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure EM-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
37%
61%
2%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
46%53%
1%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
32%
62%
6%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
17%
31%
51%
Exam Required?
N=87
N=84
N=86N=83
N=24N=49 N=39 N=20
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 33
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%
38%44%
COMLEX-USA Level 1*Exam Required?
N=84
0%
20%
40%
60%
80%
100%
Yes No
45%
55%
USMLE Step 2 CSExam Required?
Emergency MedicinePrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure EM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%23%
64%
COMLEX-USA Level 2-CE*Exam Required?
N=76
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 4%
93%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 3%
94%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%12%
71%
COMLEX-USA Level 2-PE*Exam Required?
N=84N=76
N=71 N=70
N=68 N=64
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 34
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 95%
74%81%
25%
7%
42%
14%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure EM-5Emergency MedicineProgram's Interview Activities
0
200
400
600
800
1,000
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
11
940
193146 136
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
30%
64%
Average Percentage of Applicants Rejected and Reviewed
N=86N=82N=92* N=90*N=90N=90
N=88
2% of Emergency Medicine programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=87
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1% 6% 5%
77%
34%22%
56%
2%
71%
35%
23%
64%
66%
44%
97%
24%
60%
2%12%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 7% 6%
77%
39%27%
60%
1%
66%
33%
23%
55%64%
39%
96%
27%
62%
6% 8%1%
Often Seldom Never
Ranking
n=85n=86
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 35
Figure EM-6Emergency MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=79
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
6%
60%
21%
8% 4%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
2%10%
36% 34%
18%
Percentage of Program's InterviewsConducted During Each Time Period
N=84 N=84
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.8
4.8
4.5
4.4
4.1
3.6
3.7
2.8
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 36
Family Medicine(N=152)
NRMP Program Director Survey Results, 2018 37
Figure FM-1Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=142)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
3.8
4.1
3.8
3.9
4.1
3.8
4.2
3.5
4.6
4.3
4.0
4.3
4.7
4.2
3.6
4.4
3.9
4.3
4.1
4.0
4.2
3.4
3.9
3.7
3.7
3.4
4.2
4.8
3.6
4.0
3.8
3.4
3.4
100% 50% 0%
89%
85%
75%
87%
87%
63%
80%
49%
92%
69%
53%
66%
68%
60%
34%
66%
61%
82%
63%
51%
70%
42%
39%
42%
42%
14%
56%
46%
16%
10%
41%8%
23%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 38
Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=142)
Figure FM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.84.8
3.74.03.94.0
4.74.6
4.34.1
3.64.4
4.33.94.0
4.34.24.1
4.03.5
3.94.24.1
3.83.73.73.9
3.64.6
4.43.7
3.94.1
3.63.83.7
3.3100% 50% 0%
96%96%
89%89%
72%67%
76%73%
53%86%
73%64%
44%62%
66%67%
46%72%
66%38%
54%27%
38%52%51%
33%33%
25%23%
9%28%
31%40%
8%37%
10%22%
31%
8%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 39
170
180
190
200
210
220
230
240
250
260
Family MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure FM-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
3%
47%50%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
7%
60%
33%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
62%
38%
1%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
40%36%
24%
Exam Required?
N=141
N=141
N=136N=140
N=38N=46 N=43 N=35
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 40
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
59%
36%
4%
COMLEX-USA Level 1*Exam Required?
N=135
0%
20%
40%
60%
80%
100%
Yes No
71%
29%
USMLE Step 2 CSExam Required?
Family MedicinePrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure FM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
40%34%
26%
COMLEX-USA Level 2-CE*Exam Required?
N=116
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 1%
95%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 2%
95%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
57%
12%
31%
COMLEX-USA Level 2-PE*Exam Required?
N=135N=116
N=137 N=135
N=132 N=111
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 41
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%99%
70%
94%
42%
24%
77%
44%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure FM-5Family MedicineProgram's Interview Activities
0
200
400
600
800
1,000
1,200
1,400
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
7
1,322
113 85 72
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
61%
30%
Average Percentage of Applicants Rejected and Reviewed
N=142N=142N=152* N=148*N=148N=148
N=142
14% of Family Medicine programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=147
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1% 1%
61%
35%
5%
35%
2%
65%
10%
32%
55%
40%
42%97%
34%
90%
6% 11%
55%
23%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1% 4%
62%
38%
6%
37%
61%
11%
31%
49%
39%
42%99%
35%
89%
7% 13%
55%
21%
Often Seldom Never
Ranking
n=142n=142
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 42
Figure FM-6Family MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=135
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
29%
41%
17%9%
4%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
18%
33% 31%
17%
Percentage of Program's InterviewsConducted During Each Time Period
N=133 N=133
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
4.9
4.9
4.9
4.6
4.3
4.4
3.7
3.9
2.8
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 43
Internal Medicine(N=181)
NRMP Program Director Survey Results, 2018 44
Figure IM-1Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=164)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.2
3.9
4.3
4.3
3.5
4.1
4.6
4.2
3.9
4.0
4.3
4.1
4.5
3.8
4.0
3.9
3.7
4.3
3.6
4.0
4.1
3.5
3.8
4.0
3.7
3.3
4.1
4.9
3.4
3.6
3.8
3.1
3.2
100% 50% 0%
95%
74%
85%
90%
70%
73%
81%
76%
44%
66%
66%
51%
62%
50%
53%
52%
49%
72%
44%
59%
54%
47%
46%
39%
37%
26%
49%
44%10%
21%
33%16%
24%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 45
Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=164)
Figure IM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.8
4.74.6
4.13.9
4.34.44.5
4.14.24.14.24.14.3
3.74.34.3
4.64.2
3.84.04.0
3.74.1
4.04.0
3.93.8
3.64.9
4.03.63.8
4.13.23.2
2.53.4
100% 50% 0%
94%93%
84%75%
86%66%
85%65%
83%43%
59%56%
71%58%
49%55%
61%67%62%
50%43%43%
48%34%
43%34%
37%30%28%
20%35%
44%11%17%
31%4%
21%4%
10%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 46
170
180
190
200
210
220
230
240
250
260
Internal MedicinePrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IM-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
23%
66%
11%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
33%
61%
6%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
27%
70%
2%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
17%
51%
32%
Exam Required?
N=165
N=163
N=161N=160
N=76N=110 N=97 N=67
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 47
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24%
57%
19%
COMLEX-USA Level 1*Exam Required?
N=159
0%
20%
40%
60%
80%
100%
Yes No
61%
39%
USMLE Step 2 CSExam Required?
Internal MedicinePrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20%
46%
35%
COMLEX-USA Level 2-CE*Exam Required?
N=133
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
2% 2%
96%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
2% 5%
93%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
35%
21%
44%
COMLEX-USA Level 2-PE*Exam Required?
N=159N=133
N=139 N=138
N=136 N=119
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 48
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 97%
69%
81%
48%
27%
77%
63%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure IM-5Internal MedicineProgram's Interview Activities
0
400
800
1,200
1,600
2,000
2,400
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
14
2,220
220 177 149
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
61%
32%
Average Percentage of Applicants Rejected and Reviewed
N=168N=163N=181* N=172*N=173N=174
N=166
25% of Internal Medicine programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=172
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
9% 9%
64%
29%
9%18%
12%
60%
21%
34%
60%
40%
42%
88%
31%
70%
2%11%
51%39%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
10% 10%
66%
29%
10%20%
9%
60%
19%
31%
60%
39%
41%
91%
30%
71%
3%12%
51%39%
Often Seldom Never
Ranking
n=165n=165
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 49
Figure IM-6Internal MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=146
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
20%
51%
17%9%
4%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
13%
34% 32%
21%
Percentage of Program's InterviewsConducted During Each Time Period
N=161 N=161
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.9
4.8
4.6
4.4
4.4
3.9
3.7
3.1
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 50
Internal Medicine/Pediatrics(N=27)
NRMP Program Director Survey Results, 2018 51
Figure IMP-1Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
3.8
4.1
4.2
3.5
4.1
4.4
3.8
3.8
4.0
4.3
4.0
4.5
3.8
3.8
3.7
3.5
3.7
3.9
4.0
3.9
3.5
3.4
4.0
3.8
4.0
3.5
4.4
2.7
4.0
3.5
2.2
4.0
100% 50% 0%
100%
78%
81%
74%
63%
85%
67%
74%
67%
52%
85%
33%
44%
63%
70%
33%
37%
44%
59%
48%
41%
41%
52%
67%
44%
11%
22%
19%
11%
11%
7%
19%
4%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 52
Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)
Figure IMP-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.84.8
4.64.5
3.64.4
4.24.4
3.94.14.2
4.03.9
4.13.1
4.24.1
4.84.4
3.83.93.9
4.13.3
4.23.94.1
3.83.1
4.03.5
3.03.7
2.73.0
2.53.0
100% 50% 0%
93%96%93%89%89%
67%85%
70%78%
63%48%
78%85%
59%33%
56%67%
59%52%
63%48%
67%37%
63%30%
37%74%
33%33%
41%15%
7%
7%15%11%7%
0%7%
15%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 53
170
180
190
200
210
220
230
240
250
260
Internal Medicine/PediatricsPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IMP-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
35%
62%
4%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
48%52%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
23%
73%
4%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
15%
31%
54%
Exam Required?
N=26
N=26
N=26N=25
N=8N=19 N=19 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 54
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
17%
35%
48%
COMLEX-USA Level 1*Exam Required?
N=26
0%
20%
40%
60%
80%
100%
Yes No
50% 50%
USMLE Step 2 CSExam Required?
Internal Medicine/PediatricsPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IMP-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
17% 17%
67%
COMLEX-USA Level 2-CE*Exam Required?
N=24
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 4%
92%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%0%
91%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
33%
0%
67%
COMLEX-USA Level 2-PE*Exam Required?
N=26N=24
N=23 N=24
N=24 N=22
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 55
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
12%
65%
12%
0%
42%
19%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure IMP-5Internal Medicine/PediatricsProgram's Interview Activities
0
40
80
120
160
200
240
280
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
5
260
85
66 64
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
40%
56%
Average Percentage of Applicants Rejected and Reviewed
N=24N=23N=27* N=26*N=26N=25
N=26
0% of Internal Medicine/Pediatrics programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=25
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
19%12%
88%
64%
19%
50%
77%
42%
12%
32%
58%
35%
100%
4%
46%
4%
23%15%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
17% 12%
88%
67%
20%
48%
75%
32%
8%
25%
56%
40%
100%
8%
56%
4% 8%
24%12%
Often Seldom Never
Ranking
n=26n=25
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 56
Figure IMP-6Internal Medicine/PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
35%
52%
8%3% 2%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
11%
39%35%
15%
Percentage of Program's InterviewsConducted During Each Time Period
N=24 N=24
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.8
4.9
4.8
4.8
4.8
3.9
4.2
4.2
3.4
2.9
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 57
Interventional Radiology(N=18)
NRMP Program Director Survey Results, 2018 58
Figure IR-1Interventional RadiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=15)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.3
4.5
4.5
4.1
3.6
4.4
4.5
4.3
4.9
3.9
4.6
4.1
4.2
4.2
3.9
4.7
3.8
4.1
3.8
4.2
3.0
4.5
4.0
4.0
4.7
3.8
4.3
5.0
3.6
3.8
4.5
3.9
4.3
100% 50% 0%
100%
100%
93%
73%
93%
87%
40%
73%
87%
73%
60%
73%
67%
87%
87%
80%
60%
67%
73%
67%13%
87%
73%
47%
47%
80%
27%
33%
67%
33%
13%
47%20%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 59
Interventional RadiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=15)
Figure IR-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
5.04.94.85.0
4.74.54.64.54.64.85.0
4.14.5
4.44.4
4.04.7
4.34.5
4.33.7
4.34.6
3.64.0
3.84.34.44.5
4.25.05.0
4.04.3
5.03.3
4.33.5
4.6100% 50% 0%
93%86%86%
64%71%
79%57%
64%79%79%79%
64%64%64%
71%36%
71%43%
29%50%
21%50%
36%50%
7%43%
36%57%
29%43%
14%
14%29%
21%7%
29%21%14%
36%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 60
170
180
190
200
210
220
230
240
250
260
Interventional RadiologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IR-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
64%
36%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
43%
57%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
86%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
0%
57%
43%
Exam Required?
N=14
N=14
N=14N=14
N=5N=12 N=10
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2018 61
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%
55%
36%
COMLEX-USA Level 1*Exam Required?
N=14
0%
20%
40%
60%
80%
100%
Yes No
57%
43%
USMLE Step 2 CSExam Required?
Interventional RadiologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IR-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
45%55%
COMLEX-USA Level 2-CE*Exam Required?
N=12
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
8%17%
75%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
20%
80%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%
36%
55%
COMLEX-USA Level 2-PE*Exam Required?
N=14N=12
N=11 N=11
N=11 N=10
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 62
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 93%
43%
71%
50%
0%
57%
29%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure IR-5Interventional RadiologyProgram's Interview Activities
0
40
80
120
160
200
240
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
2
236
36 34 30
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
40%
60%
Average Percentage of Applicants Rejected and Reviewed
N=15N=16N=18* N=18*N=18N=18
N=14
0% of Interventional Radiology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=18
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
7% 7%
79%
36% 33%
57%
7%
67%
36%
21%
50%47%
36%
93%
27%
57%
14% 20%7%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
13%7% 7%
79%
36% 33%
57%
60%
36%
21%
43% 47%
36%
87%
33%
57%
21% 20%7%
Often Seldom Never
Ranking
n=15n=14
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 63
Figure IR-6Interventional RadiologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=13
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
13%
49%
22%12%
3%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
13%
33%27% 27%
Percentage of Program's InterviewsConducted During Each Time Period
N=15 N=15
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.8
4.8
4.8
4.8
4.7
4.9
4.7
4.7
3.8
4.4
3.8
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 64
Neurological Surgery(N=28)
NRMP Program Director Survey Results, 2018 65
Figure NS-1Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=25)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.6
4.6
3.7
3.9
3.5
4.3
4.5
4.1
4.5
4.6
4.4
4.3
4.7
4.4
4.0
4.3
3.7
4.3
3.8
3.9
3.8
4.1
3.7
4.0
4.3
4.6
3.7
4.8
4.1
4.5
4.0
3.7
3.5
100% 50% 0%
100%
100%
76%
64%
72%
76%
48%
80%
80%
80%
80%
88%
84%
76%
88%
60%
72%
36%
56%
76%
48%
72%
68%
12%
64%
76%
52%
40%
68%
44%
12%
52%
8%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 66
Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=25)
Figure NS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.95.04.9
4.84.1
4.44.0
3.74.7
4.54.54.4
4.14.6
4.43.4
3.94.4
5.04.2
3.63.93.93.94.0
3.84.0
3.63.8
4.44.7
3.74.0
4.34.5
4.14.04.0
3.4100% 50% 0%
96%88%88%
80%84%84%
52%64%
44%76%
60%60%
44%64%
72%52%
60%32%28%
56%52%
36%48%44%
28%40%
12%36%
44%60%
24%24%12%
48%8%
40%4%
28%28%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 67
170
180
190
200
210
220
230
240
250
260
Neurological SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure NS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
58%
38%
4%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
43%
52%
5%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
16%
84%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
29%25%
46%
Exam Required?
N=25
N=24
N=24N=21
N=6N=21 N=16 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 68
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18% 18%
64%
COMLEX-USA Level 1*Exam Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
54%46%
USMLE Step 2 CSExam Required?
Neurological SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure NS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%9%
73%
COMLEX-USA Level 2-CE*Exam Required?
N=19
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5% 5%
89%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%
0%
82%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%
0%
73%
COMLEX-USA Level 2-PE*Exam Required?
N=24N=19
N=11 N=11
N=11 N=11
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 69
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 96%
56%
20%16%
4%
40%32%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure NS-5Neurological SurgeryProgram's Interview Activities
0
40
80
120
160
200
240
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
2
216
4736
29
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
40%
58%
Average Percentage of Applicants Rejected and Reviewed
N=25N=25N=28* N=27*N=27N=26
N=25
4% of Neurological Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4% 4%
58%
78%
29%
13%22%
83%
33%
22%
67%
79% 65%96%
13% 8% 4% 8% 13%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4% 4%
58%
83%
33%
13%22%
75%
29%
17%
63%
74%70%
96%
21%13%
4%13% 9%
Often Seldom Never
Ranking
n=24n=24
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 70
Figure NS-6Neurological SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=25
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
8%
59%
7%
18%
8%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%7%
36% 34%
24%
Percentage of Program's InterviewsConducted During Each Time Period
N=24 N=24
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
4.9
5.0
5.0
4.5
5.0
4.2
4.5
3.8
4.1
4.3
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 71
Neurology(N=41)
NRMP Program Director Survey Results, 2018 72
Figure N-1NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=37)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
4.2
4.1
4.0
4.1
4.3
4.3
4.2
4.6
4.4
4.3
3.8
4.6
4.0
3.7
4.5
3.7
4.3
3.8
4.3
4.3
3.7
4.0
3.7
3.8
3.5
4.2
4.7
3.4
4.2
4.1
3.3
3.4
100% 50% 0%
89%
81%
78%
78%
73%
86%
89%
78%
65%
68%
84%
51%
62%
54%
62%
46%
59%
49%
43%
49%
78%
57%
62%
51%
51%
35%
27%
43%
19%
30%
24%22%
14%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 73
NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=37)
Figure N-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.84.7
4.14.14.24.1
4.64.6
4.34.24.14.34.3
3.84.14.1
4.44.3
3.63.7
4.23.8
4.34.1
3.94.0
4.33.2
4.83.83.7
4.24.4
5.04.34.3
3.3100% 50% 0%
100%100%
94%97%
83%77%
66%74%
77%80%
57%60%
69%54%
34%60%60%
49%60%
46%46%
40%46%
37%46%
40%29%
31%29%
34%26%26%17%
26%29%
3%17%11%
11%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 74
170
180
190
200
210
220
230
240
250
260
NeurologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure N-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
27%
59%
14%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
36%
58%
6%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
38%
62%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
25%33%
42%
Exam Required?
N=37
N=37
N=36N=36
N=10N=22 N=18 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 75
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
22%
48%
30%
COMLEX-USA Level 1*Exam Required?
N=36
0%
20%
40%
60%
80%
100%
Yes No
56%
44%
USMLE Step 2 CSExam Required?
NeurologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure N-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
19% 19%
62%
COMLEX-USA Level 2-CE*Exam Required?
N=35
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
8%0%
92%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24%
8%
68%
COMLEX-USA Level 2-PE*Exam Required?
N=36N=35
N=27 N=26
N=25 N=26
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 76
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 97%
64%
74% 72%
21%
85%79%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure N-5NeurologyProgram's Interview Activities
0
100
200
300
400
500
600
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
6
502
8565 58
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
50%47%
Average Percentage of Applicants Rejected and Reviewed
N=40N=37N=41* N=40*N=40N=40
N=39
21% of Neurology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=40
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
14% 16%
63%
24%
8% 5%3%
46%
21%
29%
51%
39% 47%
97%
41%
63%
8%
24%
53% 47%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
13%19%
66%
24%
8% 5%3%
50%
19%
29%
45%
39% 41%
97%
37%
61%
5%
32%
53% 54%
Often Seldom Never
Ranking
n=37n=38
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 77
Figure N-6NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=36
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
31%
48%
11% 9%1%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
17%
37%31%
14%
Percentage of Program's InterviewsConducted During Each Time Period
N=36 N=36
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
4.9
4.9
4.8
4.8
4.6
4.1
4.4
4.3
3.4
3.2
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 78
Obstetrics and Gynecology(N=85)
NRMP Program Director Survey Results, 2018 79
Figure OG-1Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=79)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
4.0
3.9
4.1
3.6
4.3
4.7
4.0
4.1
4.1
4.4
4.2
4.5
4.3
3.8
4.0
3.8
4.3
3.6
4.1
3.9
3.4
3.6
3.8
3.7
3.6
4.5
4.7
3.1
3.8
3.4
3.1
3.3
100% 50% 0%
94%
87%
87%
78%
81%
85%
66%
78%
53%
57%
73%
65%
65%
63%
66%
57%
61%
41%
54%
47%
49%
52%
39%
59%
49%
41%
27%
24%10%
23%
19%16%8%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 80
Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=79)
Figure OG-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.94.94.8
4.03.94.1
4.04.6
4.24.0
4.34.0
4.34.2
3.54.0
4.34.7
3.93.8
3.74.0
3.94.2
3.53.93.93.9
3.54.6
4.13.9
3.63.0
3.54.0
3.63.0
100% 50% 0%
96%95%93%
85%67%
51%61%63%
56%44%
51%57%
51%45%
52%43%
49%36%
25%36%
41%39%
32%24%20%
25%35%
20%20%20%15%11%
28%17%
9%3%8%12%1%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 81
170
180
190
200
210
220
230
240
250
260
Obstetrics and GynecologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure OG-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
34%
65%
1%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
51% 49%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
23%
76%
1%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
22%
51%
27%
Exam Required?
N=79
N=79
N=78N=77
N=31N=54 N=43 N=26
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 82
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%
37%
53%
COMLEX-USA Level 1*Exam Required?
N=78
0%
20%
40%
60%
80%
100%
Yes No
65%
35%
USMLE Step 2 CSExam Required?
Obstetrics and GynecologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure OG-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%
34%
55%
COMLEX-USA Level 2-CE*Exam Required?
N=58
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 2%
95%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
2% 2%
96%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%
15%
56%
COMLEX-USA Level 2-PE*Exam Required?
N=78N=58
N=59 N=58
N=59 N=47
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 83
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 97%
40%
63%
24%
4%
28%19%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure OG-5Obstetrics and GynecologyProgram's Interview Activities
0
100
200
300
400
500
600
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
6
534
9174 67
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
50%46%
Average Percentage of Applicants Rejected and Reviewed
N=82N=79N=85* N=84*N=83N=83
N=78
4% of Obstetrics and Gynecology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=82
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1%11% 6%
83%
46%
23%
46%
3%
74%
48%
17%
46%
67%
49%
96%
16%
45%
8% 10% 5%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1%15% 9%
83%
47%
28%
49%
3%
67%
45%
17%
45%
60%
47%
96%
19%
45%
8% 12%4%
Often Seldom Never
Ranking
n=76n=77
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 84
Figure OG-6Obstetrics and GynecologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=74
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
20%
60%
13%5%
2%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
15%
42%36%
6%
Percentage of Program's InterviewsConducted During Each Time Period
N=79 N=79
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.9
4.8
4.8
4.3
4.2
4.1
3.7
3.2
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 85
Orthopaedic Surgery(N=49)
NRMP Program Director Survey Results, 2018 86
Figure OS-1Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=47)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.5
4.3
3.6
4.1
3.5
4.2
4.8
4.1
4.2
4.1
4.3
4.6
4.6
4.3
4.0
4.0
3.6
4.0
4.1
4.1
3.9
4.1
3.8
3.5
4.0
3.9
4.6
4.9
3.9
3.1
4.0
3.5
5.0
100% 50% 0%
91%
89%
72%
53%
72%
79%
28%
70%
49%
74%
64%
77%
64%
72%
81%
45%
62%
19%
55%
47%
36%
64%
47%
32%
43%
66%
17%
17%
34%
17%
4%
32%2%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 87
Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=47)
Figure OS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.84.8
4.14.14.1
3.64.3
3.94.24.3
4.04.2
4.53.4
4.34.0
4.64.2
3.63.9
4.53.6
4.23.9
3.74.14.2
3.45.0
4.74.0
2.94.0
4.33.5
4.03.6
100% 50% 0%
91%91%87%
78%71%69%
47%51%
44%38%
42%49%44%44%
69%44%44%
16%
18%33%36%
53%36%
42%11%
33%16%
36%27%
40%9%7%
2%22%
2%20%4%7%
18%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 88
170
180
190
200
210
220
230
240
250
260
Orthopaedic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure OS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
65%
35%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
63%
37%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
2%
98%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
2%
45%
53%
Exam Required?
N=48
N=46
N=47N=41
N=18N=45 N=34 N=16
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 89
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
19%
81%
COMLEX-USA Level 1*Exam Required?
N=46
0%
20%
40%
60%
80%
100%
Yes No
39%
61%
USMLE Step 2 CSExam Required?
Orthopaedic SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure OS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
19%
81%
COMLEX-USA Level 2-CE*Exam Required?
N=43
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
2%7%
91%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
12%
88%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
15%
85%
COMLEX-USA Level 2-PE*Exam Required?
N=46N=43
N=26 N=26
N=26 N=25
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 90
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 98%
46%
15% 13%4%
10%6%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure OS-5Orthopaedic SurgeryProgram's Interview Activities
0
100
200
300
400
500
600
700
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
5
641
72 62 55
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
51%47%
Average Percentage of Applicants Rejected and Reviewed
N=47N=46N=49* N=49*N=49N=48
N=48
4% of Orthopaedic Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=48
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2%10%
67%
85%
60% 60%
79%71%
27%
15%
38% 36%
21%
98%
19%6% 2% 4%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
11%
68%
89%
62% 67%82%
72%
26%
11%
36% 30%
18%
100%
17%6% 2% 2%
Often Seldom Never
Ranking
n=48n=48
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 91
Figure OS-6Orthopaedic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=46
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
3%
21%
52%
20%
4%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
2% 1%10%
33%
54%
Percentage of Program's InterviewsConducted During Each Time Period
N=48 N=48
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.8
4.9
4.7
4.9
4.2
4.1
3.6
3.8
3.8
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 92
Otolaryngology(N=35)
NRMP Program Director Survey Results, 2018 93
Figure O-1OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=34)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
4.5
3.8
3.7
3.7
4.2
4.7
3.8
4.3
4.6
4.3
4.5
4.3
3.9
3.9
4.1
3.6
3.8
3.5
3.8
3.2
3.7
3.7
3.6
3.7
4.0
4.1
4.2
3.4
3.4
3.0
3.3
100% 50% 0%
97%
97%
74%
56%
91%
88%
47%
68%
76%
74%
74%
71%
65%
79%
88%
74%
65%
32%
62%
56%
26%
68%
41%
50%
65%
74%
26%
32%
41%
29%
15%
41%
0%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 94
OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=34)
Figure O-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.84.7
3.94.3
3.63.7
4.64.44.34.3
4.24.74.7
3.43.9
3.34.03.93.9
3.84.0
3.9
3.53.83.73.84.0
4.43.3
3.04.1
3.54.0
4.03.3
100% 50% 0%
97%91%
97%91%
56%85%
29%74%
35%59%
68%62%
38%65%
59%47%
50%18%
6%44%44%
56%24%24%
0%24%
15%29%29%
56%15%12%
9%24%12%18%0%6%
12%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 95
170
180
190
200
210
220
230
240
250
260
OtolaryngologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure O-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
56%
44%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
58%
42%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
35%
65%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12%18%
70%
Exam Required?
N=34
N=32
N=33N=26
N=21 N=16
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2018 96
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
12%6%
82%
COMLEX-USA Level 1*Exam Required?
N=33
0%
20%
40%
60%
80%
100%
Yes No
33%
67%
USMLE Step 2 CSExam Required?
OtolaryngologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure O-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
COMLEX-USA Level 2-CE*Exam Required?
N=28
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
COMLEX-USA Level 2-PE*Exam Required?
N=33N=28
N=17 N=17
N=17 N=17
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 97
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
79%
29%
12%6%
29%
18%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure O-5OtolaryngologyProgram's Interview Activities
0
40
80
120
160
200
240
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
3
219
58
42 39
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
14%
86%
Average Percentage of Applicants Rejected and Reviewed
N=35N=32N=35* N=35*N=35N=35
N=34
0% of Otolaryngology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=35
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
3%
38%
82%
47%
24%
44%
71%
56%
18%
53%
74%
53%
100%
26%
6% 3% 3%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
3%
44%
82%
47%
24%
47%
68%
50%
18%
47%
74%
50%
100%
29%
6% 6% 3% 3%
Often Seldom Never
Ranking
n=34n=34
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 98
Figure O-6OtolaryngologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=34
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
44% 40%
10% 6%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1% 1%10%
47%42%
Percentage of Program's InterviewsConducted During Each Time Period
N=35 N=35
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
4.9
5.0
4.8
4.8
4.2
4.3
3.6
3.7
3.7
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 99
Pathology(N=43)
NRMP Program Director Survey Results, 2018 100
Figure PA-1PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=40)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
4.6
4.1
3.9
4.0
4.1
4.6
3.8
4.4
4.1
4.2
4.0
4.5
4.0
3.8
4.1
3.6
3.7
3.5
4.2
4.0
3.7
4.1
3.9
3.7
3.5
3.9
4.9
3.9
3.5
4.6
3.6
3.3
100% 50% 0%
90%
95%
75%
85%
80%
75%
78%
65%
88%
60%
63%
78%
63%
50%
45%
50%
38%
65%
30%
53%
60%
38%
55%
23%
40%
50%
38%
48%
35%
33%
18%
30%25%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 101
PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=40)
Figure PA-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.94.94.8
4.24.04.2
4.04.6
4.34.3
4.13.9
4.34.3
3.74.14.1
4.63.9
3.64.2
3.73.4
4.14.5
4.03.4
3.93.6
4.83.7
3.43.4
4.74.0
3.34.04.0
100% 50% 0%
97%97%95%
87%74%
85%67%64%
72%87%
49%46%
59%46%
59%59%
41%46%
56%38%
28%31%
38%21%
41%28%
13%21%
26%23%
36%23%21%
28%18%
18%18%
3%15%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 102
170
180
190
200
210
220
230
240
250
260
PathologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PA-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
28%
68%
5%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
21%
74%
5%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
46%54%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
26%
38% 36%
Exam Required?
N=39
N=40
N=39N=39
N=11N=17 N=14 N=10
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 103
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
35%
51%
14%
COMLEX-USA Level 1*Exam Required?
N=38
0%
20%
40%
60%
80%
100%
Yes No
55%
45%
USMLE Step 2 CSExam Required?
PathologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PA-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
22%
38% 41%
COMLEX-USA Level 2-CE*Exam Required?
N=33
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 3%
94%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 3%
94%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%22%
51%
COMLEX-USA Level 2-PE*Exam Required?
N=38N=33
N=37 N=37
N=37 N=32
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 104
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 98%90% 93%
78%
48%
85%90%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure PA-5PathologyProgram's Interview Activities
0
50
100
150
200
250
300
350
400
450
500
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
5
454
7150 41
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
46%52%
Average Percentage of Applicants Rejected and Reviewed
N=40N=38N=43* N=42*N=42N=41
N=40
43% of Pathology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=41
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
3% 3%
41%
10%3%
36%
23%
49%
59%
38%23%
97%
62%
77%
11%
31%
62%74%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
43%
11%3% 3%
43%
24%
49%
58%
37%24%
100%
57%
76%
9%
32%
61%74%
Often Seldom Never
Ranking
n=39n=39
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 105
Figure PA-6PathologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=38
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
33%
46%
15%
5% 1%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
18%
39%30%
13%
Percentage of Program's InterviewsConducted During Each Time Period
N=40 N=40
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
4.9
4.8
4.9
4.6
4.7
4.2
4.6
3.5
4.0
3.4
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 106
Pediatrics(N=78)
NRMP Program Director Survey Results, 2018 107
Figure P-1PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=73)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
3.9
4.3
3.9
3.5
4.1
4.5
3.8
4.0
4.1
4.1
4.0
4.5
4.2
4.0
3.9
3.8
4.0
4.0
4.0
3.9
3.5
3.8
4.0
3.5
3.3
3.9
4.8
3.5
3.3
3.5
2.5
3.7
100% 50% 0%
95%
73%
81%
82%
73%
77%
78%
75%
51%
70%
64%
75%
53%
53%
70%
45%
56%
60%
58%
55%
60%
42%
55%
71%
41%
26%
36%
42%15%
10%
30%
22%
15%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 108
PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=73)
Figure P-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.9
4.74.0
3.94.0
4.44.5
4.13.8
4.24.24.14.1
3.44.14.3
4.64.2
3.94.03.93.93.83.83.93.93.9
3.34.8
3.53.3
3.83.6
3.04.2
3.42.8
100% 50% 0%
99%96%94%93%
86%66%
87%72%
77%51%51%
66%72%73%
76%59%
65%79%
61%62%59%
65%55%
63%58%54%
58%34%35%
24%37%
30%27%
14%28%
17%15%
23%20%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 109
170
180
190
200
210
220
230
240
250
260
PediatricsPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure P-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
4%
79%
16%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
15%
79%
5%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
56%
40%
4%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
24% 23%
53%
Exam Required?
N=72
N=73
N=70N=73
N=14N=27 N=24 N=12
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 110
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
39% 41%
20%
COMLEX-USA Level 1*Exam Required?
N=70
0%
20%
40%
60%
80%
100%
Yes No
44%
56%
USMLE Step 2 CSExam Required?
PediatricsPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure P-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
19% 22%
59%
COMLEX-USA Level 2-CE*Exam Required?
N=60
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 0%
97%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%0%
95%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
28%
9%
63%
COMLEX-USA Level 2-PE*Exam Required?
N=70N=60
N=66 N=64
N=64 N=56
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 111
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 97%
65%
87%
56%
16%
67%61%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure P-5PediatricsProgram's Interview Activities
0
200
400
600
800
1,000
1,200
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
16
1,174
249190 178
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
53%
43%
Average Percentage of Applicants Rejected and Reviewed
N=70N=70N=78* N=77*N=77N=75
N=75
13% of Pediatrics programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=76
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
5% 3%
73%
14%7% 12%
3%
72%
23%
24%
70%
57%58%
97%
23%
74%
3%16%
36%30%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
6% 4%
76%
17%7%
14%3%
66%
19%
21%
62%
56%55%
97%
28%
77%
3%
21%
37% 32%
Often Seldom Never
Ranking
n=74n=74
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 112
Figure P-6PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=70
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
37%45%
10%6% 2%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
18%
34% 32%
15%
Percentage of Program's InterviewsConducted During Each Time Period
N=71 N=71
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
4.9
4.8
4.8
4.6
4.3
4.2
4.1
3.4
2.8
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 113
Physical Medicine and Rehabilitation(N=28)
NRMP Program Director Survey Results, 2018 114
Figure PMR-1Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=23)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.2
4.3
4.4
4.2
3.9
4.0
4.8
3.8
4.7
4.0
4.1
3.9
4.6
4.3
4.1
4.1
3.5
4.3
3.7
4.3
4.3
3.2
3.6
4.0
3.5
2.9
3.5
4.6
3.3
2.7
3.0
3.3
3.6
100% 50% 0%
96%
96%
83%
78%
87%
70%
70%
70%
87%
65%
78%
70%
83%
65%
48%
70%
52%
65%
65%
70%
78%
39%
48%
52%
48%
43%
35%
30%
35%
26%
4%
43%48%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 115
Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=23)
Figure PMR-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.84.8
4.54.04.03.9
4.24.6
4.34.14.4
3.84.24.4
3.94.14.2
4.84.5
3.94.3
4.23.8
4.43.94.0
3.43.9
3.04.8
3.73.4
3.03.5
4.03.84.0
3.5100% 50% 0%
100%100%96%96%91%
78%87%
70%83%
87%61%61%
74%57%
70%78%
61%74%
57%52%48%
39%61%
70%61%
48%43%
35%30%
35%26%
30%30%26%
9%35%
43%22%
26%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 116
170
180
190
200
210
220
230
240
250
260
Physical Medicine and RehabilitationPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PMR-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
8%
79%
13%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
13%
74%
13%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
38%
54%
8%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
29%25%
46%
Exam Required?
N=24
N=24
N=24N=23
N=5N=12 N=11 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 117
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
38%
50%
13%
COMLEX-USA Level 1*Exam Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
46%54%
USMLE Step 2 CSExam Required?
Physical Medicine and RehabilitationPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PMR-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
33%25%
42%
COMLEX-USA Level 2-CE*Exam Required?
N=22
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
14%5%
82%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
14%5%
82%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
42%
13%
46%
COMLEX-USA Level 2-PE*Exam Required?
N=24N=22
N=24 N=24
N=24 N=22
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 118
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
63%
100%
33%
17%
42%
21%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure PMR-5Physical Medicine and RehabilitationProgram's Interview Activities
0
50
100
150
200
250
300
350
400
450
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
4
416
7764
50
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
41%
57%
Average Percentage of Applicants Rejected and Reviewed
N=27N=27N=28* N=28*N=28N=28
N=24
8% of Physical Medicine and Rehabilitation programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=27
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
70%
29%
13%
50%
4%
71%
26%
67%
75%
46%
96%
29%
100%
4% 4%13%
4%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4%
74%
33%
17%
50%
4%
75%
22%
63%
71%
46%
96%
21%
100%
4% 4%13%
4%
Often Seldom Never
Ranking
n=24n=24
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 119
Figure PMR-6Physical Medicine and RehabilitationProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=23
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
14%
47%
30%
8%0%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%8%
38% 37%
15%
Percentage of Program's InterviewsConducted During Each Time Period
N=22 N=22
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
5.0
4.8
4.9
4.7
4.5
4.5
3.7
3.7
3.4
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 120
Plastic Surgery(N=22)
NRMP Program Director Survey Results, 2018 121
Figure PS-1Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=21)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.3
4.8
3.1
4.0
3.3
4.1
4.7
3.8
4.1
4.6
4.2
4.8
4.7
4.0
4.1
3.7
3.7
5.0
3.7
3.8
3.0
3.5
3.9
3.3
4.0
4.1
5.0
4.8
4.2
3.8
3.0
3.5
100% 50% 0%
86%
95%
48%
57%
62%
71%
48%
57%
71%
81%
71%
67%
62%
52%
76%
33%
57%
10%
48%
43%
24%
52%
48%
14%
33%
90%
14%
19%
43%
43%
5%
10%0%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 122
Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=21)
Figure PS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.84.94.8
3.94.6
3.23.7
4.64.5
4.04.3
3.75.0
4.73.4
4.74.0
5.04.4
3.74.14.0
3.84.0
4.34.04.2
4.54.2
4.55.0
3.8
4.0
3.5100% 50% 0%
95%90%86%86%
52%90%
24%48%
14%67%
52%38%33%
57%71%
24%29%
5%10%
24%33%
43%5%
24%5%
29%10%
29%10%
67%10%
14%0%
43%0%
19%
0%0%
10%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 123
170
180
190
200
210
220
230
240
250
260
Plastic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
80%
20%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
81%
19%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
15%
85%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
10%
35%
55%
Exam Required?
N=20
N=20
N=20N=16
N=7N=17 N=16 N=6
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 124
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20% 20%
60%
COMLEX-USA Level 1*Exam Required?
N=20
0%
20%
40%
60%
80%
100%
Yes No
40%
60%
USMLE Step 2 CSExam Required?
Plastic SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20% 20%
60%
COMLEX-USA Level 2-CE*Exam Required?
N=19
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
11%
0%
89%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20%
0%
80%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20% 20%
60%
COMLEX-USA Level 2-PE*Exam Required?
N=20N=19
N=10 N=10
N=10 N=10
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 125
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
68%
26%32%
16%
42%
26%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure PS-5Plastic SurgeryProgram's Interview Activities
0
40
80
120
160
200
240
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
2
211
4232 28
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
33%
70%
Average Percentage of Applicants Rejected and Reviewed
N=19N=19N=22* N=21*N=21N=19
N=19
11% of Plastic Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
5%
37%
74%
32%21%
53%
10%
80%
58%
26%
63%74%
42%
90%
15%5% 5% 5% 5%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
5%
42%
74%
32% 26%
56%
10%
70%
53%
26%
63% 68%
39%
90%
25%
5% 5% 5% 6%
Often Seldom Never
Ranking
n=20n=20
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 126
Figure PS-6Plastic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
40% 40%
18%
2%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0% 0%
13%
30%
57%
Percentage of Program's InterviewsConducted During Each Time Period
N=20 N=20
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
5.0
5.0
5.0
4.7
4.9
4.2
4.4
3.9
4.0
4.0
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 127
Psychiatry(N=61)
NRMP Program Director Survey Results, 2018 128
Figure PSY-1PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=58)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
3.6
4.0
4.3
3.7
4.2
4.0
4.4
4.0
4.4
4.3
4.2
4.2
4.5
4.2
3.7
4.4
3.9
4.1
3.9
4.2
4.2
3.7
3.6
3.9
3.8
3.5
4.4
4.9
3.6
3.7
4.3
2.7
3.4
100% 50% 0%
91%
84%
91%
84%
90%
69%
74%
62%
84%
76%
67%
71%
66%
40%
43%
60%
60%
64%
52%
43%
74%
40%
50%
52%
45%
34%
47%
53%16%
28%
31%
17%26%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 129
PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=58)
Figure PSY-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.94.94.8
3.83.93.9
4.34.6
4.44.14.1
3.84.34.4
3.84.04.24.24.1
3.84.04.0
3.83.93.93.9
3.63.9
3.74.7
4.43.7
4.14.1
3.03.2
3.52.4
100% 50% 0%
100%96%93%
96%75%
67%81%
67%72%
88%77%
58%58%
63%60%
74%47%
70%65%
46%56%
42%44%42%
72%44%40%
32%30%
35%40%
26%28%
21%33%
16%26%
19%12%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 130
170
180
190
200
210
220
230
240
250
260
PsychiatryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PSY-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
3%
62%
34%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
14%
55%
31%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
62%
36%
2%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
26%21%
53%
Exam Required?
N=58
N=58
N=57N=58
N=9N=18 N=16 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 131
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
48%
33%
19%
COMLEX-USA Level 1*Exam Required?
N=55
0%
20%
40%
60%
80%
100%
Yes No
40%
60%
USMLE Step 2 CSExam Required?
PsychiatryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PSY-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24%18%
59%
COMLEX-USA Level 2-CE*Exam Required?
N=48
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%2%
92%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%0%
95%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%
8%
65%
COMLEX-USA Level 2-PE*Exam Required?
N=55N=48
N=54 N=51
N=51 N=44
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 132
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 97%
64%
88%
44%
20%
71%
49%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure PSY-5PsychiatryProgram's Interview Activities
0
200
400
600
800
1,000
1,200
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
8
1,091
119 94 80
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
48%
36%
Average Percentage of Applicants Rejected and Reviewed
N=59N=56N=61* N=61*N=60N=61
N=59
14% of Psychiatry programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=60
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 5% 7%
61%
33%
4%
26%
3%
60%
16%
35%
42%
46%
42%95%
35%
77%
4%
25%
51%
32%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 5% 7%
66%
36%
4%
27%
5%
62%
16%
29%
45%
46%
48%93%
33%
77%
5%20%
50%
25%
Often Seldom Never
Ranking
n=57n=58
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 133
Figure PSY-6PsychiatryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=56
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
19%
54%
18%
6%2%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
18%
31% 30%
20%
Percentage of Program's InterviewsConducted During Each Time Period
N=58 N=58
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.9
4.8
4.4
4.4
4.0
4.1
3.4
3.1
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 134
Radiation Oncology(N=27)
NRMP Program Director Survey Results, 2018 135
Figure RO-1Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=26)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
3.9
4.5
4.0
3.5
4.0
4.5
4.6
4.0
4.7
4.4
4.7
4.6
4.7
4.3
3.6
4.3
3.9
3.8
3.7
4.2
3.8
4.0
3.5
3.8
4.4
4.4
4.3
4.6
3.4
4.4
3.4
3.5
4.0
100% 50% 0%
100%
100%
81%
62%
92%
77%
50%
81%
92%
65%
58%
69%
73%
58%
62%
77%
65%
23%
58%
77%
19%
54%
62%
35%
42%
96%
35%
35%
50%
54%
31%
31%8%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 136
Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=26)
Figure RO-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.9
4.84.6
3.84.4
3.84.0
4.74.54.4
4.14.0
4.54.3
3.74.1
4.74.34.3
4.03.6
4.23.84.0
3.84.0
3.54.24.34.4
4.25.0
4.24.04.1
5.05.0
3.2100% 50% 0%
100%96%92%92%
69%77%
31%62%
54%65%
73%46%
50%58%58%58%
50%12%12%
42%31%31%
42%23%12%
23%15%
23%23%
69%19%19%4%
46%8%
31%4%
4%19%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 137
170
180
190
200
210
220
230
240
250
260
Radiation OncologyPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure RO-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
52%
40%
8%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
52%
43%
5%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
42%
58%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
19%23%
58%
Exam Required?
N=26
N=25
N=26N=21
N=15 N=11
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2018 138
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
19%
0%
81%
COMLEX-USA Level 1*Exam Required?
N=26
0%
20%
40%
60%
80%
100%
Yes No
42%
58%
USMLE Step 2 CSExam Required?
Radiation OncologyPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure RO-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
COMLEX-USA Level 2-CE*Exam Required?
N=22
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%0%
95%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%0%
94%
COMLEX-USA Level 2-PE*Exam Required?
N=26N=22
N=16 N=16
N=16 N=15
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 139
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 96%
56%
37%
19%
7%
33%
22%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure RO-5Radiation OncologyProgram's Interview Activities
0
20
40
60
80
100
120
140
160
180
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
2
174
3829 27
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
19%
78%
Average Percentage of Applicants Rejected and Reviewed
N=27N=27N=27* N=26*N=26N=27
N=27
7% of Radiation Oncology programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4% 8%
27%
84%
60%50% 50%
73%
62%
16%
32%
35%42%
96%
19%12% 8%
15%8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4%13%
38%
88%
64%54% 58%
63%
50%
12%
28%
31%35%
96%
25%13% 8%
15%8%
Often Seldom Never
Ranking
n=26n=26
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 140
Figure RO-6Radiation OncologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
31%
49%
14%5%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1% 0%9%
39%
51%
Percentage of Program's InterviewsConducted During Each Time Period
N=27 N=27
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
4.8
5.0
4.7
4.7
4.8
4.2
4.6
3.5
3.3
4.1
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 141
Radiology(N=48)
NRMP Program Director Survey Results, 2018 142
Figure RD-1Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=43)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.4
3.9
4.3
4.1
3.6
4.3
4.5
4.2
4.1
4.0
4.4
3.8
4.4
4.1
3.9
4.0
3.8
4.2
3.5
4.1
3.7
3.8
3.8
3.9
3.8
3.5
4.3
4.8
2.8
3.2
3.8
3.3
4.0
100% 50% 0%
95%
88%
91%
67%
84%
84%
70%
70%
70%
58%
58%
70%
65%
72%
72%
70%
60%
40%
49%
79%
51%
56%
42%
44%
53%
44%
44%
37%
19%
21%
12%
37%14%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 143
Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=43)
Figure RD-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.84.94.9
4.23.9
4.24.34.5
4.24.5
4.04.1
4.04.0
3.64.24.4
4.94.2
3.74.1
4.33.6
3.93.7
4.13.5
3.83.7
4.84.5
3.93.1
4.22.0
3.73.63.7
100% 50% 0%
93%93%93%
82%93%
70%61%64%
73%64%
73%59%
66%45%45%
52%66%
27%43%
36%43%
48%55%
27%39%
32%34%30%27%23%
30%25%
36%16%11%
7%14%11%16%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 144
170
180
190
200
210
220
230
240
250
260
Radiology-DiagnosticPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure RD-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
43%
52%
5%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
36%
56%
8%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
30%
67%
2%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
17%
36%
48%
Exam Required?
N=43
N=42
N=42N=39
N=13N=27 N=25 N=13
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 145
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%
52%
33%
COMLEX-USA Level 1*Exam Required?
N=41
0%
20%
40%
60%
80%
100%
Yes No
46%54%
USMLE Step 2 CSExam Required?
Radiology-DiagnosticPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure RD-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
16%
28%
56%
COMLEX-USA Level 2-CE*Exam Required?
N=37
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 0%
97%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%
10%
61%
COMLEX-USA Level 2-PE*Exam Required?
N=41N=37
N=33 N=32
N=31 N=28
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 146
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
74%67%
37%
12%
60%
37%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure RD-5Radiology-DiagnosticProgram's Interview Activities
0
100
200
300
400
500
600
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
6
534
11694 82
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
38%
56%
Average Percentage of Applicants Rejected and Reviewed
N=46N=44N=48* N=46*N=46N=46
N=43
12% of Radiology-Diagnostic programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=46
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 7%
72%
28%
7%
35%
5%
58%35%
28%
51%
65%
49%95%
40%
58%
21%28%
16%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
2% 7%
71%
30%
7%
35%
2%
51% 40%
26%
53%
70%
53%
98%
47%53%
2%16%
23%12%
Often Seldom Never
Ranking
n=43n=43
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 147
Figure RD-6Radiology-DiagnosticProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=44
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
21%
52%
19%
6% 3%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
17%
33% 33%
16%
Percentage of Program's InterviewsConducted During Each Time Period
N=42 N=42
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.8
4.9
4.8
4.8
4.7
4.8
4.3
4.2
3.6
3.2
3.3
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 148
Surgery(N=107)
NRMP Program Director Survey Results, 2018 149
Figure GS-1SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=99)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.4
4.2
4.0
4.2
3.5
4.1
4.8
3.9
4.3
4.3
4.2
3.9
4.6
4.3
4.0
4.2
3.9
4.0
3.8
3.9
3.9
3.8
3.6
3.8
3.9
3.9
4.1
4.7
3.4
4.1
3.9
3.7
3.5
100% 50% 0%
94%
88%
74%
81%
74%
74%
68%
76%
54%
64%
76%
48%
55%
61%
63%
57%
52%
38%
44%
41%
49%
56%
51%
52%
49%
54%
44%
34%
13%
30%
18%
19%
4%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 150
SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=99)
Figure GS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.84.84.7
4.44.04.2
3.84.6
4.34.1
4.43.9
4.34.1
3.44.04.2
4.84.14.04.2
3.93.93.9
3.74.04.04.1
3.84.8
4.23.94.1
3.93.9
3.53.63.7
100% 50% 0%
95%95%93%
78%81%79%78%
53%59%54%
65%59%
66%56%
36%48%50%
38%35%
57%43%43%
33%32%
40%39%
33%39%39%36%
21%20%
30%22%16%8%4%
9%15%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 151
170
180
190
200
210
220
230
240
250
260
SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure GS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
46%53%
1%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
52%46%
2%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
85%
1%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12%
58%
30%
Exam Required?
N=101
N=97
N=99N=94
N=45N=81 N=62 N=35
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 152
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6%
31%
63%
COMLEX-USA Level 1*Exam Required?
N=98
0%
20%
40%
60%
80%
100%
Yes No
55%
45%
USMLE Step 2 CSExam Required?
SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure GS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%
23%
72%
COMLEX-USA Level 2-CE*Exam Required?
N=80
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 4%
94%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
2% 5%
93%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%9%
75%
COMLEX-USA Level 2-PE*Exam Required?
N=98N=80
N=67 N=64
N=65 N=61
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 153
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 99%
62%58%
37%
17%
65%
52%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure GS-5SurgeryProgram's Interview Activities
0
100
200
300
400
500
600
700
800
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
5
742
85 66 52
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
58%
36%
Average Percentage of Applicants Rejected and Reviewed
N=100N=98N=107* N=105*N=105N=100
N=100
12% of Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=101
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1%7%
21%
61%
31%
7%21%
4%
52%
41%
34%
56%
60%
52%95%
41% 37%
4%13%
33%27%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
1%9%
23%
63%
33%
9%24%
5%
55%37%
33%
53%
58%
53%94%
36% 39%
4%14%
33%23%
Often Seldom Never
Ranking
n=98n=99
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 154
Figure GS-6SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=93
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
9%
47%
28%
10%6%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%5%
33% 34%28%
Percentage of Program's InterviewsConducted During Each Time Period
N=95 N=95
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
4.9
4.9
4.9
4.7
4.8
4.3
4.2
4.0
3.9
3.6
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 155
Thoracic Surgery(N=11)
NRMP Program Director Survey Results, 2018 156
Figure TS-1Thoracic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=9)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.1
4.4
4.1
4.5
3.9
4.3
4.0
3.8
4.3
4.2
4.5
3.8
4.5
4.2
4.1
4.3
3.7
4.5
4.0
4.8
3.8
4.0
4.7
3.3
4.3
4.2
3.3
4.0
3.7
4.3
3.0
4.5
3.0
100% 50% 0%
89%
100%
89%
56%
100%
78%
56%
67%
100%
78%
78%
67%
78%
89%
89%
44%
78%
56%
56%
67%
67%
67%
78%
56%
78%
67%
44%
22%
67%
56%
22%
33%22%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 157
Thoracic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=9)
Figure TS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.84.64.84.74.8
4.34.8
4.34.6
4.44.54.6
4.24.8
4.43.8
4.35.0
4.34.2
3.74.6
4.44.3
4.04.3
4.04.2
3.84.2
4.02.7
3.54.2
3.82.5
4.04.0
100% 50% 0%
89%89%89%
78%56%
89%44%
56%78%
89%44%
78%67%
44%78%
67%67%
22%33%
56%33%
56%56%
44%44%44%44%
67%56%
67%22%
33%22%
67%0%
56%22%11%
33%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 158
Thoracic SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure TS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
25%
75%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
25%
75%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
30%
70%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
20%
50%
30%
Exam Required?
N=10
N=8
N=10N=8
N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2018 159
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
17%
83%
COMLEX-USA Level 1*Exam Required?
N=9
0%
20%
40%
60%
80%
100%
Yes No
67%
33%
USMLE Step 2 CSExam Required?
Thoracic SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure TS-4
N=8
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13% 13%
75%
USMLE Step 3Exam Required?
N=9N=8
N=6
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
Not reported because of low response rate
Not reported because of low response rate
Not reported because of low response rate
NRMP Program Director Survey Results, 2018 160
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100%
89%
78%
67%
33%
67%
56%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure TS-5Thoracic SurgeryProgram's Interview Activities
0
20
40
60
80
100
120
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
1
117
3125
22
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
42%
54%
Average Percentage of Applicants Rejected and Reviewed
N=10N=10N=11* N=11*N=10N=10
N=9
22% of Thoracic Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=11
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
11%
63%
44%
22%33%
11%
56%
67%
38%
33%
56%
56%89%
44%
22% 22% 22%11%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
11%22%
75%
33%22%
33%
56%
56%
25%
44%56%
56%89%
44%
22% 22% 22%11%
Often Seldom Never
Ranking
n=9n=9
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 161
Figure TS-6Thoracic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=9
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
7%
48%
32%
13%
0%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%5%
28% 28%
40%
Percentage of Program's InterviewsConducted During Each Time Period
N=10 N=10
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
5.0
5.0
5.0
4.7
4.7
4.6
4.4
4.5
4.1
3.9
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 162
Transitional Year(N=30)
NRMP Program Director Survey Results, 2018 163
Figure TY-1Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.4
4.1
4.4
4.3
4.1
4.4
4.8
4.3
4.4
4.1
4.0
3.9
4.7
4.3
3.7
4.4
4.2
4.5
4.1
4.2
4.4
4.0
3.9
3.9
4.0
3.7
4.4
4.9
4.0
3.8
3.8
3.0
3.3
100% 50% 0%
93%
56%
70%
81%
74%
67%
67%
59%
48%
67%
30%
48%
67%
56%
67%
63%
48%
67%
56%
67%
52%
48%
52%
56%
19%
37%
52%
37%
7%
15%
22%
19%
11%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 164
Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)
Figure TY-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.75.0
4.64.64.6
4.24.34.3
4.84.7
4.54.24.5
4.03.84.0
4.44.3
4.94.4
4.24.34.2
4.04.5
4.34.4
4.14.7
4.34.9
4.44.7
4.04.34.5
3.34.0
4.5100% 50% 0%
100%100%96%
85%74%
52%70%67%63%
44%78%
63%41%
67%48%
63%59%
63%44%
19%33%
48%67%
44%48%
41%41%
33%11%22%
37%41%
22%11%
11%7%
11%7%
7%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 165
170
180
190
200
210
220
230
240
250
260
Transitional YearPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure TY-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
44%52%
4%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
42%
54%
4%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
26%
70%
4%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
13%
46%42%
Exam Required?
N=27
N=27
N=24N=26
N=10N=17 N=13 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 166
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
21%
50%
29%
COMLEX-USA Level 1*Exam Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
58%
42%
USMLE Step 2 CSExam Required?
Transitional YearPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure TY-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%
41%50%
COMLEX-USA Level 2-CE*Exam Required?
N=19
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5% 5%
89%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%6%
94%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%24%
48%
COMLEX-USA Level 2-PE*Exam Required?
N=24N=19
N=24 N=22
N=21 N=18
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 167
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 93%
36%
71%
32%
14%
36%
18%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure TY-5Transitional YearProgram's Interview Activities
0
100
200
300
400
500
600
700
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
9
629
10485 68
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
53%
35%
Average Percentage of Applicants Rejected and Reviewed
N=27N=27N=30* N=29*N=29N=27
N=28
11% of Transitional Year programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=27
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4%16%
4%
78%
48%40%
54%
64%
33%
22%
43%44%
38%
96%
20%
63%
9%16%
8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
4%13%
4%
83%
45% 42%
57%
71%
38%
17%
45%42%
38%
96%
17%
58%
9%17%
5%
Often Seldom Never
Ranking
n=25n=25
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 168
Figure TY-6Transitional YearProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=17
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
11%
48%
25%
10% 7%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
1%
15%
29% 28% 27%
Percentage of Program's InterviewsConducted During Each Time Period
N=27 N=27
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
4.9
5.0
4.9
4.9
4.9
4.1
4.4
4.6
4.0
3.9
3.4
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 169
Vascular Surgery(N=16)
NRMP Program Director Survey Results, 2018 170
Figure VS-1Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=14)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.4
4.8
3.9
4.1
3.4
4.3
4.3
4.4
4.8
4.1
4.6
4.4
4.6
3.6
3.8
4.4
3.9
4.0
4.0
4.0
4.0
4.0
4.2
3.8
3.6
3.9
3.8
5.0
3.7
4.2
3.3
3.0
4.0
100% 50% 0%
100%
93%
64%
93%
93%
64%
50%
57%
64%
64%
57%
57%
57%
86%
79%
36%
57%
71%
43%
43%
36%
64%
71%
29%
57%
57%
43%14%
21%
36%
21%
14%21%
USMLE Step 1/COMLEX Level 1 score
Letters of recommendation in the specialty
Medical Student Performance Evaluation (MSPE/Dean's Letter)
USMLE Step 2 CK/COMLEX Level 2 CE score
Personal Statement
Grades in required clerkships
Any failed attempt in USMLE/COMLEX
Class ranking/quartile
Perceived commitment to specialty
Personal prior knowledge of the applicant
Grades in clerkship in desired specialty
Audition elective/rotation within your department
Evidence of professionalism and ethics
Leadership qualities
Alpha Omega Alpha (AOA) membership
Perceived interest in program
Other life experience
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Volunteer/extracurricular experiences
Consistency of grades
Lack of gaps in medical education
Awards or special honors in clinical clerkships
Graduate of highly‐regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Visa status*
Applicant was flagged with Match violation by the NRMP
Away rotation in your specialty at another institution
Interest in academic career
Fluency in language spoken by your patient population
Awards or special honors in basic sciences
USMLE/COMLEX Step 3 score
NRMP Program Director Survey Results, 2018 171
Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=14)
Figure VS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* International Medical Graduates only
1 2 3 4 5
4.94.84.8
4.74.3
4.54.24.24.4
4.84.34.34.4
4.84.3
3.64.8
4.25.0
4.54.0
3.94.7
4.04.0
4.63.5
4.34.04.1
5.04.2
4.43.3
3.84.0
0
100% 50% 0%
100%100%
93%79%
86%86%
64%36%36%
64%64%
50%50%
57%71%
64%29%
43%21%
29%36%
50%21%21%14%
36%14%
29%14%
50%7%
36%0%
36%21%
29%21%
7%
0%
Interactions with faculty during interview and visit
Interpersonal skills
Interactions with housestaff during interview and visit
Feedback from current residents
USMLE/COMLEX Step 1 score
Letters of recommendation in the specialty
USMLE/COMLEX Step 2 score
Evidence of professionalism and ethics
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Perceived interest in program
Leadership qualities
Class ranking/quartile
Personal prior knowledge of the applicant
Audition elective/rotation within your department
Personal Statement
Grades in required clerkships
Passing USMLE Step 2 CS/COMLEX Level 2 PE
Any failed attempt in USMLE/COMLEX
Grades in clerkship in desired specialty
Other life experience
Alpha Omega Alpha (AOA) membership
Consistency of grades
Volunteer/extracurricular experiences
Lack of gaps in medical education
Graduate of highly regarded U.S. medical school
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clinical clerkships
Awards or special honors in clerkship in desired specialty
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status*
Other post‐interview contact
Interest in academic career
Fluency in language spoken by your patient population
Away rotation in your specialty at another institution
USMLE/COMLEX Step 3 score
Second interview/visit
Awards or special honors in basic sciences
NRMP Program Director Survey Results, 2018 172
170
180
190
200
210
220
230
240
250
260
Vascular SurgeryPrograms That Use USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure VS-3
Percentage of Programs That Use USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
23%
77%
0%
Would your program consider applicants who fail theexam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
31%
69%
0%
Would your program consider applicants who failthe exam on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
86%
0%
Exam Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
7%
71%
21%
Exam Required?
N=14
N=13
N=14N=13
N=9N=10 N=11 N=9
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
Percentage of Programs That Use USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (or IQR, which is the range between the 25th and 75th percentiles) and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2018 173
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 1*Exam Required?
N=14
0%
20%
40%
60%
80%
100%
Yes No
79%
21%
USMLE Step 2 CSExam Required?
Vascular SurgeryPrograms That Use USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure VS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 2-CE*Exam Required?
N=13
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%8%
77%
USMLE Step 3Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Exam Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 2-PE*Exam Required?
N=14N=13
N=5 N=5
N=5 N=5
* Osteopathic applicants only
Percentage of Programs That Use USMLE Step 2 CS and Step 3 Scores
Percentage of Programs That Use COMLEX-USA Level 1 and Level 2-CE Scores
Percentage of Programs That Use COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2018 174
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%100% 100%
54%46%
15%
69%
54%
U.S. Senior U.S. Graduate OsteopathicPhysician
Canadian Fifth Pathway U.S. IMG Non-U.S. IMG
Percentage of Programs that Typically Interview and Rank Each Applicant Type
Figure VS-5Vascular SurgeryProgram's Interview Activities
0
10
20
30
40
50
60
70
80
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants
ranked
1
75
34
2622
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a
standardized screening process
Percentage of applicantsreceiving an in-depth review
29%
74%
Average Percentage of Applicants Rejected and Reviewed
N=14N=14N=16* N=15*N=15N=15
N=13
15% of Vascular Surgery programs consider all applicant groups
* Data from the NRMP database. All Responding programs included
N=15
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
7% 7% 7%
79%
36%
7% 7%
36%
64%
21%
50%
71%86%
93%
57%
29%14%
21%7%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
US Senior Prev.Grad
Osteo Pathway Canadian US IMG Non-USIMG
7%14%
31%
77%
50%
7%21%
29%
46%
23%
36%
64%
79%93%
57%
23%14%
29%
Often Seldom Never
Ranking
n=14n=14
Frequency of Programs Interviewing and Ranking Candidates
NRMP Program Director Survey Results, 2018 175
Figure VS-6Vascular SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=14
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%
61%
18%14%
7%
Percentage of Program's Interviews ExtendedDuring Each Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior toOct. 12017
October2017
November2017
December2017
January2018 or
later
0%5%
27%
41%
28%
Percentage of Program's InterviewsConducted During Each Time Period
N=13 N=13
0 1 2 3 4 5
Clinical competency
Professionalism
Quality of patient care
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Research and publications
5.0
4.8
4.9
4.7
4.6
4.9
4.0
4.2
3.4
4.0
3.6
Importance of Factors in Assessing Residents' SuccessAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2018 176