Does Clinical Exposure Affect Medical StudentExamination Performance?

Sharif H. Ellozy, MD, Susan Kaiser, MD, PhD, Joel J. Bauer, MD, New York, New York

BACKGROUND: Traditionally important componentsof the surgical clerkship curriculum include lec-tures, small-group sessions, readings, clinicalexposure, and testing. Time constraints requirecompromise among all these elements. At ourinstitution, clinical exposure of medical studentsvaries according to their team assignments.They differ primarily in exposure to such topicsas trauma, inflammatory bowel disease (IBD),and vascular surgery.

PURPOSE: The goal of this study was to quantifythe influence of clinical exposure on medical stu-dent education, testing whether it influences ex-amination performance.

METHODS: At the beginning of the clerkship, stu-dents were given a written multiple-choice ex-amination covering these three topics. Theycompleted two 4-week blocks on surgery ser-vices in various settings. At the end of the clerk-ship, they were reexamined. Differences betweenpretest and posttest overall mean rank were an-alyzed using the Wilcoxon signed ranks test. Dif-ferences in improvement based on clinical expo-sure were analyzed using the Mann-Whitney Utest.

RESULTS: Statistically significant improvementwas seen in overall examination performance, aswell as in each of the subsections. However,these improvements could not be accounted forby clinical exposure. There was no statisticallysignificant difference in mean rank in improve-ment based on clinical exposure to vascular,IBD, or trauma.

CONCLUSIONS: Student examination scores im-proved over the course of the clerkship. No im-provement could be attributed to greater clinicalexposure to a topic. Am J Surg. 2000;179:282–285. © 2000 by Excerpta Medica, Inc.

Traditionally important components of the surgicalclerkship curriculum include lectures, small-groupsessions, readings, clinical exposure, and testing. Of

all these elements, the one with the greatest variability

among students is clinical exposure. Increasing specializa-tion and limited resources result in a variety of clerkshipsettings. Clerkship directors struggle to accommodate thedemands of scheduling without compromising the educa-tional experience of the students.

This study was undertaken to assess the impact of clinicalexposure on medical student education. The two assump-tions underlying much of the work written on surgicalclerkship site and its role in medical student performanceare that (1) students are exposed to varying levels ofdifferent topics (ie, greater trauma exposure at an inner-city public hospital than a suburban private hospital), and(2) greater clinical exposure translates directly to a betterunderstanding of a specific topic and improved clerkshipperformance. Other authors have used global markers ofperformance (end-of-clerkship written examinations, pa-tient-management problems [PMPs], the Objective Struc-tured Clinical Examination [OSCE]) to see if expertise inone area translates to improved overall performance. In-terestingly, most of the evidence shows that site of rotationcompletion and amount of clinical responsibility are notrelated to overall student performance.1–4

We investigated the underlying assumption of many ofthese publications: namely, do clear differences in clinicalexposure lead automatically to improved student examina-tion performance in those particular areas? The answer tothis question has profound implications for clerkship de-sign.

METHODSFifty students were enrolled in our study over a 6-month

period. At the beginning of the clerkship, they took awritten multiple-choice examination with five questionson each of the following topics: vascular surgery, inflam-matory bowel disease (IBD), and trauma. The questionswere a mixture of clinical and basic science and had beenadapted from SESAP. At our institution, clear differencesexist in students’ exposure to these three topics. The clerksthen completed 8 weeks on various surgery services. Stu-dent assignments were not randomized, as they were al-lowed to request particular rotations. All students spent 4weeks on one of three general surgery services at ourtertiary-care university hospital, as well as 4 weeks on anyof the following services: 1 month on the general surgeryservice at a city hospital with a level 1 trauma center; 2weeks at a community-based hospital and 2 weeks in thesurgical intensive care unit (SICU); 2 weeks on a vascularsurgery service and 2 weeks at a Veterans Administration(VA) hospital SICU; or 4 weeks at a community-basedhospital. Two of the three general surgery services at thetertiary-care hospital offer extensive experience in IBD;the general surgery services at the community-based hos-

From the Department of Surgery, Mount Sinai School of Med-icine, New York, New York.

Requests for reprints should be addressed to Sharif H. Ellozy,MD, 328 West 89th Street, Apt. 10, New York, New York 10024.

Manuscript submitted September 23, 1999, and accepted inrevised form February 7, 2000.

ASSOCIATION FOR SURGICAL EDUCATION

282 © 2000 by Excerpta Medica, Inc. 0002-9610/00/$–see front matterAll rights reserved. PII S0002-9610(00)00338-X

pitals were notable for having extensive vascular surgeryexperience. All students returned to the main teachinghospital on Wednesday afternoons for didactic teachingthat included case presentations, lectures on general sur-gery and subspecialty topics, problem-solving sessions, andevidence-based medicine sessions. At the end of the8-week period, they retook the written multiple-choiceexamination, and spent 3 weeks in one of four surgicalsubspecialties (urology, otolaryngology, ophthalmology,and orthopedics).

Differences between pretest and posttest overall meanrank were analyzed using the Wilcoxon signed ranks test.Differences in improvement based on clinical exposurewere analyzed using the Mann-Whitney U test.

RESULTSTwenty-one of the 50 students rotated through services

with vascular experience, 35 had significant exposure toIBD, and 19 rotated through the trauma center. Perfor-mance on the pretest was approximately equivalent, withexception of the trauma section: students who did notrotate on trauma seemed to perform better on the pretestthan those who did (mean rank 23.06 versus 30.95); how-ever, with P 5 0.053, this was not statistically significant.With regard to calendar blocks of students, a statisticallysignificant difference was noted in performance on thepretest in the trauma (P 5 0.046) and overall scores (P 50.026). The earlier block performed significantly betterthan the following two blocks. No statistically significantdifference was noted in improvement from pretest to post-test among the blocks.

Statistically significant improvement was present in allsubsections, as well as in overall score (Table I, Figure1). However, no statistically significant difference wasnoted in mean rank in improvement based on clinicalexposure to vascular, IBD, or trauma (Table II, Figures2, 3, and 4).

COMMENTSStudent examination scores improved over the course

of the clerkship; however, this finding could not beattributed to greater clinical exposure to a particulartopic. In previous studies, the effects of clinical exposureon overall clerkship performance were examined using avariety of measures. Papp and colleagues1 evaluatedwhether differences exist among types of surgical rota-tion as measured by student performance on PMPs. Theyfound that rotation did not affect performance on whatwere thought to be related PMPs (ie, students on theoncology service did not perform better on the breast

problem). They stated, in the setting of a curriculumthat emphasizes a “consistent approach to surgical prob-lems and is supplemented by appropriate didactic mate-rial, students . . . achieve similar scores on PMPs thatcover areas in which they have not had actual patientcontact.”

Baciewicz and coworkers2 performed a multivariateanalysis of variance reviewing influence of calendarblock, hospital site, and mix of general surgical versussubspecialty rotations on surgery clerkship scores on theoral and written examinations. Their initial hypothesiswas that different combinations of teaching environ-ments would result in different levels of performance;this was not borne out by their findings. They noted nosignificant site or rotation effects on examination scores.

Chatenay and colleagues3 examined whether type andvolume of clerkship experience affected performance on

Figure 1. Display of improvement in overall score.

TABLE IIMean Rank in Performance Improvement,* with the

Significance of the Difference

Vascular Trauma IBD

Exposure 22.67 24.71 24.99No exposure 27.55 25.98 26.70Significance P 5 0.227 P 5 0.758 P 5 0.695

* Mean rank in score improvement of each subgroup, with a best possiblescore of 1 and a worst possible score of 50.IBD 5 inflammatory bowel disease.

TABLE IRaw Score (Mean 6 SD) on the Pretest and Posttest, with the Significance of the

Difference

Vascular Trauma IBD Overall

Pretest 1.72 6 1.20 1.90 6 0.95 1.82 6 1.02 5.44 6 1.75Posttest 3.02 6 1.25 2.64 6 1.05 2.46 6 1.11 8.12 6 2.08Significance P ,0.001 P ,0.001 P 5 0.002 P ,0.001

IBD 5 inflammatory bowel disease.

CLINICAL EXPOSURE EFFECT ON STUDENT EXAMINATION PERFORMANCE/ELLOZY ET AL

THE AMERICAN JOURNAL OF SURGERY® VOLUME 179 APRIL 2000 283

exit examinations. They controlled for preclerkship levelof knowledge by incorporating the results of surgical ex-amination scores from the preclinical years. Outcomeswere assessed using an end-of-clerkship written multiple-choice examination, an OSCE, and a subjective evaluationof the student by the preceptor. A significant differencewas found in volume of clinical experience between, aswell as within, sites. For the majority of performance vari-ables, no difference was noted between high and low ex-posure. The only difference they found was in performanceon the OSCE. Students who attended more outpatientclinics had lower OSCE scores than those who attendedmore emergency cases. This finding was attributed to thenature of the feedback: students in the emergency depart-ment were more likely to get immediate feedback from aresident, with correction of mistakes and reinforcement ofgood habits.

Poenaru and colleagues4 examined whether a generalsurgery rotation was necessary in the surgical clerkship.Two 4-week blocks of either general surgery or subspecial-

ties (orthopedics, neurosurgery, cardiothoracic, vascular)comprised their clerkship. Thirty-five students rotatedthrough general surgery and another subspecialty, whereas19 rotated through two subspecialties other than generalsurgery. All students participated in centralized weeklysmall-group seminars. Outcome assessment included a writ-ten short-answer examination, a nine-station OSCE, and aclinical objective self-assessment test (COSAT). No dif-ferences were noted on either the ward evaluations, writtenscore, or OSCE score. COSAT scores improved signifi-cantly from prerotation to postrotation, but there was novariance due to general surgery experience.

All the previous studies used global markers to assess theeffect of varying clinical settings. In the present study, welimited our analysis to improvement in subject areas withclearly different levels of clinical exposure. It was our hopethat a more focused examination would disclose differencesthat had been overlooked previously. Between-student dif-ferences in baseline knowledge were identified with thepretest. Interestingly, we found a significant block effect onperformance on the pretest trauma section and overallscore, yet no effect was noted on performance improve-ment, for which no ready explanation was available. Onecould hypothesize, however, that students interested ingoing into surgery would be more likely to request partic-ular schedules. Importantly, the significant improvementseen in all subsections and overall performance providedconcrete evidence that the students’ fund of surgicalknowledge had increased over the course of the clerkship.However, in contrast to our initial suspicion, we couldidentify no clinical effect on the improvement in exami-nation scores. This result was consistent for all three sub-sections and across all groups of students.

The fact that clinical exposure did not affect examinationscores is counterintuitive, but it is an encouraging finding.Students cannot all be provided with identical experiences,but they should complete the clerkship with the same coreof surgical knowledge. Whereas the basics of surgical careare perhaps best learned clinically, results of this study

Figure 2. Boxplot displaying the relationship of vascular expo-sure to improvement from pretest to posttest (o and * representthe outliers).

Figure 3. Boxplot displaying the relationship of trauma exposureto improvement from pretest to posttest.

Figure 4. Boxplot displaying the relationship of inflammatorybowel disease (IBD) exposure to improvement from pretest toposttest (o and * represent the outliers).

CLINICAL EXPOSURE EFFECT ON STUDENT EXAMINATION PERFORMANCE/ELLOZY ET AL

284 THE AMERICAN JOURNAL OF SURGERY® VOLUME 179 APRIL 2000

suggest that the particulars of specific disease processes canbe taught equally well through lectures, small-group ses-sions, and assigned readings, thus allowing for more free-dom in designing the surgical clerkship.

REFERENCES1. Papp KK, Williams D, Goldman MH. Relationship betweentype of surgical clerkship, order of completion, and achievement onpatient management problems. Surgery. 1984;96:102–107.

2. Baciewicz F, Arent L, Weaver M, et al. Influence of clerkshipstructure and timing on student performance. Am J Surg. 1990;159:265–268.3. Chatenay M, Maguire T, Skakun E, et al. Does volume ofclinical experience affect performance of clinical clerks on surgeryexit examinations? Am J Surg. 1996;172:366–372.4. Poenaru D, Davidson L, Donnely M, Tekian A. Is a mandatorygeneral surgery rotation necessary in the surgical clerkship? Am JSurg. 1998;175:515–517.

CLINICAL EXPOSURE EFFECT ON STUDENT EXAMINATION PERFORMANCE/ELLOZY ET AL

THE AMERICAN JOURNAL OF SURGERY® VOLUME 179 APRIL 2000 285