student perceptions of assessment and feedback in longitudinal integrated clerkships

Student perceptions of assessment and feedback inlongitudinal integrated clerkshipsJoanna Bates,1 Jill Konkin,2 Carol Suddards,3 Sarah Dobson1 & Dan Pratt1

OBJECTIVES This study was conducted toelucidate how the learning environment andthe student–preceptor relationship influencestudent experiences of being assessed andreceiving feedback on performance. Thus, weexamined how long-term clinical clerkshipplacements influence students’ experiences ofand views about assessment and feedback.

METHODS We took a constructivist groundedapproach, using authentic assessment and com-munities of practice as sensitising concepts. Werecruited and interviewed 13 students studyingin longitudinal integrated clerkships across twomedical schools and six settings, using a semi-structured interview framework. We used aniterative coding process to code the data andarrive at a coding framework and themes.

RESULTS Students valued the unstructuredassessment and informal feedback that arosefrom clinical supervision, and the sense ofprogress derived from their increasing respon-sibility for patients and acceptance into thehealth care community. Three themes emergedfrom the data. Firstly, students characterisedtheir assessment and feedback as integrated,

developmental and longitudinal. They reportedauthenticity in the monitoring and feedbackthat arose from the day-to-day delivery ofpatient care with their preceptors. Secondly,students described supportive and caring rela-tionships and a sense of safety. These enabledthem to reflect on their strengths and weak-nesses and to interpret critical feedback assupportive. Students developed similar rela-tionships across the health care team. Thirdly,the long-term placement provided for multipleindicators of progress for students. Patientoutcomes were perceived as representing directfeedback about students’ development as doc-tors. Taking increasing responsibility forpatients over time is an indicator to students oftheir increasing competence and contributes tothe developing of a doctor identity.

CONCLUSIONS Clerkship students studyingfor extended periods in one environment withone preceptor perceive assessment and feedbackas authentic because they are embedded in dailypatient care, useful because they are develop-mental and longitudinal, and constructivebecause they occur in the context of a supportivelearning environment and relationship.

longitudinal integrated clerkships

Medical Education 2013: 47: 362–374doi:10.1111/medu.12087

1Centre for Health Education Scholarship, Faculty of Medicine,University of British Columbia, Vancouver, British Columbia,Canada2Division of Community Engagement, Faculty of Medicine andDentistry, University of Alberta, Edmonton, Alberta, Canada3Office of Rural and Regional Health, Faculty of Medicine andDentistry, University of Alberta, Edmonton, Alberta, Canada

Correspondence: Dr Joanna Bates, Centre for Health EducationScholarship, Faculty of Medicine, University of British Columbia,3326-910 West 10th Avenue, Vancouver, British ColumbiaV5Z 1M9, Canada. Tel: 00 1 604 875 5185;E-mail: [email protected]

362 ª Blackwell Publishing Ltd 2013. MEDICAL EDUCATION 2013; 47: 362–374

INTRODUCTION

Lack of continuity bedevils the undergraduateclinical teaching environment to the detriment ofstudent learning. Preceptors and residents moveon and off service, patients move in and out ofhospital, and students themselves move everyfew weeks across different teaching settings.1,2

This lack of continuity affects the developmentof students’ clinical knowledge and reasoning,clinical skills and professionalism.3,4 Studentsexplain that they expend considerable energyadapting to frequent changes in staff, setting andcontent at a cost to their learning.3,5 O’Brien et al.3

further suggest that no preceptor spends suffi-cient time with students to know about theirprevious experience, to understand their strengthsand weaknesses, and to work towards addressingthese.

As a result of increasing awareness of the impact ofsuch change on students, clinical education is beingredesigned to increase continuity. Hirsh et al.4 andCooke et al.6 describe educational continuity asdemonstrated in continuity of curriculum, continuityof preceptor and continuity of context. In this spirit, anew model of clinical clerkship that offers greatercontinuity is emerging: the longitudinal integratedclerkship (LIC). The LIC has been implemented as acomplex educational intervention in multiple set-tings, including in different countries, new medicalschools, longstanding faculties and both urban andrural settings.7–13 The common elements of LICs aredefined as follows:

‘Longitudinal integrated clerkships are character-ised by clinical education where students partici-pate in the comprehensive care of patients overtime; participate in continuing learning relation-ships with these patients’ clinicians; and meet themajority of the year’s core clinical competenciesacross multiple disciplines simultaneously throughthese experiences.’13

Students in LICs have been reported to performbetter on clinical examinations,14 to maintain theiraltruism and patient centredness,4,15 and to perceivetheir assessment as fairer and more accurate, andfeedback as more useful to learning than do studentsin traditional clerkship rotations.16,17 We chose tofurther investigate the findings of two studies16,17 thatLIC students rate their assessments and feedbackmore positively than students on rotational clerk-ships.

The relationship between assessment and feedback

The assessment of students in medical educationserves both summative and formative purposes.Although assessment does not necessarily lead tofeedback for students, observations and judgementsabout a student’s performance are necessary compo-nents of effective feedback. In clinical settings,assessment for formative purposes is generallyachieved through direct observation.18 However,opportunities for direct observation in clinical set-tings are often variable,19 which limits the utility offormal feedback mechanisms such as in-trainingevaluations of trainees.20

Attempts to increase direct observation andstrengthen the link between assessment and feedbackhave focused on the development of new instrumentssuch as the mini-clinical examination (mini-CEX),21

which structures an episode of direct observation andspecific feedback for the trainee. After a preceptorhas observed an interaction between the trainee andpatient, the trainee receives feedback from thepreceptor identifying areas of strength, directions forfuture development, and an action plan for address-ing weaknesses. However, even when a mini-CEX isinstituted and a trainee observed, the resultingfeedback is often lacking in either content, such asthe formulation of an action plan, or the dialoguebetween student and preceptor necessary for effectivefeedback.22,23

Even when feedback is given well, other factors maylessen its acceptance by and hence usefulness to thetrainee. If feedback is based on assessment that atrainee feels is unfair, in which lack of contact withthe preceptor is the primary reason for the percep-tion of unfairness, he or she is likely to reject thefeedback.24 Not only the amount of contact, but alsothe interpersonal dynamics between the trainee andpreceptor and the emotions generated by the inter-action can influence how the trainee hears andresponds to feedback.25 The credibility of thepreceptor,26 the source of the feedback,27 the timingof the feedback28 and the engagement the traineefeels in the process29 also influence trainee accep-tance of feedback and thus support or undermine theopportunity to foster learning. This is particularlytrue of critical feedback. In fact, although traineesagree on the value of feedback,30,31 they often findcritical feedback difficult to accept.29

Given the complex interactions that influencestudent perceptions of the fairness and usefulness ofassessment and feedback, findings that LIC students

ª Blackwell Publishing Ltd 2013. MEDICAL EDUCATION 2013; 47: 362–374 363

Longitudinal integrated clerkships

perceive their assessments and feedback morefavourably than students on rotational clerkshipsraise more questions than they answer.16,17 In thesestudies,16,17 the structured elements of assessmentand feedback were common to both groups ofstudents. We conjecture that longitudinal placementschange the context in which the assessment andfeedback interactions between preceptors andstudents occur.

Mazotti et al.17 found on survey that LIC preceptorsand students were more willing to provide and receiveconstructive feedback, including negative feedback,and speculated that this finding is a result ofincreased engagement between the student andpreceptor. However, as yet we know very little abouthow assessment and feedback processes arise and areexperienced by students in longitudinal clerkships.Nor do we know much about the ongoing relation-ship between a student and preceptor and theevolving nature of feedback and assessment arisingfrom that relationship. Our goal, therefore, was toexamine how long-term clinical clerkship placementsinfluence students’ experiences of and views aboutassessment and feedback. Our research question was:what are the influences of continuity of clinicalplacement and preceptor on student experiences offeedback and assessment in LICs?

METHODS

This study took place at two schools of medicine inwestern Canada, the University of British Columbia(UBC) and the University of Alberta (UA). Theseuniversities implemented LICs in 2004 and 2007,respectively. Both schools selected communitiesdistant from the academic medical centre andrecruited family doctors as preceptors for the stu-dents. Preceptors underwent basic faculty develop-ment, which included at least one session on assessingstudents and giving feedback. Students self-selectedthe LIC option and spent almost all of the clinicalyear (36 weeks at UA, 42 weeks at UBC) with theirpreceptors, integrated into the patient care team ofthe practice, working with family doctors and (whereavailable) specialists across the health care system intheir respective communities. Students learning inthis setting spent most of their clinical time consult-ing on undifferentiated patients with their primaryfamily practice preceptors.

Curriculum and assessment processes differedbetween the two schools, but both schools requiredstudents to meet learning objectives in the core

clinical disciplines of medicine, surgery, paediatrics,obstetrics and gynaecology, psychiatry, anaesthesiaand family practice. UBC also required students tomeet additional learning objectives in emergencymedicine, orthopaedics and dermatology. Both pro-grammes mandated structured formative assessmentand feedback through scheduled mini-CEXs (30 perstudent at UA, 24 per student at UBC) and visits ofsenior faculty members to the communities (betweenfour and six per year per site at UA, between four and12 at UBC). Students at UA also participated in aformative multiple-choice question (MCQ) examina-tion and objective structured clinical examination(OSCE) midway through the year. Summative assess-ment consisted of an integrated OSCE at the end ofthe clerkship (UBC and UA), a summative MCQexamination (UA) at the end of the clinical year, anddiscipline-specific examinations in six major disci-plines interspersed throughout the year (UBC).

Methodology

Our methodological approach used constructivistgrounded theory (CGT). Constructivist groundedtheory ‘places priority on the phenomenon of studyand sees both data collection and analysis as createdfrom the shared experiences of researchers andparticipants’.32 Using CGT required us to pay closeattention to the trainees’ narratives and the contextsof the study, while acknowledging that data gatheringand analysis also reflected our thinking as research-ers. For example, in the initial phases of the study, wewere guided by concepts derived from authenticassessment.33–35 Authentic assessment is meant toreflect the conditions of the work environment inwhich the student is expected to function and ofteninvolves complex ill-structured challenges thatrequire judgement. Following the first iteration ofcoding, and as a result of our initial finding ofmeaning in the data, our further analysis was guidedby sensitising concepts derived from communities ofpractice theory.36,37 The concept of communities ofpractice, as a theoretical lens, emphasises that learn-ing occurs through participation in practice along-side more experienced members, and differs fromlearning in a traditional apprenticeship in its focus onthe community and its cultural practices, in additionto learning from master practitioners. Learning, inthis view, is a complex mix of participation, engage-ment, and the development of competence andidentity as the trainee moves from the peripherytowards the centre of a community of practice. Fromthis approach we developed a theoretical explanationthat is grounded in the data, but necessarily filteredthrough our use of the theoretical lenses and the


J Bates et al

various points of view represented by our team ofresearchers.

The research team included a medical educationresearcher who is a family doctor and senior educa-tion programme administrator (JB), an associatedean responsible for rural medical education who is arural family doctor (JK), a professor from the facultyof education with expertise in adult education andlearning theory (DP), an educational contextsspecialist, programme evaluator and qualitativeresearcher (CS), and a research assistant trained inqualitative methods and population health (SD). Allbut two (DP, SD) of the research team participated inthe development or evaluation of the LICs in theirrespective settings and had informal discussions withstudents about their experiences. Two of theresearchers (JB, JK) were clinician-educators, experi-enced in teaching students in clinical settings,assessing student performance and giving feedback.

The study was approved by the ethics board of eachuniversity. We invited all of the 19 students studyingin integrated clerkships at the two faculties ofmedicine in six different geographical sites across twoprovinces to participate, and recruited 13 (nine fromschool A and four from school B). These includedfour men and nine women who described both ruraland urban backgrounds prior to entering medicalschool. They were studying either alone or with otherstudent clerks in communities 100–1000 km from theteaching centre with populations of 5000–80000. Allhad elected to be considered for the LIC and hadbeen selected by the programme from a pool ofapplicants. Their reasons for their choice werediverse and included an interest in rural medicine orfamily practice, the desire to try something different,a preference for a small programme, an opportunityto move closer to friends and family, and interestsparked at an information session attended in returnfor a free lunch.

The research team conducted individual semi-struc-tured interviews between weeks 26 and 36 of theclerkships. Interviews lasted 30–90 minutes and wereconducted face to face, by video or by telephone. Allinterviews except one included two interviewers fromthe research team. Interviewers debriefed after eachinterview and developed new probes based on theprevious interviews. These debriefs were recordedand transcribed, and served as field notes. Interviewswere taped, transcribed and anonymised.

In the interviews, we asked students to tell us aboutthemselves, the settings they were in and their

programmes. We then asked open-ended questionsabout their experiences of assessment and feedback,using probes to explore the feedback they received,what made feedback meaningful to them, how theyknew that they were making progress, and when andfrom whom they received feedback that was useful tothem.

We used an iterative coding process to code the dataand arrive at a mutually agreed coding frameworkand themes.38 Firstly, all members of the researchteam read six interview transcripts coded by threemembers of the research team (JB, DP, CS). Theentire team discussed the meaning of the narrativesand agreed on an interim coding scheme. Remainingtranscripts were analysed and coded by one memberof the research team (JB). After the research teamhad agreed on the initial coding scheme, we exploreddifferent conceptual lenses and agreed that thetheoretical lens of communities of practice wouldprovide sensitising concepts for the data analysis. Wemoved between the data and the conceptual frame-work to interpret the data. Coding schemes under-went three rounds of refinement, each of which wasreviewed and evaluated by the research team. Thisdata analysis resulted in an initial five themes. Sixmonths after the trainees had completed theirclerkships, we invited all original participants toreview our initial themes and discuss our findings,and seven students (five from school A and two fromschool B) participated in a second interview. Thetranscripts of those interviews were reviewed by twomembers of the research team (SD, JB) and discussedby the entire team. This process led to the merging ofsimilar themes and grouping of the data into threefinal themes (Table 1).

RESULTS

Students spoke initially in the interviews about theirexperiences of both summative and formative assess-ment processes. They described meeting the pro-gramme requirements of a number of mini-CEXsduring the clerkship. When asked about useful andmeaningful assessment and feedback, however, theyimmediately described the informal feedback thatarose from day-to-day immersion in practice along-side their preceptors. As one student commented:

‘Assessment means to me that somebody who’s amentor or even a [person who doesn’t know me], Iguess, is evaluating me and giving me constructivefeedback and actually will monitor my progress aswell and do further assessments so that I can – he



and myself can see the changes and improvementsand deficits.’

(D1-1)

(Interviews were anonymised according to LIC site,interviewee and interview number. For example, D1-1refers to interview 1 for the first interviewee from siteD.)

According to our CGT methods, we followed thestudent leads and probed more deeply into theunstructured processes of assessment by the precep-tor and the resulting informal feedback that thestudents most valued. In the results, then, the term‘assessment’ refers not to the more common under-standing of assessment as involving a structuredprocess or tool designed for assessing students, but tothe often invisible and unstructured ‘point of careassessment’39 conducted by preceptors upon whichthey judge the level of supervision a student requiresaccording to the student’s developing competence.Likewise, the term ‘feedback’ refers to the informalformative feedback given in day-to-day clinical inter-actions rather than the formal feedback deliveredthrough mechanisms such as in-training evaluations.The following themes represent our analysis of thestudents’ experiences of this assessment and feedbackwithin LICs and are illustrated by quotes derived fromthe participants.

Theme 1: assessment and feedback that arose fromparticipation in practice over time were perceived bystudents as authentic

This theme sets out the structure of studentthinking about authentic assessment and usefulfeedback. Students characterised authentic assess-ment and feedback as integrated, developmentaland longitudinal. They reported seeing suchauthenticity in the monitoring, assessments andfeedback that arose from their day-to-day delivery ofcare together with the preceptor and other healthcare professionals.

Assessment and feedback were integrated into daily practice

Students described multiple encounters each day thatled to preceptors watching, reviewing and correctingtheir clinical performance. Many of these encountersreferred to undifferentiated patient problems(students estimated they saw 15 patients per day),ensuring that the student and preceptor were bothdrawing on new areas of knowledge and skills andreinforcing established skills such as building patientrapport. The context of delivering care alongside thesame preceptor throughout the year afforded ampleopportunity for frequent and immediate informaldiscussion that built on previous feedback ofperformance:

Table 1 Themes and sub-themes

Themes Sub-themes

1 Assessment and feedback that arose from

students’ participation in practice over time

were perceived by students as authentic

Assessment and feedback were integrated into daily practice

Assessment and feedback were targeted to the student’s development

Assessment and feedback were built on earlier assessments by the same preceptor

2 Relationships with preceptors influenced the

interpretation of feedback and assessment

Trusting relationships with preceptors afforded honesty, self-reflection and

constructive interpretation of critical feedback

Trusting relationships with preceptors created a safe learning environment

Trusting relationships also developed with other members of the medical community

Trusting relationships with preceptors led to shared reflection on student progress

Students were concerned that the close relationships with preceptors might bias

feedback

3 Continuity of context provided multiple

indicators of progress

Students used patient outcomes as an important source of feedback

Students interpreted their increasing responsibility for patient care as feedback on

their progress

Students described their own feelings of increasing competence as feedback

Students interpreted patients’ gradual acceptance of them as progress towards their

becoming doctors


J Bates et al

‘We pretty much reviewed every patient rightafterwards […] he would actually be in the roomwith me to – to see how I interacted with thepatient and how I performed the physical exami-nation. […] He would come in and he would listento my spiel on what I would say to the patient,evaluate my physical examination on the cardio-vascular system. And then he wouldn’t give anyinput at the time with the patient there, butafterwards in his office when we convened hewould say what I did well and what I didn’t do welland then give me suggestions for improvementnext time.’

(B2-2)

‘And then pretty much she does an assessmentalmost all the time because when […] I’m in theclinic with her […] we usually go in either togetherand we usually discuss the patient right in front ofthe patient […] And then she tells me, ‘‘You know,I think maybe you should have asked a little morethis or you should have asked that.’’ […] So shepretty much almost – it almost feels like she’sassessing all the time.’

(C1-1)

Assessment and feedback were targeted to the student’sdevelopment

Through this process of participation in the healthcare community, feedback was targeted to thespecific knowledge and skills that each individualstudent needed to acquire in order to develop into adoctor. Students reported that preceptors and oth-ers were able to recognise and acknowledge theirdevelopment as well as their weaknesses because thesame individuals assessed them over the course ofthe year. Irrespective of formal learning objectives,learning in the practice setting moved the studentcloser to his or her objective of becoming a doctor.This rendered the assessment and feedback mean-ingful because it was derived from direct observationof patient care and was specific to helping thestudent learn:

‘The day to day […] clinical duties and just yourconversations with them […] you definitely had asense of feedback for sure on areas of strength andweaknesses […] if you’re weak in a particular area[…], it’s highlighted that that’s not an area ofstrength. And your preceptor will point that outand say […] you need to read about this or youneed to do this.’

(A2-2)

‘I think that was probably the majority of thefeedback that I got, that it was just sort of as wewere going along and we just sort of discussedpatients as we were going along. And with thatdiscussion came learning objectives for me to learnaround or things that I could work on or thingsthat I was doing well at.’

(D2-2)

‘They’re actually paying attention to my skill set,and I knew that that was something that I was weakin and my clinical evaluation reflected that [...] Youdo get a fair assessment, because the preceptorsknow you so well and because they’re workingdirectly with you, they’re the ones that are evalu-ating you.’

(A4-1)

Assessment and feedback were built on earlier assessments bythe same preceptor

Students talked about the importance of continuityof assessment and feedback. They emphasised howimportant it was that someone knew the trajectoryof their learning and could compliment andcritique their work, based on the longer view andthe longitudinal nature of the relationship:

‘Having a common person for feedback through-out the year allowed me […] a sounding board thatI felt, you know, somebody who had been there atthe beginning, and somebody who had been therein the middle and somebody who had been there atthe end. So somebody who could sort of criticallyassess how I was doing throughout.’

(A3-2)

‘When I get feedback on how I’m suturing […] oron how I’m doing a history or a physical and thoseindividuals who are supervising you, if they give mefeedback to say yes, you’re really improving, you’redoing a great job, I think that that’s really legiti-mate feedback. Because they’ve seen where I’vestarted in September […] they see this progression[…] by the time we get to August, if someone says,‘‘Yeah, like, you’ve really improved,’’ I’m going totake something like that at face value.’

(B1-1)

Theme 2: relationships with preceptors influencedthe interpretation of feedback and assessment

Another critical aspect of assessment and feedbackfrom the students’ point of view was the nature of



their relationships with preceptors and other healthcare team members. Relationships were described assupportive and caring, providing students with asense of safety in which to reflect on their perfor-mance and to interpret even critical feedback assupportive. Sub-themes within this theme provideboth evidence and elaboration of the influence ofrelationships upon students’ interpretation of feed-back and assessment as authentic to their learning,including the possibility that positive relationshipsmay have biased the feedback students received.

Trusting relationships with preceptors afforded honesty, self-reflection and constructive interpretation of critical feedback

Students described how the relationship they hadwith their preceptor encouraged honesty, self-reflec-tion and the interpretation of critical feedback assupportive, rather than challenging of their compe-tency:

‘It’s truly constructive criticism. It’s not negative;it’s there to help you become a better physician.And I think it comes from the heart, in a sense. Itcomes from people who are truly interested in yourlearning and it comes from people who want tohelp make that difference and show you what’sright and what’s wrong.’

(A4-1)

‘It’s actually quite different. It’s very different.[The] most important aspect is that it feels likesomeone actually cares for my grades and myperformance. It’s never been evaluated and lookedinto in a good aspect, this closely before. Not onlyin medical school, but also in university in general.You’re a number in amongst the thousands. So youget your grading, you pass, no-one really cares. Youfail, well maybe someone cares because they didn’twant – they think that you’re not performing andthey’re wondering whether your seat should betaken up by someone else. But no-one really sitsdown with you and say[s], ‘‘Hey, what’s notworking and what can we do?’’ So that’s the biggestdifference that I find. It’s never really happened tome before this year.’

(A1-1)

Trusting relationships with preceptors created a safelearning environment

When talking about assessment and feedback, stu-dents described a safe learning environment in whichthey could acknowledge when they were ‘stuck’ with a

patient or out of their depth and the safety they feltwhen admitting that to their preceptor:

‘With the preceptor very often I say […] you know,I really don’t know. I don’t even know what’s goingon […] I’ve never heard of that.’

(C1-1)

‘I think that I felt quite comfortable telling mypreceptor that I’m in over my head or I need theirhelp to manage something or try to figure outsomething that I didn’t understand.’

(D2-2)

‘They can be honest with you, you can evaluate yourtrue weaknesses, and at the same time, you cancelebrate your strengths. Because again, it’s in thatsafe learning environment where even when you’rebeing criticised, you feel like the reason you’re beingcriticised is because they’re trying to help you learnand they’re trying to give you those skills.’

(A4-1)

The trusting relationship allowed students toacknowledge their feelings of not knowing what to donext during clinical encounters and opened uppossibilities for teaching that could be targeted to thestudents’ level of development.

Trusting relationships also developed with other members ofthe medical community

Students described a positive relationship, particularlywith their preceptor, as key to the provision of a safelearning environment. However, preceptors were notthe only members of the community with whomstudents developed important relationships as support-ive relationships developed across the health care team:

‘…the relationship you develop with your precep-tors and with your medical community […] makesall the other things really strong. Like, assessmentand, you know, feedback and clinical abilities andknowing your limits, all that stuff, I mean, I thinkthe underlying theme is really all – how good yourrelationship is with your community, whether it’snurses or physicians or preceptors or residents orwhatever.’

(A2-2)

Although these relationships were noted as impor-tant, the student–preceptor relationship remainedcentral to students’ references to a ‘trusting rela-tionship’ within which they were assessed and givenfeedback.


J Bates et al

Trusting relationships with preceptors led to sharedreflections on student progress

Within the theme of the trusted relationshipemerged the concept of shared responsibility forstudent progress that included reflection on thatprogress. Students described multiple experiences ofsitting down with preceptors to discuss weaknessesand map out future learning objectives and oppor-tunities:

‘And then we also talk about the assessments and[…] these goals can be set out or suggested orthings can be planned. Because they know thatwe’ll be coming back and then go, okay, you know,in the first time around, we’ve worked on this sofar, next time I’m going to see you, we’ll try to workon these as well and we’ll push the limits further…’

(A1-1)

‘Each one of us interact[s] with almost every singlephysician here that’s involved in teaching and wesee them throughout the year. They know who weare, we know who they are and we communicate ona very professional, but also an informal level aswell. They treat us as colleagues, which is a rarity, asa third-year medical student.’

(A1-1)

The opportunity to have this discussion embedded ina longitudinal relationship with a preceptor withshared responsibility for correcting deficienciesemerged as a key component of the relationship.Students trusted the preceptors’ monitoringprocesses and reported that the student and precep-tor together planned the educational interventionsrequired. Because of the shared responsibility forstudents’ progress, the assessments were experiencedas truly formative.

Students were concerned that close relationships withpreceptors might bias feedback

Such close relationships also raised concerns aboutpossible bias in that some students wondered iffeedback might be biased by the positive collegialrelationships they had developed with theirpreceptors:

‘And then I guess our relationship, like, we getalong really well and I – and so in terms of when hegives me feedback then it’s hard for me to assesswhether or not it’s just because we get along reallywell and he likes me and it’s, like, just the easyprofessional relationship.’

(D2-1)

‘I think that quite possibly the continuity inhib-ited me from getting constructive feedbackbecause they knew me well enough and maybe itwas more difficult for them to compare me toother students ’cause they hadn’t seen otherstudents in a while…’

(D2-2)

Although such concerns did not appear to be wide-spread according to student comments, they did leadto the valuing of the intermittent visits by faculty stafffrom the central programme for the purposes ofassessing student progress.

Theme 3: continuity of context provided multipleindicators of progress

This theme speaks of indicators of progress derivedfrom continuity of context and another authenticsource of feedback: patient outcomes. Continuity ofcontext throughout the clerkship meant that studentswere able to see patients repeatedly and over time.Patient outcomes were perceived as representing adirect form of authentic feedback about students’development as future doctors. The continuity ofcontext also allowed for increasing responsibility forthe provision of care and an increasing sense ofcompetence. For some students, it also meant thatthey were increasingly accepted as doctors by ‘their’patients.

Students used patient outcomes as an important source offeedback

Seeing patients repeatedly over time allowed studentsto determine whether or not their diagnoses andmanagement plans were optimal for their patients.Actual patient outcomes as a result of their clinicaldecision making were trusted not only as a source offeedback, but as a valued source of formative assess-ment about what to focus on for further develop-ment:

‘Certainly seeing the patients in different environ-ments helps, because […] being able to go throughthat process, conceptualise the problem, come upwith a structure, think it through and then giveyour sort of: ‘‘Here’s my diagnosis and here’s whatI think we should do.’’ I think all of us are learninghow to do that this year. I think the difference forus is that then we see that patient again, a monthlater, 2 months later, 3 months later, and we canreally follow through on that clinical reasoning. Sowere we correct? If we weren’t, what did we dowrong? What could we have done differently? How



is that patient doing now? What kind of specialistshave they seen? […] I think that’s huge in buildingclinical confidence, in really seeing the outcomesand seeing that you’re clinically appropriate andthat your patients are doing well because of it.’

(A4-1)

Students felt that there was no more authentic sourceof feedback than a patient’s lack of improvement orprogress. They interpreted their patients’ progress asgiving direct indications of their own progress asdeveloping doctors.

Students interpreted their increasing responsibility forpatient care as feedback on their progress

Students experienced being given increased respon-sibility for patient care and interpreted this as anindicator of their progress towards becoming adoctor. As a result of the reciprocity of trust thatdeveloped between students and preceptors withongoing supervision over time, students wereencouraged to do more and to take more responsi-bility with less direct supervision. Students alsoexperienced increased trust in their abilities on thepart of nurses and residents over time:

‘And it’s also pushing yourself because we alwaysknow more than we give ourself credit for, sohaving someone beside you, for example, theanaesthetist to say, ‘‘I think you could do a spinal;go ahead and try it.’’ For me, if that person hadn’tsaid, ‘‘You can do it, try it,’’ I would have said, ‘‘No,I don’t think I’m comfortable with it.’’ But theysaid that, ‘‘From what I’ve seen of you, you can doit, so I’ll be right beside you and watch[ing].’’’

(A2-2)

Students described their own feelings of increasingcompetence as an important source of feedback on theirprogress

Students also recognised their own increasing capa-bility over time as a meaningful indicator of their ownprogress. As the year progressed, they were increas-ingly able to provide a full consultation, includingtaking a history, conducting a physical examination,and delivering a diagnosis, management strategy andpatient education. Along with this increasing sense ofcompetence came an increasing sense of identity as alegitimate member of the health care team:

‘I am standing in the Emerg, I’ve got a couple ofstacks of charts going for either doctor that’s oncall for the Emerg. I’m calling one doc and giving

them the rundown, calling the other doc andgiving them the rundown on the patients that I’vegot there. And I stopped at one point and Ithought, ‘‘How did I get here? How did I get to thispoint where I’m able to do this?’’ Because back inSeptember, October, December even February, Icouldn’t have done that. I wasn’t able to do that.But somehow, this ability to manage those casesand know what to do or what to recommend, thatjust sort of sneaked up on me.’

(B1-2)

Students interpreted patients’ gradual acceptance of them asindicative of their progress towards becoming a doctor

Particularly important to students were indicationsthat patients were also gaining confidence in them.These indications were central to the students’development of a doctor identity:

‘And, you know, I’ve had patients that refused tosee a doctor until they see me and […] thatwouldn’t happen early in the programme. Thatwouldn’t happen in midway through either.’

(D1-1)

DISCUSSION

We began by noting that there is little evidence in theliterature to describe how students in longitudinalclinical placements experience assessment and feed-back. We found that the embedded nature oflearning afforded by long-term clinical placementsprovided an essential structure for informal assess-ment and meaningful feedback. Students in LICsperceived more structured formative assessment andfeedback such as those derived from mini-CEXs torepresent programme requirements, but to be intru-sive and less valuable than the judgements about theirperformance and resulting feedback that occurred inthe course of daily clinical work. In addition to directfeedback from preceptors, students also perceivedthat important information about their general pro-gress was derived from the level of responsibility forpatient care they were granted, their own developingfeelings of competence, and the acceptance of thehealth care community.

They experienced this unstructured assessment andinformal feedback as authentic and useful becausethey arose from the students’ direct participation inshared patient care, enabled by continuity of bothpreceptor and context over time. From the students’point of view, ‘authenticity’ was a product of the


J Bates et al

provision of assessment and feedback in the course ofauthentic work. However, it was also necessary thatthis assessment and feedback were targeted to eachstudent’s level of development and contributed to hisor her personal and emergent scaffolding for learn-ing, thus supporting the student’s growth anddevelopment over time.

In this context, students frequently engaged inreflection on their strengths and limitations withtheir preceptors. Students perceived critical feedbackas supportive because preceptors shared the respon-sibility for addressing weaknesses with the students.Students experienced multiple indicators of theirprogress towards clinical competency arising from thehealth care environment.

These findings are consistent with existing literatureabout the pedagogical role, interpersonal dynamics,the process, and the fairness of assessment andfeedback in medical education. In clinical place-ments, a key purpose of assessment is to producefeedback that has the potential to drive studentlearning.40 Workplace-based formative assessment iscurrently characterised by instruments designed toincrease the direct observation of trainees, to struc-ture feedback and to contribute to learning goals.41

As Norcini and Burch42 point out, these instrumentsmay focus on the assessment of students’ learningrather than on assessment for the purpose ofproviding useful feedback.42 Although direct obser-vation is a strategy for deriving formal feedback,clinical supervision provides an important source ofmonitoring and feedback for trainees.43 Norcini andBurch42 characterise such feedback as effective whenit focuses on important aspects of performance in theworkplace and when it is consistent with the needs ofthe learner. Clinical preceptors in LICs can tailorfeedback and teaching to each student because theygain insights into student strengths and weaknessesthroughout the year.44

We also found that students accept and use criticalfeedback when it is delivered in the context of anongoing supportive relationship in which responsi-bility for addressing deficiencies is shared; underthese circumstances critical feedback is described asbeing intended to draw the student more clearly intothe clinical community, rather than deeming him orher not yet competent. Seen through the lens ofcommunities of practice theory, critical feedback isintended to move the student from legitimateperipheral participation to a more active participatoryrole. If the pedagogical focus of assessment andfeedback during the clerkship year is to guide student

learning, then engaging the student in his or her owndevelopment becomes of paramount importance. AsBoud45 outlines in his assessment principles for thefuture, assessment has the greatest effect when itderives from a partnership between student andpreceptor and is used to engage the student inproductive learning. The importance of such part-nerships has been highlighted by students in theirpositive descriptions of multiple opportunities forreflection on their progress.46 The quality of therelationship enables a level of honesty that is difficultto achieve in usual feedback processes.42 Althoughmost discussions of formative feedback focus onpreceptor skills in delivering feedback,47 we foundthat the style of delivering feedback was irrelevant tothe students when the relationship with the preceptorwas experienced as supportive.

The relationships that developed between studentsand their preceptors often supported an environ-ment in which students were able to focus onlearning goals rather than performance goals.Learning goals are intended to increase one’s com-petence or proficiency and reflect a desire to learnmore and to improve one’s skill, whereas performancegoals are about demonstrating competence andavoiding negative judgement from others.48

According to Dweck,48 tasks that are best forlearning often involve taking risks or displayingignorance. Tasks that are best for appearing com-petent or smart often involve activities that carrylittle or no risk because people are already good atthem. Although we have no direct evidence of this,we suggest that students in our study chose alearning orientation more often than a performanceorientation and that they did so because: (i) theywere in a longitudinal clerkship and had time ontheir side; (ii) they would find multiple opportuni-ties to demonstrate competence; (iii) informalassessments came from trusted sources, and (iv)feedback and guidance were scaffolded to theindividual student’s level of need.

Engaged longitudinally in the joint enterprise ofpatient care, students described a trajectory of growthfrom their status as ‘student’ at the periphery of theenterprise through the gradual development of amore central role as ‘colleague’. This trajectory oflearning, development and emerging identity as alegitimate member of the community is captured inthe collective themes and voices of our students. Inthis sense, we heard a description of professionaldevelopment as involving the cultivation of both asense of competence and a sense of professionalidentity. This description resonates with Wenger’s37



theory of the process of inclusion into a communityof practice.

Some students alluded to changing interactions withpatients, as well as other health professionals, thatconfirmed their evolving identity as a doctor. Identityand competence, in this sense, are reciprocal andhighly related concepts.37 Both graded responsibilityand patient trust were interpreted as evidence of thestudent’s evolving competence. We found thatassessment and feedback are not just about perfor-mance; they also refer to the student’s evolvingidentity, membership and legitimacy in a communityof practice. Understanding how feedback, and espe-cially critical feedback, might contribute to andinteract with the developing identity of medicalstudents as doctors is critical to the further develop-ment of feedback strategies in medical education.49

Taken separately, the three themes reveal importantaspects of what students said about their experiencesof assessment and feedback during their LICs. Takentogether, they help us see more clearly, from thestudents’ point of view, how assessment and feedbackcan become not just meaningful, but central to thedevelopment of competence and identity.

CONCLUSIONS

As Strasser and Hirsh50 comment, LICs have thepotential to transform medical education worldwide.Although we do not advocate that all students shouldbe trained in LICs, we contend that this newclerkship structure allows us to examine thornyissues about student acceptance of assessment andfeedback in clinical education. Students studying inLICs perceive their assessment and feedback asauthentic because they are embedded in dailypatient care, useful because they are developmentaland longitudinal, and constructive because theyoccur in the context of a supportive learningenvironment and relationships. Their perception ofprogress towards their goal of being able to provideclinical care as a doctor is formed by multipleindicators derived from the learning environmentand leads to an increasing sense of professionalidentity. Irrespective of the specific clerkship struc-ture that led to these findings, however, all clinicallearning environments might take heed of theseaccounts of students’ experiences. Given the chal-lenges associated with the acceptance of criticalfeedback and the development of reflective practi-tioners, our study contributes to the discussion onhow to achieve these outcomes with students in all

clinical environments. The move in medicaleducation towards more formal assessment processesand the training of faculty members to deliverformative feedback, despite being driven by univer-sity and accreditation requirements, may not servetheir intended pedagogical purpose unless continu-ity is built into both clinical supervision and thelearning environment. We need to think beyondcurrent perceptions of assessment and feedback as aset of processes and skills, and recognise thateffective assessment and feedback must beembedded within supportive learning environmentsand relationships. This shift in thinking isparticularly important for students’ acceptance ofcritical feedback and development of reflectivepractice.

Our study also raises further questions for investiga-tion. More explanation is needed as to how relation-ships affect assessment and feedback, and whetherthese informal processes actually drive studentlearning. Although we speculate in our study aboutthe link between assessment and feedback in theservice of identity formation, this link requires moreexploration. To our knowledge, the importance ofpatient indicators to students of their progress hasnot been previously described. Our study findingshave implications not just for further investigation,but also for clinical education and for understandingof the nature of assessment and feedback. The lack ofcontinuity that bedevils the undergraduate clinicalteaching environment undermines not studentlearning alone, but the assessment and feedback thatunderlie learning.

Contributors: JB and JK were involved in the conceptionand design of the study. JB, DP and SD contributed to theacquisition of data. All authors contributed to the analysisand interpretation of data. JB and DP drafted the paper. Allauthors contributed to the critical revision of the paper andapproved the final manuscript.Acknowledgements: the authors would like to acknowledgethe help of Lucy Chen, Centre for Health EducationScholarship, University of British Columbia, in preparingthe manuscript, and of Tim Dornan, Department ofEducational Development and Research, Faculty of Health,Medicine and Life Sciences, Maastricht University, indeveloping of the manuscript.Funding: this study was funded by the Medical Council ofCanada.Conflicts of interest: none.

Ethical approval: this study received ethics approval fromthe Behavioural Research Ethics Board, University of BritishColumbia, and the Health Research Ethics Board,University of Alberta.


J Bates et al

REFERENCES

1 Armstrong EG, Mackey M, Spear SJ. Medical educationas a process management problem. Acad Med2004;79:721–8.

2 Irby D. Educational continuity in clinical clerkships. NEngl J Med 2007;356:856–7.

3 O’Brien B, Cooke M, Irby DM. Perceptions and attri-butions of third-year student struggles in clerkships: dostudents and clerkship directors agree? Acad Med2007;82:970–8.

4 Hirsh DA, Ogur B, Thibault GE, Cox M. ‘Continuity’ asan organising principle for clinical education reform. NEngl J Med 2007;356:858–66.

5 Bernabeo EC, Holtman MC, Ginsburg S, RosenbaumJR, Holmboe ES. Lost in transition: the experience andimpact of frequent changes in the in-patient learningenvironment. Acad Med 2011;86:591–8.

6 Cooke M, Irby DM, O’Brien BC. Educating physicians: acall for reform of medical school and residency, 1st edn. SanFrancisco, CA: Jossey-Bass 2010;320 pp.

7 Hansen LA, Talley RC. South Dakota’s third-year pro-gramme of integrated clerkships in ambulatory caresettings. Acad Med 1992;67:817–9.

8 Worley P, Silagy C, Prideaux D, Newble D, Jones A. Theparallel rural community curriculum: an integratedclinical curriculum based in rural general practice. MedEduc 2000;34:558–65.

9 Peters AS, Feins A, Rubin R, Seward S, Schnaidt K,Fletcher RH. The longitudinal primary care clerk-ship at Harvard Medical School. Acad Med2001;76:484–8.

10 Ogur B, Hirsh D, Krupat E, Bor D. The Harvard Med-ical School–Cambridge integrated clerkship: an inno-vative model of clinical education. Acad Med2007;82:397–404.

11 Mihalynuk T, Bates J, Page G, Fraser J. Student learningexperiences in a longitudinal clerkship programme.Med Educ 2008;42:729–32.

12 Zink T, Halaas GW, Finstad D, Brooks KD. The ruralphysician associate programme: the value of immersionlearning for third-year medical students. J Rural Health2008;24:353–9.

13 Norris TE, Schaad DC, DeWitt D, Ogur B, Hunt DD.Longitudinal integrated clerkships for medical stu-dents: an innovation adopted by medical schools inAustralia, Canada, South Africa, and the United States.Acad Med 2009;84:902–7.

14 Worley P, Esterman A, Prideaux D. Cohort study ofexamination performance of undergraduate medicalstudents learning in community settings. BMJ2004;328:207–9.

15 Ogur B, Hirsh D. Learning through longitudinalpatient care – narratives from the Harvard MedicalSchool–Cambridge integrated clerkship. Acad Med2009;84:844–50.

16 Bell SK, Krupat E, Fazio SB, Roberts DH, SchwartzsteinRM. Longitudinal pedagogy: a successful responseto the fragmentation of the third-year medical

student clerkship experience. Acad Med 2008;83:467–75.

17 Mazotti L, O’Brien B, Tong L, Hauer KE. Perceptionsof evaluation in longitudinal versus traditional clerk-ships. Med Educ 2011;45:464–70.

18 Fromme HB, Karani R, Downing SM. Direct observa-tion in medical education: a review of the literatureand evidence for validity. Mt Sinai J Med 2009;76:365–71.

19 Barrows HS. The scope of clinical education. In: Coo-per JAD, ed. Clinical education of medical students. J MedEduc 1986;61:23–33.

20 Haber RJ, Avins AL. Do ratings on the American Boardof Internal Medicine resident evaluation form detectdifferences in clinical competence? J Gen Intern Med1994;9:140–5.

21 Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann InternMed 2003;138:476–81.

22 Fernando N, Cleland J, McKenzie H, Cassar K. Identi-fying the factors that determine feedback given toundergraduate medical students following formativemini-CEX assessments. Med Educ 2008;42:89–95.

23 Holmboe ES, Yepes M, Williams F, Huot SJ. Feedbackand the mini-clinical evaluation exercise. J Gen InternMed 2004;19:558–61.

24 Barclay LJ, Skarlicki DP, Pugh SD. Exploring the roleof emotions in injustice perceptions and retaliation.J Appl Psychol 2005;90:629–43.

25 Higgins R, Hartley P, Skelton A. Getting the messageacross: the problem of communicating assessmentfeedback. Teach Higher Educ 2001;6:269–74.

26 Bing-You Rg TRL. Why medical educators may befailing at feedback. JAMA 2009;302:1330–1.

27 Higgins RSD, Bridges J, Burke JM, O’Donnell MA,Cohen NM, Wilkes SB. Implementing the ACGMEgeneral competencies in a cardiothoracic surgery resi-dency programme using 360-degree feedback. AnnThorac Surg 2004;77:12–7.

28 Lingard L, Schryer C, Garwood K, Spafford M. ‘Talkingthe talk’: school and workplace genre tension inclerkship case presentations. Med Educ 2003;37:612–20.

29 Watling CJ, Kenyon CF, Zibrowski EM, Schulz V,Goldszmidt MA, Singh I, Maddocks HL, Lingard L.Rules of engagement: residents’ perceptions of thein-training evaluation process. Acad Med 2008;83(Suppl):97–100.

30 Torre DM, Simpson D, Sebastian JL, Elnicki DM.Learning ⁄ feedback activities and high-quality teaching:perceptions of third-year medical students during anin-patient rotation. Acad Med 2005;80:950–4.

31 Greenberg LW, Goldberg RM, Jewett LS. Teaching inthe clinical setting: factors influencing residents’perceptions, confidence and behaviour. Med Educ1984;18:360–5.

32 Charmaz K. Qualitative interviewing and groundedtheory analysis. In: Holstein JA, Gubrium JF, eds. Inside



Interviewing: New Lenses, New Concerns. Thousand Oaks,CA: Sage Publications 2003;347–65.

33 Boud D, Falchikov N. Rethinking assessment in highereducation: learning for the longer term. New York, NY:Routledge 2007.

34 Schuwirth LWT, van der Vleuten CPM. Changingeducation, changing assessment, changing research?Med Educ 2004;38:805–12.

35 Wiggins GP. Assessing student performance: exploring thepurpose and limits of testing, 1st edn. San Francisco, CA:Jossey-Bass 1993;336 pp.

36 Lave J, Wenger E. Situated Learning: Legitimate PeripheralParticipation. New York, NY: Cambridge University Press1991;123 pp.

37 Wenger E. Communities of Practice: Learning, Meaning,and Identity. Cambridge: Cambridge University Press1998;318 pp.

38 Srivastava P, Hopwood N. A practical iterative frame-work for qualitative data analysis. Int J Qual Methods2009;8:76–84.

39 Kennedy TJT, Regehr G, Baker GR, Lingard L. Point-of-care assessment of medical trainee competence forindependent clinical work. Acad Med 2008;83 (Sup-pl):89–92.

40 van der Vleuten CPM. The assessment of profes-sional competence: developments, research andpractical implications. Adv Health Sci Educ 1996;1:41–67.

41 Kogan JR, Holmboe ES, Hauer KE. Tools for directobservation and assessment of clinical skills ofmedical trainees: a systematic review. JAMA2009;302:1316–26.

42 Norcini J, Burch V. Workplace-based assessment as aneducational tool: AMEE Guide No. 31. Med Teach2007;29:855–71.

43 Kilminster SM, Jolly BC. Effective supervision in clinicalpractice settings: a literature review. Med Educ 2000;34:827–40.

44 Teherani A, O’Brien BC, Masters DE, Poncelet AN,Robertson PA, Hauer KE. Burden, responsibility, andreward: preceptor experiences with the continuity ofteaching in a longitudinal integrated clerkship. AcadMed 2009;84 (Suppl):50–3.

45 Boud D. Student Assessment for Learning In and AfterCourses. Sydney, NSW: Australian Learning and Teach-ing Council 2010;1–31.

46 Schon DA. Educating the Reflective Practitioner: Toward aNew Design for Teaching and Learning in the Professions, 1stedn. San Francisco, CA: Jossey-Bass 1987.

47 Norcini J. The power of feedback. Med Educ 2010;44:16–7.

48 Dweck CS. Self-Theories: Their Role in Motivation, Person-ality, and Development. Philadelphia, PA: PsychologyPress 2000;195 pp.

49 Monrouxe LV. Identity, identification and medicaleducation: why should we care? Med Educ 2010;44:40–9.

50 Strasser R, Hirsh D. Longitudinal integrated clerkships:transforming medical education worldwide? Med Educ2011;45:436–7.

Received 8 March 2012; editorial comments to authors 17 April2012, 28 June 2012; accepted for publication 26 September 2012


J Bates et al

student perceptions of assessment and feedback in longitudinal integrated clerkships

Documents