open access research clinical instructors …while teaching were learnt during circuit and itinerant...

Clinical instructors’ perception ofa faculty development programmepromoting postgraduate year-1 (PGY1)residents’ ACGME six corecompetencies: a 2-year study

Fa-Yauh Lee,1 Ying-Ying Yang,2,3 Hui-Chi Hsu,2,3 Chiao-Lin Chuang,2

Wei-Shin Lee,2 Ching-Chih Chang,2 Chia-Chang Huang,4 Jaw-Wen Chen,4

Hao-Min Cheng,4 Tjin-Shing Jap1

ABSTRACTObjective: The six core competencies designated byAccreditation Council for Graduate Medical Education(ACGME) are essential for establishing a patient centreholistic medical system. The authors developeda faculty programme to promote the postgraduate year1 (PGY1) resident, ACGME six core competencies. Thestudy aims to assess the clinical instructors’perception, attitudes and subjective impressiontowards the various sessions of the ‘facultydevelopment programme for teaching ACGMEcompetencies.’

Methods: During 2009 and 2010, 134 clinicalinstructors participated in the programme to establishtheir ability to teach and assess PGY1 residents aboutACGME competencies.

Results: The participants in the faculty developmentprogramme reported that the skills most often usedwhile teaching were learnt during circuit and itinerantbedside, physical examination teaching, mini-clinicalevaluation exercise (mini-CEX) evaluationdemonstration, training workshop and videotapes of‘how to teach ACGME competencies.’ Participantsreported that circuit bedside teaching and mini-CEXevaluation demonstrations helped them in theinterpersonal and communication skills domain, andthat the itinerant teaching demonstrations helped themin the professionalism domain, while physicalexamination teaching and mini-CEX evaluationdemonstrations helped them in the patients’ caredomain. Both the training workshop and videotapesession increase familiarity with teaching andassessing skills. Participants who applied the skillslearnt from the faculty development programme themost in their teaching and assessment came frominternal medicine departments, were young attendingphysician and had experience as PGY1 clinicalinstructors.

Conclusions: According to the clinical instructors’response, our faculty development programmeeffectively increased their familiarity with variousteaching and assessment skills needed to teach PGY1

residents and ACGME competencies, and these clinicalinstructors also then subsequently apply these skills.

INTRODUCTIONThe outbreak of the severe acute respiratorysyndrome (SARS) epidemic in 2003 exposed

To cite: Lee F-Y, Yang Y-Y,Hsu H-C, et al. Clinicalinstructors’ perception ofa faculty developmentprogramme promotingpostgraduate year-1 (PGY1)residents’ ACGME six corecompetencies: a 2-yearstudy. BMJ Open 2011;1:e000200. doi:10.1136/bmjopen-2011-000200

< Prepublication history forthis paper is available online.To view these files pleasevisit the journal online (http://bmjopen.bmj.com).

Received 1 June 2011Accepted 1 September 2011

This final article is availablefor use under the terms ofthe Creative CommonsAttribution Non-Commercial2.0 Licence; seehttp://bmjopen.bmj.com

1Department of Medicine,Taipei Veterans GeneralHospital, Taipei, Taiwan2Division of GeneralMedicine, Taipei VeteransGeneral Hospital, Taipei,Taiwan3Department of Medicine,National Yang-MingUniversity, Taipei, Taiwan4Department of MedicalResearch and Education,Taipei Veterans GeneralHospital, Taipei, Taiwan

Correspondence toDr Ying-Ying Yang;[email protected]

ARTICLE SUMMARY

Article focus- In order to train PGY1 residents, we need to help

clinical instructors to become familiar with theteaching and assessment skills that form theAccreditation Council for Graduate MedicalEducation six core-competencies.

- Our study used a self-reported questionnairesbased analysis to evaluate the clinical instructors’perception to our faculty development programme.

Key messages- Participants reported that their most commonly

used skills were learnt from itinerant and circuitbedside teaching, and mini-clinical evaluationexercise evaluation demonstration in ourprogramme.

- Participants also reported that the 40 h basictraining course improved their abilities to trainand assess PGY1 residents in patient care,interpersonal and communication skills, andmedical knowledge domains whereas postcoursetraining workshop and videotape sessionenhanced their ability in system-based practice,practice-based learning and improvement, andprofessionalism domains.

- A serial follow-up questionnaire suggested thatthe degree of participant application of skillslearnt from our programme increased progres-sively after finishing the 40 h basic trainingcourse, the postcourse training workshop andvideotape session.

Lee F-Y, Yang Y-Y, Hsu H-C, et al. BMJ Open 2011;1:e000200. doi:10.1136/bmjopen-2011-000200 1

Open Access Research

on August 22, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2011-000200 on 24 N

ovember 2011. D

ownloaded from

http://bmjopen.bmj.com/

serious deficiencies in Taiwan’s healthcare and medicaleducation systems.1 A project entitled the ‘PostgraduateGeneral Medical Training Programme’ was announcedby the Department of Health (DOH), Executive Yuan inAugust 2003. In this project, each PGY1 resident isrequired to complete a 6-month holistic healthcare-centred training programme before entering theirsubspecialist courses.2

In 2006, Taiwan Association of Medical Education(TAME) helped many teaching hospitals to establish‘General Medical Training Demonstration Centres’ asmeans of improving medical education of clinicalinstructors, residents and medical students. Additionally,the DOH arranged to extend the current 6-monthpostgraduate general medicine training programme toa 1-year programme.3 After 2009, the DOH addeda ‘Training Programme for Clinical Instructors’ to the‘General Medical Training’ programme to help preparethe clinical instructors needed to train the more than1300 PGY1 residents every year during the 1-year PGY1programmes.4 5 The goal of the programme is to helpclinical instructors such that they have the ability toteach to PGY1 residents the Accreditation Council forGraduate Medical Education (ACGME) core compe-tencies, including medical knowledge, interpersonal andcommunication skills, system-based practice, practice-based learning and improvement, professionalism andpatient care (http://www.acgme.org).6

Taipei Veterans General Hospital (VGH) is a regionalmedical centre that provides primary and tertiary care toactive-duty and retired military personnel and theirdependents, and is the primary teaching hospital for itsgeneral medicine residency programme.Our hospital has continuously received economic

support from the DOH for the ‘General MedicineTraining Demonstration Center Programme’ since 2006.Over the past few years, we have deployed teachingresources and faculty development programmes atTaipei VGH to help establish a patient-centred health-care system.7 Since 2009, Taipei VGH has aggressivelyimplemented a ‘Clinical Instructor Competencies’faculty development programme and sought to boost theskills of clinical instructors teaching competencies thattarget PGY1 residents.

This study uses data collected from the self-reportedquestionnaires given to all participants in this facultydevelopment programme, and in it we have sought toevaluate the participants’ feelings about the value of thevarious training sessions. Moreover, we have exploredthe contributions of the different activities of thisprogramme to instilling ACGME competency teachingabilities in the participants. Finally, we have assessed thedegree to which the abilities learnt within thisprogramme by participants are applied.

MATERIALS AND METHODSBackground of postgraduate training of doctor in TaiwanThe system for postgraduate training of doctor is similarto the American model, which was developed by theACGME. The postgraduate training of doctor is reg-ulated by DOH, Executive Yuan of Taiwan ROC.The expectations of the postgraduate training of doctorare that residents are competent in the six corecompetencies.

ParticipantsThis study involved 134 clinical instructors (physicians)in 10 specialties at Taipei VGH and cooperatinghospitals between January 2009 and January 2011. Allclinical instructors participated in the training coursevoluntarily.

Content of the ‘faculty development programme of ACGMEcompetencies for clinical instructors’Overall, the design of the content of faculty develop-ment programme by TAME was based on the coherenteducational theory proposed by Hewson.8 It has beensuggested that the programme should include video-tapes, mini-lectures, group discussions, demonstrations,role play and simulated teaching experiences in order topromote a change in the attitudes, values, beliefs andassumptions about teaching of clinical instructors.8

Accordingly, TAME announced that the facultydevelopment programme consisted of two parts, namelythe ACGME competencies-based 40 h basic clinical-practice training course and a postcourse trainingworkshop together with a videotape session. In general,the second part of training course is designed tore-enforce the efficacy of the faculty developmentprogramme. The postcourse training workshop forclinical instructors was held with the assistance ofTAME. The lectures in the training workshop emphas-ised teaching skills related to the competencies andlasted at least 7 h. The aims of the training workshopand videotapes were to teach and discuss the ACGMEcompetence-related teaching and assessment skills. Atthe same time, we design portfolios for all participantsto allow them to record and certify their trainingcourses. Finally, TAME certificated their teaching abilityaccording to the records within the portfolios.9 Thecourse can be implemented intermittently or continu-ously over a period of 3 months. All participants had

ARTICLE SUMMARY

Strengths and limitations of this study- According to the clinical instructors’ responses, our

programme effectively increased their familiarity with teachingand assessment skills needed when teaching PGY1 residents’Accreditation Council for Graduate Medical Education compe-tencies and that these skills were subsequently applies.

- This study was limited by the fact that questionnaire used totrack and assess the effectiveness of the training programmemay have had information and recall bias. In addition, thisstudy had a relatively small sample size and did not containa control group. However, no controlled educational trials onthis subject have been published as yet.

2 Lee F-Y, Yang Y-Y, Hsu H-C, et al. BMJ Open 2011;1:e000200. doi:10.1136/bmjopen-2011-000200

Clinical instructors training programme



jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


chances to demonstrate and practice the teaching andevaluation skills learnt in the course.

First part: 40 h basal training courseOutpatient department (OPD) teaching demonstrationThe application of competencies in OPD teaching wasdemonstrated once every week by a senior professor whois a member of the VGH educational committee. In thefirst hour, a senior professor demonstrated the teachingskills of seeing the patients, talking to families, inter-viewing and physical examination (PE). In the secondhour, PGY1 residents visited a real patient independently,made a short case presentation and completed thewriting of the charts. In the last hour, a senior professordemonstrated the teaching skills needed to createa care plan, how to carry out appropriate administrativeactivities and needs, and how to review the signs,orders and written notes placed on a chart by a PGY1resident.

Itinerant bedside teaching demonstrationItinerant bedside teaching was provided three times perweek by different senior professors whose are not takingcare of patients directly. In the first 0.5 h, PGY1 residentsgave a case presentation in the consulting room anddiscussed clinical reasoning, communication andproblem-solving skills with the senior professor andother clinical instructors. With the agreement of thepatient, the bedside teaching team visited the patients toconfirm the physical signs, make a provisional diagnosisand decide on the best diagnostic and therapeuticoptions in the following 1 h. Finally, in the last hour, thebedside teaching team returned to the consulting roomand discussed how to apply the knowledge gained,including diagnostic test results together with the abilityto interpret medical literature as well as the synthesis ofinformation skills in the last hour.

Circuit bedside teaching demonstrationCircuit bedside teaching was demonstrated five timesevery week. Circuit bedside teaching involved a medicalteam including medical students, interns, PGY1 residentsand an in-charge physician whose taking care of thepatients that discussed. Circuit bedside teaching wascarried out in the routine ward rounds. Before visitingthe patients, in-charge physicians demonstrated theteaching skills of history taking, PE and communicationabilities in the first 0.5 h. The medical team and partic-ipants demonstrated and practised the teaching skills ofevidence-based medicine and self-directed learningability in the following 1 h of ward runs. Finally, theteaching skills of administrative time management andrecord-keeping abilities were demonstrated and prac-tised by all participants.

PE teaching demonstrationPE teaching was demonstrated three times every week.The lecture content consisted of instructional skillsconcerning PE. In the first hour, a trained lecturer

demonstrated and practised teaching systemic PE skillsto PGY1 residents and clinical instructors. Over the nexthalf hour, trained lecturers, clinical instructors and PGY1residents visited real patients to verify the specific phys-ical signs reported by the PGY1 residents. In the last halfhour, all participants discussed the meaning and signif-icance of physical findings found by members of thePE teaching team.

Case-based discussion teaching demonstrationSenior professors and trained lectures also hosted case-based discussion (CbD) meetings five times each week.CbD mainly assesses the clinical reasoning and deci-sion-making abilities of PGY1 residents. The detailedaspects of evaluation include medical record keeping,clinical assessment, investigation and referrals, treat-ment, follow-up and future planning, professionalismand overall clinical judgement abilities of PGY1 resi-dents. In the first half hour, the PGY1 residents gavea case presentation to show their abilities. Next, thesenior professors, trained lectures and clinical instruc-tors discussed and interacted directly with the PGY1residents about their performance over the last halfhour.

Evidence-based medical teaching demonstrationOne-hour evidence-based medical (EBM) meetings totrain core competency were hosted by trained lecturersonce every week. During the first week of each month,a lecture was given to PGY1 residents and participants. Inthe second and third week, PGY1 residents presented theapplication of EBM skills in solving difficult clinicalproblems presented by their patients in weekly EBMmeeting. In the EBM meeting of the last week, partici-pants were invited to comment and demonstrate theirinstructional skills with respect to EBM meetings.

Objective structural clinical examination teachingdemonstrationThe objective structural clinical examination (OSCE)consisted of 15 min at each of 12 competency-basedstations, and was held once a month by the hospital’sOSCE committee as in our previous report.10 Allparticipants observed and practised serving as monthlyPGY1 OSCE raters during the first 2 h. Finally, partici-pants reviewed the PGY1 OSCE videos and discussedproblems concerning OSCE application with membersof the OSCE committee in the third hour of OSCEinstruction.

Mini-clinical evaluation exercise teaching demonstrationMini-clinical evaluation exercise (mini-CEX) demon-strations were provided four times every month bytrained lecturers as in our previous report.11 In the firsthalf hour, there were brief lectures on core competency-based mini-CEX evaluation. Over the next half hour,trained lecturers and participants then practised anddiscussed evaluating PGY1 residents’ mini-CEX bywatching videos.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


Second partPostcourse training workshop on ‘how to teach ACGMEcompetencies’This training workshop was held once every 2 months.Lecturers gave 45 min of instruction on teaching skills ineach of six competencies and discussed teaching prob-lems with all participants during the last 15 min of eachsession. After seven lecture/discussion sessions, partici-pants were invited to share their teaching and compe-tency application experiences during the last hour. Theorganisers also tried to establish a consensus concerningan adequate methodology for evaluation of the compe-tencies and teaching of PGY1 residents. Finally, allparticipants were asked to fill out questionnaires inorder to evaluate their perception of the training courseas a whole.

Postcourse training videotapes on ‘how to teach ACGMEcompetencies’The Taipei VGH educational committee has producedvideos of actual and simulated patients for use in‘competency instructional skills’ tutorials. The video-based tutorials were provided by General MedicineTraining Demonstration Center. Patients consented tothe filming, which was carried out by a professionalaudio visual team in actual clinical settings, includingoutpatient clinics and hospital wards. Three to four shortvideo clips were produced for each patient and demon-strated the application of competencies. The averagelength of these clips was 2 min. The first clip usuallyconsisted of competency-based medical consultationsdemonstrated by a trained physician. Subsequent clips

usually consisted of the patient’s physical examinationand follow-up medical consultations focusing onthe discussion of investigation results and treatmentoptions. When videos were used in the ‘competenciesteaching skills’ tutorials, no written information wasgiven to the participants. In other words, the onlymaterials presented to the clinical instructors were thevideo clips.

QuestionnairesEvaluation approaches included objectives, expertise,management and participant-oriented aspects of thetraining programme.9 In our study, an anonymous38-item questionnaire Kirkpatrick theory-based partici-pant-oriented questionnaire was designed to evaluatethe clinical instructors’ perception of our trainingprogramme (tables 1, 2). Kirkpatrick has described fourlevels of training-programme outcomes that may beassessed.12 This first level is a measure of the partici-pants’ initial reaction to the programme. The secondlevel is to assess the amount of knowledge and skill thatparticipants learnt, while the third level evaluates theamount of knowledge and skills learnt that participantsactually use in everyday work. The fourth level is anevaluation of the impact of the programme on theinstitution and society. It has been suggested thateducational institutions should develop an institution-specific evaluation model to meet their particular needsincluding educational processes and outcomes. There-fore, although we chose Kirkpatrick’s four level modelsas a guide for the evaluation, we adapted it to suit ourneeds. We interpreted the levels to be:

Table 1 Items from questionnaires for participants (Part 1)





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


Level 1dreaction, an evaluation of participants’ initialattitude and familiarity with the ACGME competencies,and ability to train ACGME competencies, which is thetarget of our programme. These questions are includedin table 1.Level 2dlearning, an evaluation of participants’

acquisition of each domain of ACGME competenciesby different teaching and training activities. Thesequestions were included in point 5 of table 2.Level 3dtransfer, an evaluation of the degree of

participants’ application of skills learnt from differentteaching and training activities. These questions areincluded as point 6 of table 1. In the questionnaires,participants were requested to rate the frequency oftheir application of competencies using a five-point scale(1: always (100% of their teaching time); 2: frequently(75e100% of their teaching time); 3: often (50e75% oftheir teaching time); 4: occasionally (25e50% of theirteaching time); 5: rarely (0e25% of their teachingtime)).Preliminary studies with senior physicians recruited

from the Department of Internal Medicine permitted usto identify and eliminate unreliable and ambiguousitems within our questionnaires in assessing levels 1e3 ofKirkpatrick theory. After a first validation of the ques-tionnaires, six points and 13 items were chosen based onexperts’ comments and validated again using a group ofexperts who confirmed the quality of the selected six

points and 13 items for evaluation purposes. The finalquestionnaire was adjusted to reflect feedback from thepilot session. To estimate the reliability of our ques-tionnaires, the Cronbach a coefficient was calculatedand our questionnaires found to have a reliability of0.81. Additionally, the re-evaluation reliability of ourquestionnaires was around 0.85.Before respondents answered each questionnaire, we

provided written definitions of the six competencies toparticipants. In the first part of questionnaire, a five-itemLikert scale was used to rate the degree of respondents’agreement with the teaching skills provided by differentactivities in the competency training programme forclinical instructors.13 In the second part, respondentswere asked to indicate which of the corresponding sixareas were learnt in each of the structured activities ofthe training courses. Recognising that the language ofthe competencies is rather general, we encouragedparticipants to use their judgement when decidingwhether a particular teaching activity provided trainingin one or more area (tables 1, 2). The questionnaireswere filled out after participants completed the basic40 h training course, after the training workshop andafter the videotape session. To assess the degree ofapplication of skills learnt from different activities,participants were asked to fill out a follow-up question-naire 3 months after completion the training course. Allthe activities in our programme were divided into three

Table 2 Items of questionnaires (Part 2); always: 100%; frequently: 75e100%; often: 50e75%; occasionally: 25e50%; rarely:0e25% times

5. Acquired competencies (can chose >1 domain) from each activity MK ICS SBP PBLI P PC

-Outpatient department (OPD) teaching training

- Itinerant bedside teaching training

- Physical examination (PE) teaching training

-Circuit bedside teaching training

- Evidence based medicine (EBM) teaching training

- Case-bared discussion (CbD) evaluation training

-OSCE evaluation training

-Mini-CEX evaluation training

-Training workshop of “how to teach ACGME competencies?”

-Training video clips of “how to teach ACGME competencies?

6. Degree of application of skills learnt from training activities Rarely (1) Occasionally (2) Often (3) Frequently (4) Always (5)

-Outpatient department (OPD) teaching training

- Itinerant bedside teaching training

- Physical examination (PE) teaching training

-Circuit bedside teaching training

- Evidence based medicine (EBM) teaching training

- Case-bared discussion (CbD) evaluation training

-OSCE evaluation training

-Mini-CEX evaluation training

- Training workshop on “how to teach ACGME competencies?”

- Training videotapes on “how to teach ACGME competencies?”





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


parts, namely teaching activities, evaluation activities andpostcourse training workshop/video session (table 2).

AnalysisThe average application of and attitudes towardscompetency instructional skills were analysed usingpaired t tests. Differences in the application of activity-orientated instructional skills, attitudes, familiarity andteaching ability between different groups were analysedby ANOVA. The degree of changes in the participants’attitude and familiarity to teach and train ACGMEcompetence after 40 h basic training course, postcoursetraining workshop and videotapes were analysed usingpaired t tests and the p values of trends. Furthermore,the effect of previous training (years as an attendingphysician and whether the participant had taken theTAME course) and teaching experiences (being a PGY1clinical instructors or mentor) on the average degree ofapplication of competencies in teaching was analysedusing the c2 test.

Ethics statementThis study was approved by the Ethics Committee ofTaipei Veteran General Hospital and complied with theprinciples of the Declaration of Helsinki Guidelines. Inagreement with these standards, written informedconsent was obtained from each participant.

RESULTSCharacteristics of participantsAmong the original 134 clinical instructors, 17 clinicalinstructors were not included in the study because theydid not complete all aspects of the training, andadditional seven clinical instructors did not complete thequestionnaire; this yielded a final total of 110 study-subjects. Among the 17 clinical instructors who did notcomplete the course, there were eight males (two fromGynaecology; three from Neurology; three from Emer-gency Medicine) and nine females (two from Surgery;three from Rehabilitation; two from Family Medicine;two from Psychiatrics). Additionally, only one male andone female clinical teacher among the above 17 clinicalinstructors had previous experience as a PGY1 mentorand clinical instructor (2/17, 12%). On the otherhand, two females (one from Surgery; one from Paedi-atrics) and five males (two from Chest Medicine; onefrom Rehabilitation; two from Family Medicine) did notcomplete the questionnaire, and only one femaleamong them (1/7, 14%) had experience as a PGY1mentor. Moreover, most of the above clinical instructors(19/24, 79%) came from other cooperating hospitalsrather than Taipei VGH. In a retrospective interview byemail, most stated that their main reason for failing tocomplete the course was the time limit of 3 months.Thus it is probable that the complete rate of theprogramme would be increased if the programme didnot insist that the participants complete all course within3 months.

It can be seen from table 3 that 90 male and 20 femaleclinical instructors completed the programme. Partici-pants came from different specialties including InternalMedicine (40/110, 37%), Surgery (20/110, 19%),Gynaecology (8/110, 8%), Paediatrics (10/110, 9%),Emergency Medicine (10/110, 9%) and others(Neurology, Psychiatrics, Chest Medicine, Rehabilitationand Family Medicine; 12/110, 18%). Table 3 shows that42% and 66% of the clinical instructors in our study hadexperience of being a PGY1 mentor or a clinicalinstructor, respectively. The average degree of applica-tion of competency instructional skills was markedlyhigher for participants from internal medicine than forparticipants from other specialties. Interestingly, thedegree of application of skills learnt from trainingshowed a significant decrease with the increase in theindividual’s years as an attending physician (table 3,p value for trend: 0.0028). In other words, the applica-tion of competency teaching skills was negatively corre-lated with how many years the person had been anattending physician.

Table 3 Basal characteristics of participants (clinicalinstructors) (n¼110)

Case no:cases/total (%)

Degree ofapplication

GenderMale 90 (82) 3.560.8Female 20 (18) 4.160.3

SpecialtyInternal Medicine 40 (37) 4.260.9*Surgery 20 (19) 2.360.5Gynaecology 8 (8) 1.960.2Paediatrics 10 (9) 3.160.6Emergency Medicine 10 (9) 2.260.9Other (Neurology,Psychiatrics, ChestMedicine, Rehabilitation,Family Medicine)

12 (18) 1.961.3

No of years as an attending physician<8 years 34 (31) 4.160.7y9e10 years 20 (18) 3.260.9y17e24 years 14 (12) 3.161.4y>24 years 42 (39) 2.961.3y

Postgraduate year-1 resident’s mentor experienceYes 48 (44) 4.361.1No 62 (56) 3.060.7

Postgraduate year-1 resident’s clinical instructorexperienceYes 66 (60) 3.961.5*No 44 (40) 2.760.4

Taken Taiwan Associate of Medical education coursebefore?Yes 81 (74) 3.461.3No 29 (26) 4.160.3

Degree of application was rated by Likert scale: (5: always; 4:frequently; 3: often; 2: occasionally; 1: rarely).*p Value <0.05 versus corresponding groups.yp Value for trend <0.01.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


In addition, the degree of application of competencyinstructional skills was significantly higher in participantswith previous experience of being PGY1 clinical instruc-tors than those without such previous experience.However, there were no differences in the degree ofapplication of skills learnt from training attributable togender, having experience as a PGY1 mentor or havingTAME course experience (table 3).

Average degree of application of skills learnt from activitiesof the faculty development programme in participants’teachingIn table 4, a comparison between teaching activitiesshowed that clinical instructors reported that the skillsalways (100% of teaching times) used were mainly learntfrom itinerant bedside teaching demonstrations, whilethe frequently (75e100% of teaching times) used skillswere learnt from circuit bedside teaching demonstra-tions. Rarely (<25% of teaching times) used skills werelearnt from PE teaching demonstrations. In the evalua-tion activities, clinical instructors reported that thealways (100% of teaching times) and frequently(75e100% of teaching times) used skills were learntfrom CbD evaluation demonstrations, while rarely(<25% of teaching times) used skills were learnt fromOSCE evaluation demonstration. After addition of thepercentages of always, frequently and often appliedinstructional skills, it was found that the most (>80%

teaching times) commonly used skills were learntfrom itinerant teaching, circuit bedside teaching andmini-CEX evaluation demonstration.

Average percentage of acquisition of skills of ACGMEcompetencies from activities of the faculty developmentprogrammeIn teaching activities, participants reported that skillsinvolving medical knowledge and system-based practicedomains were mainly learnt from OPD teachingdemonstrations, while skills involving interpersonal andcommunication skills domain were learnt from circuitbedside teaching demonstrations. Furthermore, skillsinvolving practice-based learning and improvementdomain were learnt from EBM teaching demonstrations,and skills involving professionalism domain were learntfrom itinerant bedside teaching demonstrations. Finally,skills involving patient care domain were learnt fromPE teaching demonstrations (table 5).In terms of evaluation activities, participants reported

that skills involving interpersonal and communicationskills and patient care domains were mainly learnt frommini-CEX demonstrations, while skills involving system-based practice and practice-based learning andimprovement domains were learnt from CbD evaluationdemonstrations. In addition, the skills involving profes-sionalism domain were learnt from OSCE evaluationdemonstrations.

Table 4 Degree of avenge application of skills learnt from different teaching and training activities in their teaching (n¼110)

Demonstrationactivities in trainingprogram

Percentagealways (1)

Percentagefrequently (2)

Percentageoften (3)

Percentageoccasionally (4)

Percentagerarely (5)

Percentage1+2+3

Outpatient departmentteaching

1162 3764 2565 1969 860.7 7361

Itinerant bedside teaching 2763* 2563 3068 1568 360.9 8262Physical examinationteaching

1364 2865 1967 1965 2160.8* 6065

Circuit bedside teaching 2361 4863* 2563 463 0 9664**Evidence-based medicineteaching

2168 3869 1568 1963 760.9 7462

Case-based discussionevaluation

1263* 3967* 361 2762* 1960.1 5463

Objective structural clinicalexamination evaluation

960.8 2765 2765 1263 2560.3* 6366

Mini-clinical evaluationexercise evaluation

863 2963 3763* 2364 360.4 7468*

Training workshop of‘how to teach ACGMEcompetencies’

1862 3662 3162 1362 260.5 8569

Training videotapes of‘how to teach ACGMEcompetencies’

1263 3764 3564 1564 160.7 8463

Frequency of application (always: 100%; frequently: 75e100%; often: 50e75%; occasionally: 25e50%; rarely: 0e25% of teaching times ofparticipants); The results were averaged data from questionnaires filled by participants at the end and follow-up 3-month after the training course.*p<0.05 and **p<0.01 versus others (comparison the percentage of application of skills learnt from different teaching and training activities intheir teaching).ACGME, Accreditation Council for Graduate Medical Education.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


Effects of postcourse training workshop on the participant’sattitudes to ACGME competenciesParticipants reported that postcourse training workshopeffectively improved their agreement with the aspects of‘should be learnt,’ ‘should be taught’ and ‘should beevaluated’ (figure 1). However, attitudes concerning‘meeting social expectations’ of competencies were notchanged by postcourse training workshop.

Effects of postcourse training workshop and videotapesession on the participant’s familiarity to ACGMEcompetenciesParticipants reported that both training workshop andvideotapes significantly improved their familiarity withteaching and definitions of competencies among partic-ipants (figure 2A). In addition, the postcourse trainingvideotapes also significantly improved participants’familiarity with assessment of competencies. Further-more, participants felt that their familiarity withteaching and assessment of competencies improvedprogressively as the 40 h basic training course, postcoursetraining workshop and videotapes were completed.Participants reported that most of them were familiar

with instructional skills in the domains of patient care,interpersonal and communication skills, and medical

knowledge (Likert scale of >3.5) after the 40 h basictraining course (figure 2B). However, participants feltthat they were still not very familiar with the instructionin system-based practice, practice-based learning andimprovement, and professionalism domains. Notably,participants felt that they were familiar with system-basedpractice, practice-based learning and improvement, andprofessionalism competencies progressively after

Table 5 Average percentage of acquisition of teaching and assessment skills of Accreditation Council for Graduate MedicalEducation (ACGME) competencies from training activities reported by participants (n¼110)

Demonstrationactivities intrainingprogramme

Percentagemedicalknowledge(1)

Percentageinterpersonalandcommunicationskills (2)

Percentagesystem-basedpractice(3)

Percentagepractice-basedlearning andimprovement(4)

Percentageprofessionalism(5)

Percen-tagepatientcare(6)

Percen-tage1+2+3+4+5+6

Outpatient departmentteaching

6864* 1262.7 1560.4* 1761 1563 3562 162621**

Itinerant bedsideteaching

3162 360.3 160.03 1462 42611* 2566 116618

Physical examinationteaching

2763 5.160.2 860.05 360.9 1063 3164* 84621

Circuit bedsideteaching

1267 5360.9* 1263 862 662 2262 11369

Evidence-basedmedicine teaching

960.6 860.7 960.8 2264* 560.3 760.8 126615*

Case-based discussionevaluation

1161 561 2962* 49611** 960.7 2363.7 6068

Objective structuralclinical examinationevaluation

2163 460.8 360.3 863 2762* 1363 7668

Mini-clinical evaluationexercise evaluation

1962 2865* 260.6 360.6 860.9 5169** 111617

Training workshop of‘how to teach ACGMEcompetencies’

1265 1963 2063 3465 3160.6 1060.8 126623*

Training videotapes of‘how to teach ACGMEcompetencies’

360.8 961.7 1460.9 2265 3165 1867 9769

The results were averaged data of questionnaires completed by participants at the end of training course and 3 months after finishing thetraining course; participants were asked to choose one or two competencies for each activity.*p<0.05 and **p<0.01 versus other teaching or evaluation activities.

Figure 1 Participants, averaged attitude to the AccreditationCouncil for Graduate Medical Education six competencies(n¼110), the degree of agreement with the asked questionswas rated by Likert scale (1: strongly disagree 2: disagree; 3:neutral; 4: agree; 5: strongly agree). *p<0.05 versus finishing40 h basal training course.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


finishing the postcourse training workshop andvideotape session.14 15

Effects of postcourse training workshop and videotapes onthe participant’s training abilities with respect to ACGMEcompetenciesAfter finishing the 40 h basic training course, postcoursetraining workshop and videotapes, participants feltthat their ability to teach, assess and improve the PGY1residents’ ACGME competencies were improvedprogressively (figure 3A).Furthermore, participants felt that postcourse training

workshop and videotapes significantly improved theirability to train PGY1 residents’ system-based practice,problem-based learning and improvement, and profes-sionalism domains (figure 3B). Nevertheless, partici-pants reported that postcourse training workshop andvideotapes did not further change their ability to teachPGY1 residents’ medical knowledge and patient carecompetencies.

Percentage change from baseline in the degree ofparticipant’s application (>50% of teaching times) of skillslearnt from activities in the follow-up questionnaireIn terms of teaching activities, participants reported inthe follow-up questionnaire that the highest degree ofimprovement in the application frequency of skills was

learnt from itinerant bedside teaching (figure 4).Among the evaluation activities, participants reportedthat the highest degree of improvement in the applica-tion frequency of skills was learnt from a CbD evaluationdemonstration. Finally, participants reported that thedegrees of improvement in the application frequency ofskills learnt from postcourse training workshop andvideotapes were similar.

DISCUSSIONAfter completion of the 40 h basic training course,participants did not agree strongly that they should learnACGME competencies. This is probably due to thefact that clinical instructors considered that the ACGMEcompetencies taught in our programme did not meetsocial expectations very well.4 15 In addition, participantsfelt that they did not acquire better instructional skills inthe domains of system-based practice, problem-basedlearning and improvement, and professionalismdomains to a significant degree after 40 h basal trainingcourse. Nonetheless, participants felt that the postcoursetraining workshop and videotapes (figures 2B, 3B)significantly improved their familiarity and teachingability in the PGY1 residents’ system-based practice,problem-based learning, and improvement, and profes-sionalism domains.16 17 Accordingly, our programmedirectors should consider extending the hours of

Figure 2 Effects of postcoursetraining workshop and videotapeson the familiarity to (A) Overall and(B). Each domain of AccreditationCouncil for Graduate MedicalEducation six competencies.*p<0.05 versus finishing 40 htraining course; yp value for trend<0.05 (progressively increasebetween different groups). ICS,interpersonal communicationskills; MK, medical knowledge;P, professionalism; PBLI,problem-based learning andimprovement; PC, patient care;SBP, system-based practice.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


activity including itinerant bedside teaching and CbDevaluation demonstration that mainly trained theparticipants’ instructional skills of the above threedomians.16

Clinical instructors felt that both postcourse trainingworkshop and videotapes markedly improved theirfamiliarity and instructional skills of the above threedomains. Previous studies had suggested that training

Figure 3 Effects of postcoursetraining workshop and videotapeson the establishment of the abilityto train (A) Overall and (B). Eachdomain of Accreditation Councilfor Graduate Medical Educationsix competencies. *p<0.05 versusfinishing 40 h training course; ypvalue for trend <0.05(progressively increase betweendifferent groups). ICS,interpersonal communicationskills; MK, medical knowledge; P,professionalism; PBLI, problem-based learning and improvement;PC, patient care; SBP, system-based practice.

Figure 4 Percentage changesfrom baseline of the degree ofapplication of skills learnt fromdifferent (A). Teaching activities;(B). Training activities; (C).Postcourse training workshop/videotapes in follow-upquestionnaires (3-month) reportedby participants. p<0.05 and**p<0.01 versus other teaching/evaluation activities.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


videotapes consistently enhance trainees’ observationalpowers, improves their ability to integrate differentinformation and motivates them to learn.5 18e20 Ourprogramme director should probably increase the hoursof postcourse training workshop and videotapes sessionto improve training in the domains of system-basedpractice, problem-based learning and improvement, andprofessionalism in the future. Alternatively, theprogramme director can also eliminate the trainingsessions such as PE teaching and OSCE evaluationdemonstrations that were not preferred by participantsand spend more time on those training methodspreferred by participants.In addition, the participants also responded in the

follow-up (3 months) questionnaires that the most skillsthat they applied in teaching had been mainly learntfrom itinerant and circuit bedsides, and PE teachingdemonstrations, postcourse training workshop andvideotapes (table 4). Conversely, participants answeredthat they rarely used instructional skills learnt from CbDevaluation demonstrations. At the same time, our studysuggested that the CbD evaluation demonstration mainlyestablishes teaching and assessment skills in the domainsof system-based practice, problem-based learning andimprovement. The lack of application of CbD instruc-tional skills might have resulted from participants’ poorfamiliarity with CbD instructional skills after completionof the programme. Thus, the programme directorsshould also consider extending the hours of CbDevaluation demonstration in future.This study had several limitations. First, the question-

naire used to track and assess the participants’ percep-tion of the faculty development programme may sufferfrom information and recall bias. In other words, theevaluation of participants’ perception to training shouldtherefore be assessed immediately after each trainingsession by persons other than the programme directors.7

In fact, a self-reported questionnaire might not reflectbehaviours in authentic setting. Thus, instead of partic-ipants’ feelings about the value of the various trainingsessions, we should assess the improvement in teachingability of participants for PGY1 residents’ competencies.Second, we only assessed the participants’ perception

about levels 1e3 of the Kirkpatrick approach with regardto our programme. In other words, we did not evaluatethe participants’ basal attitude and familiarity with theACGME six core competencies at the beginning of thefaculty development programme. In order to validateour programme, a basal assessment of participants’perception is needed in the future.Third, according to the ‘original’ definition, Kirkpa-

trick’s third level evaluates ‘the amount of knowledgeand skills learnt that participants actually use in everydaywork.’ In our study, we modified the third level to be the‘self-reported degree of participants’ application of skillslearnt from different teaching and training activities.’Furthermore, in order to validate the results of ourfaculty development programme effectively, a clear defi-

nition of the desirable teacher’s desirable behavioursneeds to be given to participants in future works. Thebehaviours perhaps should include the following:creating an appropriate learning climate; being learner-centred; facilitating participants’ learning; encouragingself-awareness through reflection; tailoring teaching toparticipants’ needs and wants, etc. suggested by Hewson.8

Additionally, it is well established that when one goesthrough an evaluation process, the evaluation processbecomes more difficult and time-consuming as onemoves from Kirkpatrick level 1 to level 4, although higherlevel provides more information than lower level, that is,of increasingly significant value than lower level. Thus,the level 4 needs to be assessed in the future for ourprogramme.9 12 It should be noted that, as yet, nocontrolled educational trials on this subject have beenpublished, and this type of trial would be useful.18 Finally,this study involved only a short follow-up period, and asa result, the study’s findings may represent only short-term changes in attitude, familiarity and teaching skills.9

Despite these limitations, the present study providesinformation about the participants’ perception of thevarious training sessions of our ‘faculty developmentprogramme about competency for clinical instructors’designed and organised by the Taipei VGH educationalcommittee. To date, there are no well-established stan-dards that specifically address the competency teachingand assessment skills of clinical instructors.21

CONCLUSIONSThe participants in our ‘faculty developmentprogramme of ACGME competencies for clinicalinstructors’ reported that our programme effectivelyincreased their familiarity with various teaching andassessment skills of competencies for PGY1 residents.Additionally, clinical instructors also reported that theysubsequently applied these skills in their work.

Acknowledgements The authors express their gratitude to all members ofthe General Medicine Teaching Demonstration Center for their input for thisarticle. We also gratefully appreciate R Kirby’s help in correcting the Englishin our manuscript.

Funding This work was supported by grant no VGH100C-21 from the TaipeiVeterans General Hospital, Taipei, Taiwan.

Competing interests None.

Ethics approval Ethics approval was provided by the Ethics Committee ofTaipei Veteran General Hospital.

Contributors All authors actively participated in the analysis, writing andrevision of the paper. YYY, FYL and HCH were responsible for study design.YYY, FYL, HCH, JWC, WSL and CCH participated in the questionnaires. CCC,HMC and CCH participated in the creation and management of the database.YYY, FYL, HCH and CCH were responsible for the statistical analysis andwriting of the manuscript.

Provenance and peer review Not commissioned; externally peer reviewed.

Data sharing statement No additional data available.

REFERENCES1. Hsieh BS. Primary care training as basis for clinical education. J Med

Educ 2000;4:273e4.





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


2. Weed HG. Some thoughts about the teaching of general internalmedicine in Taiwan. J Med Educ 2006;10:162e4.

3. Harrison JP, Curran L. The hospitalist model: does it enhance healthcare quality? J Health Care Finance 2009;35:22e34.

4. Lowsy S. Teaching the instructors. BMJ 1993;306:127e30.5. Chan CY. Impacts of and methods of coping with the extension of the

post-graduate year general medicine training program. J Med Educ2009;13:226e31.

6. ACGME Outcome Project. Available at http://www.acgmec.org.7. Sefton A. Innovative health professionals in a current global context.

J Med Educ 2005;9:197e204.8. Hewson MG. Theory-based faculty development program for

clinician-educators. Acad Med 2000;75:409e501.9. Leung KK, Wang WD, Chen CY, et al. Evaluation of the faculty

development program. J Med Educ 1997;1:436e43.10. Yang YY, Lee FY, Hsu HC, et al. A core competence-based objective

structured clinical examination (OSCE) in evaluation of clinicalperformance of postgraduate year-1 (PGY1) residents. J Chin MedAssoc 2011;74:198e204.

11. Yang YY, Lee FY, Hsu HC, et al. Evaluation of the clinicalcompetences of dental interns using a mini-CEX. J Med Educ2010;14:251e61.

12. Salvatore VF. Evaluating training programs: the four levels, byDonald L. Kirkpatrick, Berrett-Koehler Publishers. Am J Eval1998;19:259e61.

13. Ullian JA, Bland CJ, Sirnpson DE. An alternate approach todefining the role of the clinical instructors. Acad Med1994;69:832e8.

14. Miller GE. The assessment of clinical skills/competence/performance.Acad Med 1990;65(9 Suppl):S63e7.

15. Wass V, Van der Vleuten C, Shatzer J, et al. Assessment of clinicalcompetence. Lancet 2001;357:945e9.

16. Johnson CE, Hurtabise LC, Castrop J, et al. Learningmanagement systems: technology to measure the medicalknowledge competency of the ACGME. Med Educ 2004;38:599e608.

17. Larkin GC, Binder L, Houry D, et al. Defining and evaluatingprofessionalism. A core competency for graduate emergencymedicine education. Acad Emerg Med 2002;9:1249e56.

18. Wilkerson L, Irby DM. Strategies for improving teaching practices:a comprehensive approach to faculty development. Acad Med1998;73:387e96.

19. Chan LK, Patil NG, Chen JY, et al. Advantages of video trigger inproblem-based learning. Med Teach 2010;32:760e5.

20. Balslev T, de Grave WS, Muijtjens AM, et al. Comparison of text andvideo cases in a postgraduate problem-based learning format. MedEduc 2005;39:1086e92.

21. Natarajan P, Ranji SR, Auerback AD, et al. Effect of hospitalistattending physicians in trainee educational experiences: a systematicreview. J Hosp Med 2009;4:490e8.

PAGE fraction trail=11.5





jopen.bmj.com

/B

MJ O


ovember 2011. D

ownloaded from


open access research clinical instructors …while teaching were learnt during circuit and itinerant...

Documents