The Mini Clinical The Mini Clinical

Evaluation ExerciseEvaluation Exercise

(mini(mini--CEX)CEX)

Dr Jeremy MortonDr Jeremy Morton

Medical Teaching Medical Teaching OrganisationOrganisation

University of Edinburgh Medical SchoolUniversity of Edinburgh Medical School

OverviewOverview

•• PerformancePerformance--based assessment (PBA) in based assessment (PBA) in

medical educationmedical education

•• Attitudes and barriers to PBAAttitudes and barriers to PBA

•• minimini--CEX assessment in undergraduate CEX assessment in undergraduate

medical educationmedical education

Reforms of postgraduate medical Reforms of postgraduate medical

education in the UKeducation in the UK

Undergraduate medical educationUndergraduate medical education

•• Shift from Shift from

knowledgeknowledge--based based

to competencyto competency--

based assessment.based assessment.

Assessment of clinical competenceAssessment of clinical competence

Does = Performance Based Testing eg; Mini-CEX, Multi-source Feedback

Shows How = Competency Based Testing eg; OSCE, SP testing

Knows How = Clinical Context Based Testing eg; PMPs, MEQ, EMQ

Knows = Factual Testing eg; MCQ, essay, oral

Miller’s Pyramid

PerformancePerformance--based assessmentbased assessment

Attitudes to Clinical Competence Attitudes to Clinical Competence

Assessments (CCA) Assessments (CCA) –– a survey a survey

CCA CCA –– FurtherFurther

Experiences & AttitudesExperiences & Attitudes

• 75% of clinical teachers indicated that global judgement without direct observation was principal method of rating student

• Reflected in students’ evaluation of assessment practice in Year 5

• 70% of students in favour of summative PBA strategy vs. 30% clinical teacher

• Majority of clinical teachers back a return to more formalised testing

Barriers to PBABarriers to PBA

Clinical Teachers -• Lack of time• Large class sizes• Inadequate resources• Lack of educational training• Conflicting priorities

Students Students --

•• Reticence on the part of their teachers to Reticence on the part of their teachers to

engage in the assessment of observed engage in the assessment of observed

practicepractice

•• Lack of commitment to standardising Lack of commitment to standardising

practicepractice

Is PBA a lost cause?Is PBA a lost cause?

““I think you have to be awareI think you have to be aware of of

the pressure on mainstream the pressure on mainstream

speciality consultants to provide speciality consultants to provide

teaching. I have seven students teaching. I have seven students

for a block of seven weeks four for a block of seven weeks four

times per year and I am their tutor. times per year and I am their tutor.

This is on top of a full clinical This is on top of a full clinical

workload. I struggle to find time to workload. I struggle to find time to

do what I do (and train our own do what I do (and train our own

juniors) so although what I say is juniors) so although what I say is

desirable I dondesirable I don’’t think I can t think I can

achieve it.achieve it.””

minimini--CEXCEX

•• Method for assessing clinical performance in the Method for assessing clinical performance in the workplace developed by ABIMworkplace developed by ABIM

•• Viva examinations abandoned by ABIM in 1972 Viva examinations abandoned by ABIM in 1972 and replaced with the Clinical Evaluation and replaced with the Clinical Evaluation Exercise (CEX).Exercise (CEX).

•• MiniMini--CEX superseded CEX in early to midCEX superseded CEX in early to mid--90s90s

minimini--CEXCEX

•• Postgraduate toolPostgraduate tool

•• Pocketsize cardsPocketsize cards

•• 7 competency 7 competency

domainsdomains

•• Nine point rating scale, Nine point rating scale,

4 defined as marginal4 defined as marginal

•• Snapshot of residentSnapshot of resident--

patient interactionpatient interaction

minimini--CEXCEX

•• Shown to be valid, reliable, feasible, acceptable and Shown to be valid, reliable, feasible, acceptable and

fair assessment in PG & UG practice in the USA.fair assessment in PG & UG practice in the USA.

•• Reproducibility coefficient of 0.8 following 12 Reproducibility coefficient of 0.8 following 12 --14 14

assessments in PG and 8 in UG setting.assessments in PG and 8 in UG setting.

•• Fewer assessments required to reliably identify Fewer assessments required to reliably identify

those who are clearly competent or incompetentthose who are clearly competent or incompetent..

minimini--CEXCEX

•• Strongly Strongly recommended in recommended in assessment of assessment of medical interns, medical interns, USAUSA

•• Adopted as a Adopted as a key element in key element in the FPthe FP’’s s assessment assessment toolkit for Eng.& toolkit for Eng.& WalesWales

minimini--CEX & UKCEX & UK•• FY doctors expected FY doctors expected

to undertake 4 minito undertake 4 mini--CEX assessments in CEX assessments in FY 1 & 6 in FY 2FY 1 & 6 in FY 2

•• Implemented on basis Implemented on basis of published evidence of published evidence relating to practice in relating to practice in USA & experience of USA & experience of pilot schemespilot schemes

•• Limited experience in Limited experience in undergraduate circlesundergraduate circles

minimini--CEX & undergraduate CEX & undergraduate

medical education in the UKmedical education in the UK

•• University of Southampton Medical University of Southampton Medical

School, ASME 2005School, ASME 2005

•• Evaluation of 2340 miniEvaluation of 2340 mini--CEX assessments CEX assessments

in a single yearin a single year

•• Format derived from ABIM Format derived from ABIM

The positivesThe positives

•• Liked by studentsLiked by students

•• Easy for examiners to organiseEasy for examiners to organise

•• Driver for change & improvementDriver for change & improvement

•• Less disruptive to patients and their Less disruptive to patients and their

familiesfamilies

•• Seen to be fairSeen to be fair

The negativesThe negatives

•• Organisational problems in shorter Organisational problems in shorter

attachmentsattachments

•• Less holistic than long casesLess holistic than long cases

•• Perception of lack of consistency in Perception of lack of consistency in

gradinggrading

•• Students with good communication skills Students with good communication skills

but poor overall clinical competence may but poor overall clinical competence may

be able to passbe able to pass

minimini--CEX pilot in EdinburghCEX pilot in Edinburgh

• 202 mini-CEX assessments on 94 students during acute medicine block

• Median no. of assessments per student - 2 (range 1-5)

• Median observation time – 5 minutes

(range 2-90)

Median feedback time – 10 minutes (range 1-60)

• Assessor satisfaction – 119 ratings Median value 8 on 10 point scale.

• Reduced tendency to grade towards middle

Student ExperienceStudent Experience

• 77% of students agreed or strongly agreed that mini-CEX is a fair means of assessment

• Median student satisfaction with

mini-CEX format – 8

(on same 10 point scale)

• Difficult to arrange the time with busy medical staff to carry out mini-CEX assessments

• Recognise the unreliability of a judgement made on one or two assessments

Making miniMaking mini--CEX work in CEX work in

undergraduate medical educationundergraduate medical education

•• Adapt miniAdapt mini--CEX CEX

methodology methodology

coco--operativelyoperatively

•• Encourage integration of Encourage integration of

assessment assessment

•• Set achievable targets Set achievable targets

and consider weighting and consider weighting

the burden of assessment the burden of assessment

to focus on borderline to focus on borderline

candidatescandidates

Making miniMaking mini--CEX work in CEX work in

undergraduate medical educationundergraduate medical education

•• Undertake staff Undertake staff development and quality development and quality control control

•• Factor miniFactor mini--CEX CEX performance into performance into summativesummative assessmentassessment

•• Involve senior training Involve senior training and nonand non--training grade training grade doctors in assessmentdoctors in assessment

•• Reward for excellence in Reward for excellence in teaching and assessmentteaching and assessment

SummarySummary

•• Students and consultant clinical teachers Students and consultant clinical teachers acknowledge the merits of PBA and related acknowledge the merits of PBA and related feedback but doubts persistfeedback but doubts persist

•• minimini--CEX is becoming established as a routine CEX is becoming established as a routine in postgraduate practicein postgraduate practice

•• Experience suggests that miniExperience suggests that mini--CEX testing is CEX testing is acceptable, fair and manageableacceptable, fair and manageable

•• MiniMini--CEX may offer sustainable means of CEX may offer sustainable means of achieving reliable PBA that is practical for achieving reliable PBA that is practical for clinicians and offers quality feedback to studentsclinicians and offers quality feedback to students