COLLEGE EXAMINATION

The MRCGP examination and its methods.IV: MEQ paperJ. H. WALKER, frccp

I. M. STANLEY, frccp

T. L. VENABLES, frccp

E. C. GAMBRILL, frccp

G. K. H. HODGKIN, frccp

¦THE modified essay question (MEQ) is an original-¦* College development of the patient-managementquestionnaire format widely used in undergraduate andpostgraduate medical education, both in the UnitedKingdom and overseas.

The MEQ paper is presented in the form of a booklet.Each page presents a separate problem which must beresponded to before turning over. Further informationis divulged and new problems posed as the case de-velops. Thus the dimension of time can be built in andthis important factor in general practice emphasized.

It is possible to simulate the clinical sequence ofdecision-making in actual consultations. Unlike theMCQ arid the Australian PMP (patient managementproblems), cues are not used. This open-ended ap¬proach allows the exploration of two areas affecting thecandidate's clinical decision-making: recall of clinicaland behavioural patterns; sensitivity to the constraintsof time, ar\d to the modifying influences of emotional,social and cultural factors in patient and doctor.When constructing an MEQ a compromise has to be

reached at each sequential, decision-making point. Ifthe question is too open-ended, the candidate's sensitiv¬ity and selectivity become buried in a 'blunderbuss'offering that often bears little relation to practice.

If the question is too circumscribed, it does not allowsufficient exploration of the candidate's range of clini¬cal thinking. For example the question: 'A 15-year-oldgirl reports to you with 3 months dysmenorrhoea',

J. H. Walker, Professor of Family and Community Medicine, New¬castle upon Tyne; I. M. Stanley, Lecturer in General Practice,University of Leeds; T. L. Venables, General Practitioner, Notting¬ham; E. C. Gambrill, General Practitioner, Crawley, Sussex; G. K.Hodgkin, General Practitioner, Great Ayton, Cleveland.

© Journal of the Royal College of General Practitioners, 1983, 33,804-810.

might be followed by any of the following requests tbthe candidate:

1. 'Outline briefly how you would handle the situ¬ation.'2. 'What is the single most likely hidden motivationunderlying the patient's complaint?'3. 'Enumerate in order of probability those four diag¬nostic possibilities that would guide your further ques¬tions.'

The range of answers to the third is likely to give theexaminers the best insight into the candidate's clinicalthinking, sensitivity and selectivity in dealing with thiscommon primary care problem. Nine or 10 such ques¬tions can be used to assess the candidate's ability tocollect, select and use data to make appropriate deci¬sions in the sequential situations of the consultation.The MEQ has been used to test skills of: information

gathering via history-taking, examination, and the use

of investigations and procedures; hypothesis formationand testing; evaluation of collected data; definition ofthe problems posed in physical, psychological and socialterms; decision-making; recording and communicationwith colleagues, patients and relatives; preparation ofplans for management and therapy; selection of appro¬priate treatment; understanding the problems of com¬

pliance; organization and mobilization of practice andthe resources of the community; provision of continuingcare and follow-up; and anticipation of future prob¬lems.

Constructing the question paper and the markingschedule is a time-consuming task which involves theMEQ group coordinator, a nucleus group of examiners,cells of four MEQ group members and either the entirePanel of Examiners or the increasingly numerous MEQgroup itself. A topic area is chosen to ensure appropri-

804 Journal of the Royal College of General Practitioners, December 1983

J. H. Walker et al.

ate cover in sequential examinations. Members of thegroup submit outline papers; the nucleus group edit thechosen paper and circulate it to the entire Panel whocomplete it as though they were candidates. The exam¬

iners' scripts are returned and the individual questionsthen become the responsibility of cells of MEQ groupmembers who construct and weight the marking sched¬ule. Final editing is undertaken at a meeting of represen¬tatives of each of the cells. The marking schedule toeach paper is the product of the views of the Panel ofExaminers and the content validity is thereforehigh.Marking is the responsibility of the four members of

each cell responsible for the construction of that ques

tion and each script is marked independently by twoexaminers. A computer printout similar to that for theTEQ consistently demonstrates the high reliability ofthe marking method. The two main disadvantages ofthe MEQ format are that the candidates may gain someadvantage by reading through the paper before theybegin and that they may answer the question in terms ofwhat they believe the examiner wishes to hear ratherthan what they would actually do. The first disadvan-tage is diminished by careful construction of the paperand by instruction and supervision of candidates. Thesecond cannot be controlled. Figure 1 shows a modifiedessay question accompanied by the answers of a candi¬date who obtained high marks.

Figure 1. Examples of a modified essay question and answers (continued on pp. 806, 807 and 808)

Journal of the Royal College of General Practitioners, December 1983 805

J. H. Walker et al.

f^ffe^

Figure 1. continued

806 Journal of the Royal College of General Practitioners, December 1983

J. H. Walker et al.

as essential in managing

pgmimm^._

¦^^SM^^lfM

gh, rib recession, added

flSll^^Jiilfll \km$!§0^§^nt to.kn^

tetfcsit its fi&s dtifeovered frorri soil^ofewbeMllllll^Indicate the psychological defence mechanisi

ililt^^ Bllllt

Figure 1. continued

Journal of the Royal College of General Practitioners, December 1983 807

J. H. Walker et al.

Expectinghelp fromotstside baby's at fat~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.........e..herselforbetmarital~~~~~~~~~~~~~~~~~~~...................li.r~#1a

Tryisigt~~~~~~~~~~~~~~~~~~~~~~~~~~keep ~~~~~~~~~~~~~~~~~~wsban~~~~~~~~~~~~~~~i~~~~~~~~~i si¶at~~~~~~~~~~~~~~~~~~~~y~.......~~~etti~~~~~~~~~gpregnant.~~~~~~~~~~~......'We~~~~~~~eall ~~~~~~~~~~ht~~~~W~~~~W~~. ....~ 4

Wrong':JenIW*$ Jei<iaIof .vsb~ '.........:..

it W e i~~~~~~~~~~~~~~~Wwc.l:t X:.............<endeq.isets+/dos tte%niwiout m.

L ativ>o4l;Ar=h.W ...i.,y.L..u... war

Figure 1. continued

V: Oral examinationTHE main advantages of oral examinations is their

flexibility and the fact that examiners can explorethe range and depth of the candidate's thinking and, inthe medical context, his decision-making in clinical andpractice management situations. The main disadvan-tages of oral examinations are their logistics. Oralsshould therefore be restricted to assessment in thoseareas which cannot effectively be examined by othermore economical means.

In the MRCGP examination, candidates who achievea minimum level in the written papers are called for two30-minute oral examinations, each of which is conduct-ed jointly by two examiners. The basis of the first oral isthe candidate's log diary in which he provides details ofhis practice, his workload and 50 patients. During thisoral some of the time is spent in exploring the candi-date's knowledge and attitudes in the areas of colleagueand staff relationships, practice management, readingand research; and the remainder in his decision-makingin the management of his own patients. A list of topicscovered in the first oral is passed by the candidate to theexaminers conducting the second. During this examin-ation any subject relevant to general practice may beexplored and, as in the first of the orals, a considerableproportion of the time will be spent in assessing thecandidate's reaction to clinical problems in the settingof general practice.Throughout each oral examination the examiners

independently record marks for each topic covered andat the end of the examination make independent overallassessments of the candidate. They discuss any differ-ences and produce a consensus mark. When the marksfor each oral are available, the candidate's final mark inthe entire examination is calculated. Candidates whosemarks are near the borderline are discussed by the fourexaminers, who will have then been provided with thecandidate's marks in the written papers. At this pointthey may modify the oral marks if necessary, justifyingtheir decision in the examiners' meeting which takesplace at the end of the day.

Validity and reliability

Set in the context of the experience of the examiners, allof whom are in active clinical general practice, the oralis seen as a particularly valid technique. Because of thelarge number of examiners, a number of methods havehad to be developed and adopted in an attempt toensure reliability. For every three, three-hour examiningsessions, each examiner must spend one session observ-ing orals conducted by his colleagues, during which hemust award marks as though he were an examiner. Atapproximately every third examination each examiner isvideotaped and later in the day he views the tape,appraises his examination technique, re-marks the can-didate and explores any discrepancy with his colleagues.

All examiners are expected to attend the annualexaminers' workshops at which calibration exercises ofthis type are carried out. At the beginning of their firstday at-each examination, all examiners are required toview and mark a standard 15-minute videotape of anoral examination thus calibrating their own markingstandards before they begin. All these exercises aim toencourage self-criticism rather than peer-criticism. Theyprovide the opportunity for examiners to increase thereliability of their technique and to monitor the validityof the topics they choose to cover.

Satisfactory construct validity of the oral examin-ation is dependent upon its assessing those candidateattributes which are not assessed by other techniquesand avoiding those which are. One particular problem isthat of assessing factual knowledge. While this is mostreliably assessed by the MCQ paper, it is not possible toexamine clinical decision-making without reference tothe candidate's knowledge base. For example, the can-didate who describes with excellent sensitivity hismethod of handling the parents of a baby with Down'ssyndrome should have been able to recognize the con-dition in the 'first place. The examiners would find itdifficult to award high marks to a candidate who couldnot make the initial diagnosis. While the assessment of

808 Journal of the Royal College of General Practitioners, December 1983