References

ASSOCIATION OF PROFESSORS OF MEDICINE (2004) The future of medical

student education in internal medicine, American Journal of Medicine,

116, pp. 576–580.

AUSTRALIAN MEDICAL COUNCIL (2004) Goals and Objectives of Basic

Medical Education Available at: http://www.amc.org.au/accredgoals.asp

BRANCH, W.T., KERN, D., HAIDET, P., WEISSMANN, P., GRACEY, C.,

MITCHELL, G. & INUI, T. (2001) Teaching the human dimensions

of care in clinical settings, Journal of the American Medical Association,

286, pp. 1067–1074.

BRITISH MEDICAL ASSOCIATION (2003) Medicine in the 21st Century

(London, BMA).

DEPARTMENT OF HEALTH (2004) Choosing Health: Making Healthier

Choices Easier. Available at: http://www.dh.gov.uk/Publications

AndStatistics/Publications/PublicationsPolicyandGuidance/fs/en?

CONTENT_ID¼4094559&chk¼H29li6.

CHICKERING, A.W., & Gamson, Z.F. (1991) Applying the seven principles

for good practice in undergraduate education, New Directions for

Teaching and Learning, 47 (San Francisco, Jossey-Bass Inc.).

DORAN, G.A. (1983) Scientism vs humanism in medical education,

Social Science Medicine, 17, pp. 1831–1835.

ENGEL, G.L. (1978) The biopsychosocial model and the education of

health professionals, Academic Science, 310, pp. 169–187.

GENERAL MEDICAL COUNCIL (1993) Tomorrow’s Doctors. Recommendations

on Undergraduate Medical Education (London, GMC).

GENERAL MEDICAL COUNCIL (2002) Tomorrow’s Doctors. Recommendations

on Undergraduate Medical Education (revised edition) (London, GMC).

GULL, S.E. (2005) Embedding the humanities into medical education,

Medical Education, 39, pp. 235–236.

INSTITUTE FOR THE FUTURE (2000) Health and Health Care,

2010: The Forecast, The Future, The Challenge (San Francisco, Jossey-

Bass Inc.).

JAMSHIDI, H.R. & COOK, D.A. (2003) Some thoughts on

medical education in the twenty-first century, Medical Teacher, 25,

pp. 229–238.

LUEDDEKE, G.R. (1999). Toward a constructivist framework for guiding

change and innovation in higher education, The Journal of Higher

Education, 70, pp. 235–260.

MAHEUX, B., DELORME, P., BELAND, F. & BEAUDRY, J. (1990) Humanism

in medical education: a study of educational needs perceived by trainees

of three Canadian schools, Academic Medicine, 65, pp. 41–45.

MCMANUS, I.C., KEELING, A. & PAICE, E. (2004) Stress, burnout and

doctor’s attitudes to work are determined by personality and learning

styles: a twelve year longitudinal study of UK medical graduates.

BMC Medical Education. Available at: http://www.biomedcentral.com/

1741-7015/2/29.

NASH, S. & SHERWOOD, M. (2002) The University of Southampton:

An Illustrated History (London, James and James Publishers Ltd.).

TEACHING QUALITY INFORMATION (2005) National student satisfaction

survey: University of Southampton medicine and dentistry. Available at:

http://www.tqi.ac.uk.

O’NEILL, P.A., JONES, A., WILLIS, S.C. & MCARDLE, P.J. (2003)

Does a new undergraduate curriculum based on Tomorrow’s Doctors

prepare house officers better for their first post? A qualitative study

of the views of pre-registration house officers using critical incidents,

Medical Education, 37, pp. 1100–1108.

OZUAH, P.O. (2002) Undergraduate medical education: thoughts on

future challenges, BMC Medicine, 2(8). Retrieved on 10.02.05 from

http://www.biomedcentral.com.

QUALITY ASSURANCE AGENCY (1999) Medicine: Subject Review Report

(Bristol, QAA).

QUALITY ASSURANCE AGENCY (2002) Medicine: Subject Benchmark (Bristol,

QAA).

QUALITY ASSURANCE AGENCY (2003) Institutional Audit (Bristol, QAA).

ROGERS, L. (2005) A disease called despair on the NHS front line,

The Sunday Times, 27 March 2005.

SCHOOL OF MEDICINE (2004) BM5 Medical Curriculum Booklet 2004–2005

(Southampton, School of Medicine).

SMITH, R. (2002) The Seven Levels of Thinking (Irving, Tapestry Press).

WRIGHT, S. (1996) Examining what residents look for in their role

models, Academic Medicine, 71, pp. 459–466.

Medical Teacher, Vol. 28, No. 7, 2006, pp. 656–659

Innovative learning: employing medical students towrite formative assessments*

SUZANNE CHAMBERLAIN1, ADRIAN FREEMAN1, JAMES OLDHAM1,DAVID SANDERS2, NICKY HUDSON3 & CHRIS RICKETTS1

1Peninsula Medical School, UK; 2Derriford Hospital, UK; 3University of Adelaide,South Australia

ABSTRACT Peninsula Medical School, UK, employed six

students to write MCQ items for a formative applied medical

knowledge item bank. The students successfully generated 260

quality MCQs in their six-week contracted period. Informal

feedback from students and two staff mentors suggests that

the exercise provided a very effective learning environment

and that students felt they were ‘being paid to learn’. Further

research is under way to track the progress of the students

involved in the exercise, and to formally evaluate the impact on

learning.

Introduction

Involving medical students in what are traditionally staff

concerns in the development or maintenance of an under-

graduate programme has many benefits to both staff and

Correspondence: Dr Suzanne E. Chamberlain, Research & Policy Analysis,

Assessment & Qualifications Alliance, Devas Street, Manchester M15 6EX,

UK. Tel: 0161 957 3904. Fax: 0161 958 3710. Email: schamberlain@

aqa.org.uk

*Institutions where work was carried out: Institute of Clinical Education,

Peninsula Medical School, Universities of Exeter and Plymouth.

Short communications

656

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Cal

ifor

nia

Irvi

ne o

n 10

/27/

14Fo

r pe

rson

al u

se o

nly.

students (Duffy & O’Neill, 2003). A study concerned with

the inclusion of students in a range of staff development

workshops found that each group gains greater insight into

the others’ perspective, and a deeper awareness of the

teaching–learning interface (Duffy & O’Neill, 2003). The

knowledge and experience of (final year) students has also

been utilized in the professional context to inform quality

assurance (QA) programmes during placements in a day

surgery setting (Rudkin et al., 1999). In this case, students

made significant inputs to the QA process, while participating

in an important teaching and learning opportunity that would

contribute towards their preparation for ‘medical practice in

the twenty-first century’ (Rudkin et al., 1999, p. 509). More

generally it has been found that exposure to alternative

learning environments (such as the local community,

university departments and others encountered in Special

Study Modules) aids the development of new skills and

enhances professional development (Yates et al., 2002).

Since opening in 2002, the school’s focus has been on

developing summative assessments. It has not been feasible

to steer staff resources in the direction of formative

assessment to establish a parallel formative item bank, despite

this being acknowledged as important by both students and

staff. A more innovative approach was needed to develop the

formative item bank. We recognized that students have a

close involvement with the curriculum and assessment

processes and under appropriate circumstances form a

valuable educational resource. Moreover, students’ participa-

tion in an item-writing task would promote self-directed

learning and knowledge acquisition in the context of paid

work. This report describes the workings of the group, their

productivity and quality of output, and discusses whether the

anticipated outcomes were achieved.

The task: writing items and creating a formative

assessment item bank

The aim was to develop an online bank of formative multiple-

choice questions (MCQs) relevant to each year of the

programme, case unit and curriculum theme. As part of the

formative assessment process, students would then be able to

access the item bank to test themselves as often as they

wanted. Extensive feedback would enhance the educational

value of formative tests by clarifying the reasoning behind

each of the item choices (see Figure 1). Items would also have

pointers to additional learning resources (texts, workbooks,

images, websites etc.) to encourage further self-directed

learning.

The MCQs were developed in Question Mark Perception,

a commercial web-based assessment package, and all

students were provided with training on their first day.

Students worked individually (occasionally) or in pairs (more

frequently) to develop scenarios, questions, distracters and

feedback. At regular intervals each item was reviewed by the

whole group with one academic and one clinical lead. Items

were added to the master bank only when fully approved by

the panel.

All year 1 and year 2 students were informed of the

forthcoming formative assessment student panel and were

invited to submit an application. A total of nine applications

for the full-time positions were received. Three students from

year 1 and three from year 2 were offered employment on a

full-time basis for six weeks in July and August 2004.

Productivity of the panel

It was estimated that each panel member could produce two

items per day for four days per week, with the fifth day used

for panel review. This would have resulted in approximately

288 items at the end of the six-week period. In the event, 260

items were created. The panel commented that they had

produced fewer items in their first week while they were

learning and practising their item-writing skills.

Figure 1 shows part of an item created by the student

panel.

Student feedback

At the end of the six-week period the student panel

participated in an informal evaluation and feedback session

around the following themes.

Sources used for items and their feedback

Approximately 80% of items were self-generated, stemming

from plenary lectures, study guides and other course

materials. Interestingly, students chose not to scour the

Internet for item ideas. The written feedback used a variety of

sources, mostly referenced by Phase 1 course materials for

familiarity.

Training in question construction and use

of Question Mark Perception

The panel’s first day was spent in training: first in item

writing, using Case & Swanson (1998) as a guide, and

second in the use of Question Mark Perception software.

The students commented that although the item-writing

training was useful, more reinforcement of the constituents of

a good item would have been useful.

How to maintain and enhance the item bank

To maintain and enhance the item bank it was suggested that

a team of students should be employed on a part-time basis.

One four-hour session per week was deemed sufficient to

maintain steady growth of the bank.

Perceptions of the process

Students identified the item and feedback writing as being of

huge value to their knowledge and understanding of

the curriculum. Being panel members had given students

the opportunity to consolidate some of their learning on the

programme so far. The panel were enthusiastic about the

fact that they felt they were ‘being paid to learn’.

Staff feedback

The two staff members commented that the students had

worked together very efficiently. They felt confident that the

task fostered a very effective learning environment, and that

students had used and developed their self-directed learning

Short communications

657

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Cal

ifor

nia

Irvi

ne o

n 10

/27/

14Fo

r pe

rson

al u

se o

nly.

skills. Guidance was sometimes required to help focus the

questions on clinically relevant material and avoid less

important topics. There were also times when the panel

needed extra ‘teaching’; this was particularly the case for

feasible distracters.

Future of the formative assessment student panel

The 260 items developed by the student panel will form

the foundations of online formative assessments to be

administered approximately three times per year. An

additional panel, employed on a part-time basis, has been

recruited and will continue to develop items for the

bank through term-time. It is intended that future student

panels will receive a ‘good item’ checklist or grid covering

issues such as technical item flaws (Case & Swanson, 1998)

and basic item-writing rules, against which to evaluate the

quality of items.

Further research is planned to track the progress of the

students involved in the project to formally evaluate the

impact on their learning. It will be interesting to seek

additional feedback from the students as they move through

the next year of their programme to explore the effects of

their involvement over time.

Notes on contributors

SUZANNE CHAMBERLAIN is a former Assessment Analyst at Peninsula

Medical School. She now works as a Senior Researcher for the

Assessment & Qualifications Alliance, Manchester.

ADRIAN FREEMAN is the Lead for Applied Medical Knowledge

Assessments and works part time as a GP.

JAMES OLDHAM is a teaching fellow in human structure, and a practising

osteopath.

DAVID SANDERS is a Surgical Senior House Officer at Derriford Hospital

working for six months full time as an Honorary University Fellow at the

PMS, teaching life sciences.

NICKY HUDSON, is a former Senior Lecturer at the Peninsula Medical

School. She is currently a Clinical Senior Lecturer at the University of

Adelaide in South Australia.

CHRIS RICKETTS is Director of Assessment and has been involved with

educational research for over 15 years with a particular interest in

computer-aided assessment.

References

CASE, S.M. & SWANSON, D.B. (1998) Constructing Written Test

Questions for the Basic and Clinical Sciences. Available online

at: http://www.nbme.org/about/itemwriting.asp (accessed 5 August

2004).

DUFFY, K.A. & O’NEILL, P.A. (2003) Involving medical students in staff

development activities, Medical Teacher, 25, pp. 191–194.

A 44-year-old woman has been referred by her GP to the surgical assessment unit

following an episode of right upper quadrant pain. On admission she is jaundiced with

pale fatty stools (steatorrhoea) and dark urine.

Based on this presentation where is the most likely site of a gallstone?

(1) Common bile duct

Correct. An obstruction of the common bile duct causes obstructive jaundice.

Steatorrhoea is a symptom of obstructive jaundice and is a result of impaired fat

emulsification (one of the roles of bile). The cause of dark urine and the jaundiced

appearance is due to increased circulation of conjugated bilirubin.

Kumar P, Clark M (2002) Clinical Medicine, 5th edn, pp. 388–395 (Oxford, W.B.

Saunders).

(2) Hartmann’s pouch

Wrong. This is the narrowest part of the gallbladder before it joins the cystic duct.

Impaction of a gallstone at this point does not disrupt the flow of bile through the

biliary system and therefore does not cause a clinical picture of obstructive jaundice.

Moore KL, Dalley AF (1999) Clinically Oriented Anatomy, 4th edn, p. 274 (London,

Lippincott, Williams & Wilkins).

(There were three other distracters with feedback at a similar level of detail).

Figure 1. Partial example of formative item.

Short communications

658

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Cal

ifor

nia

Irvi

ne o

n 10

/27/

14Fo

r pe

rson

al u

se o

nly.

QuestionMark Perception. Available online at: http://www.question-

mark.com/uk/home.htm (accessed 1 October 2004).

RUDKIN, G.E., O’DRISCOLL, M.C.E. & LIMB, R. (1999) Can medical

students contribute to quality assurance programmes in day surgery?

Medical Education, 33, pp. 509–514.

YATES, M.S., DREWERY, S. & MURDOCH-EATON, D.G. (2002)

Alternative learning environments: what do they contribute to profes-

sional development of medical students? Medical Teacher, 24,

pp. 609–615.

Medical Teacher, Vol. 28, No. 7, 2006, pp. 659–661

Using a filter to improve the chatroom experience ininteractive medical education*

CHRIS CANDLER1, PHEBE TUCKER2, JANICE TRIPLETT3 & RICHARD P. TRAUTMAN2

1American Association of Medical Colleges, USA; 2University of Oklahoma Health SciencesCenter, USA; 3Private Practice, Cheyenne, USA

ABSTRACT A computer chatroom exercise between faculty and

students was used in a Human Behavior course for first-year

medical students to provide an interactive exam review learning

exercise. Two conditions were compared—one that filtered out all

irrelevant student comments, and one without the filter that

permitted all student comments. Four one-hour chatroom sessions,

each with eight groups of five students, were conducted with all

comments recorded. Comments were rated as on- or off-task

by a blinded external faculty member. The filtered condition

resulted in increased on-task and decreased off-task student

comments compared with the non-filtered condition. However,

students made fewer total comments in the filtered situation.

Results suggested that a trade-off occurred between focused and

uninhibited faculty–student communication; the implications are

discussed.

Introduction

Large class sizes at most medical schools make it difficult for

faculty to connect with students one-on-one, challenging

medical educators to find educational techniques that

encourage teacher–learner interactions.(Mazmanian &

Davies, 2002; Brezis & Cohen, 2004). While the personal

contact provided by individual and small-group face-to-face

meetings may never be truly replaced, there remains a need

for more flexible communication modalities. Medical schools

have recently employed a variety of technologies to assist

computer-literate students through interactive exercises,

such as online lectures (Spickard et al., 2004), web-based

instruction with self-assessment (Turchin & Lehmann,

1999), audience response systems for feedback and evalua-

tion of lectures (Dufresne et al., 1996; Brezis & Cohen,

2004), and simulated patient scenarios (Woolridge, 1995;

Hammond, 2004). Computer-based interactive patient edu-

cation and counseling (Green & Fost, 1997; Tate & Zabinski,

2004) have been used in other health education settings.

Emerging synchronous (i.e. real-time) communication tech-

nologies such as network-based chatrooms may offer medical

students and faculty new avenues for interactive discussion

(Dufresne et al., 1996; Kantor & Gall, 2002; Anon, 2003).

The challenge of this technology is to use it in such a way that

all voices are heard and the discussion remains on task.

During a pilot exercise, the College of Medicine at the

University of Oklahoma investigated several chatroom

technologies under a variety of configurations. Student

feedback during the initial chatroom exercises was mixed;

major criticisms were not aimed at the technology itself but

rather at the disruptive off-task student commentary, which

often overwhelmed legitimate observations and questions.

A variety of techniques have been used to identify

educationally desirable interaction in other chat-room

environments (Wortham, 1999).

In this study we assessed the educational utility of a

chatroom exercise under two separate conditions—a con-

trolled state that filtered off-task student comments via a

software feature and an open state that allowed all student

comments to be displayed to all students and faculty.

The investigators were interested to know whether filtering

comments would reduce the proportion of off-task comments

generated by students.

Methods

This educational study was granted exemption by the

University of Oklahoma Health Sciences Center’s

Institutional Review Board. The chatroom activity was

conducted during the first-year Human Behavior I course

as an exam review session. Because students have tradition-

ally experienced difficulty with the often abstract style of

Correspondence: Phebe Tucker, MD, Department of Psychiatry, WP 3440,

PO Box 26901, University of Oklahoma Health Sciences Center, Oklahoma

City, OK 73190, USA. Tel: 405 271-4488. Fax: 405 271-1988. Email:

Phebe-Tucker@ouhsc.edu

*This manuscript was produced in the Office of Educational Development and

Support and the Department of Psychiatry in the College of Medicine,

University of Oklahoma Health Sciences Center, PO Box 26901, Oklahoma

City, OK 73190, USA.

Short communications

659

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Cal

ifor

nia

Irvi

ne o

n 10

/27/

14Fo

r pe

rson

al u

se o

nly.