multiple choice question writing - open michigan choice question writing caren m. stalburg, md ma...
TRANSCRIPT
![Page 1: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/1.jpg)
Multiple Choice Question Writing
Caren M. Stalburg, MD MA Clinical Assistant Professor
Obstetrics and Gynecology and Medical Education
Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike-3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
![Page 2: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/2.jpg)
Intended Learning Outcomes • Understand how to ‘blueprint’ your curriculum • Define the components of a multiple choice
question • Construct multiple choice questions to assess
higher order learning
![Page 3: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/3.jpg)
So you want to teach??
– What? – Where? – Whom? – How? – How will you know they
learned???? © Used with permission from Microsoft
![Page 4: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/4.jpg)
Curriculum Blueprint • Work backwards !!!!
– Define the ILOs before you start • Who cares??
– Consider needs assessment, define your learner • What teaching tools will you use?
– Pick a pedagogy • How will your measure their success?
– Assessments…….
![Page 5: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/5.jpg)
(Perfect?) Assessments • Reliability = reproducibility of results
– Limited by sampling errors (items, examiners, etc) • Validity = actually measuring what you think
you are measuring – Content – Construct
![Page 6: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/6.jpg)
ILO Domain Pedagogy Asssessment
Bloom’s taxonomy, Miller’s pyramid
K, S, A ? Match to ILO Matches the ILO
Knowledge Skills Attitudes
Assessment Type
MCQ’s Short answer
Problem solving Essays
Oral exams
OSCE OSATS DOPS
Essays Portfolios
Art Literature
![Page 7: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/7.jpg)
What is a multiple choice question? a) A way to assess factual knowledge? b) An efficient way to assess
individuals? c) A way to assess higher order
understanding of information? d) A question with only one right
answer? e) All of the above???????
© Used with permission from Microsoft
![Page 8: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/8.jpg)
Types of MCQs • True/False item format
– Answer choices must be unambiguous – Lends towards recall of isolated facts
• Single-best option format – Allows for best matched answer – Can incorporate multiple ‘cognitive jumps’
![Page 9: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/9.jpg)
Anatomy of an MCQ An instructor is teaching a module on designing assessments for distance learning. There are more than 5,000 students. She wants to confirm that students learn the components of a multiple choice question. Which of the following assessment types would be most suited to her purpose? a) Essays b) Group projects c) Video-taped submissions d) Multiple choice question examinations
Item Stem
Lead in question
Answer + Distractor
![Page 10: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/10.jpg)
Item Stem • Focused on a concept, not memorization • Generally in the form of a clinical vignette • Enough information that one can formulate and answer
without looking at options A 35 year old woman complains of nausea and vomiting for the past 3 weeks. She also notes urinary frequency without burning. Her last menses was 7 weeks ago. On examination, she is afebrile. Notable finding on exam includes a mildly enlarged pelvic mass. βhcg = 6,582.
![Page 11: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/11.jpg)
Lead in question • Specific to the vignette • Cannot be answered simply by reading the
lead-in alone A 35 year old woman complains of nausea and vomiting for the past 3 weeks. She also notes urinary frequency without burning. Her last menses was 7 weeks ago. On examination, she is afebrile. Notable finding on exam includes a mildly enlarged pelvic mass. βhcg = 6,582.
Which of the following is the most likely diagnosis?
![Page 12: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/12.jpg)
Answer + Distractor • All options should be similar types • Alphabetize • Avoid “cue-ing” • Plausible to the vignette A 35 year old woman complains of nausea and vomiting for the past 3 weeks. She also notes urinary frequency without burning. Her last menses was 7 weeks ago. On examination, she is afebrile. Notable finding on exam includes a mildly enlarged pelvic mass. βhcg = 6,582 Which of the following is the most likely diagnosis? a) Leiomyoma b) Pregnancy c) Ovarian cancer d) Colon cancer e) Small bowel obstruction
![Page 13: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/13.jpg)
Poorly written stem…... An instructor is teaching a module on designing assessments for distance learning. There are more than 5,000 students. She wants to confirm that students learn from the session. Which of the following assessment types would be most suited to her purpose? a) Essays b) Group projects c) Video-taped submissions d) Multiple choice question examinations
![Page 14: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/14.jpg)
Poorly written lead in…... An instructor is teaching a module on designing assessments for distance learning. There are more than 5,000 students. She wants to confirm that students learn the components of a multiple choice question. Which of the following assessment types is best for testing memory of the specific components of an MCQ? a) Essays b) Group projects c) Video-taped submissions d) Multiple choice question examinations
![Page 15: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/15.jpg)
Poorly written distractors…... An instructor is teaching a module on designing assessments for distance learning. There are more than 5,000 students. She wants to confirm that students learn the components of a multiple choice question. Which of the following assessment types would be most suited to her purpose? a) Essays b) Group projects c) Video-taped submissions d) Multiple choice question examinations
![Page 16: Multiple Choice Question Writing - Open Michigan Choice Question Writing Caren M. Stalburg, MD MA Clinical Assistant Professor Obstetrics and Gynecology and Medical Education Unless](https://reader031.vdocuments.mx/reader031/viewer/2022022508/5acfb0e97f8b9ad24f8cbebb/html5/thumbnails/16.jpg)
Very helpful resource
• National Board of Medical Examiners® (NBME®) Item writing manual