standards for question writing
DESCRIPTION
Standards for Question Writing. Mary Grantner, MA Senior Manager, Physician Assessment. Fundamentals. What is a test? An instrument that tells us something about student knowledge, skills, and/or abilities. . Fundamentals. What is a test item? - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/1.jpg)
Standards for Question Writing
Mary Grantner, MASenior Manager, Physician
Assessment
![Page 2: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/2.jpg)
Fundamentals
What is a test?
An instrument that tells us something about student knowledge, skills, and/or abilities.
![Page 3: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/3.jpg)
Fundamentals
What is a test item?Cue requiring student response with rules for scoring response.
Response can be prompted by an instruction to perform a task, or a question requiring an answer.
![Page 4: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/4.jpg)
FundamentalsBy definition, ARS employs a question requiring an answer.
The default test item is multiple-choice.• Stem• Responses• Single correct response (best-answer
format)
![Page 5: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/5.jpg)
Use those nifty keypads
A multiple-choice test item can measure which of the following student achievements?
A. Critical thinking skillB. High-inference psychomotor skillC. Declarative knowledgeD. High-inference cognitive ability
![Page 6: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/6.jpg)
A multiple-choice test item can measure which of the following student achievements?A. Critical thinking skill*B. High-inference psychomotor skillC. Declarative knowledge*D. High-inference cognitive ability*
![Page 7: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/7.jpg)
Approach to Writing• Identify Key Concepts
– What do you want to measure?• Identify Actions
– What do you want the learner to be able to do?
• Ask for Direct Action/Concept• Provide a Single Correct Answer
![Page 8: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/8.jpg)
Identify Key Concepts• Start with learning objectives• When they walk out of the room, will
learners ask “so what?” What should they now understand?
• Put each concept into a declarative sentence.
![Page 9: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/9.jpg)
Identify Actions• Drill down – what should the learner do
with the concept?• Verbs “do” – consider some verbs that
apply to the Key Concept.• Put each action into a declarative
sentence.
![Page 10: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/10.jpg)
A Word about Verbs• Strong verbs are vital to any good writing.• Be direct and active.
– Form of “to be” + past participle – Mary was given a book by John – is passive.
• Avoid forms of “to be” when possible.• And (drumroll please)…
![Page 11: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/11.jpg)
“When you see an adverb, kill it.”
-- Mark Twain
What does it mean to walk quickly?Briskly?
Urgently?Fast?
![Page 12: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/12.jpg)
Choosing VerbsMeasure Recall or Understanding, use:
CharacterizeDefine
DemonstrateDescribe
FindIllustrate
![Page 13: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/13.jpg)
Choosing VerbsMeasure Problem-solving, use:
ClassifyConcludeDiagnoseDetermine
PlanRecommend
![Page 14: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/14.jpg)
Ask for Direct Action/Concept
• Focus on a single Key Concept at a time.– Less really is more.
• Using an active verb, ask a direct question.
• Ask the learner to do something.• Keep it simple!
![Page 15: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/15.jpg)
Provide a Single Correct Answer• Focus on “real life”.• Be as brief as possible.
– Answer should contain only what is absolutely necessary.
• Keep it simple!
![Page 16: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/16.jpg)
Keypads Up!Which of the following best defines adverb?a. Modifies a noun or other substantive. b. Ob to be executed by item writer.c. Expresses action, existence, or
occurrence.d. Modifies a verb, adjective, or other
adverb.
![Page 17: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/17.jpg)
Which of the following best defines adverb?
a. Modifies a noun or other substantive. b. Obliged to be executed by item writer.c. Expresses action, existence, or
occurrence.d. Modifies a verb, adjective, or other
adverb. *
![Page 18: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/18.jpg)
Try ThisIdentify Key Concept:
Pulmonary embolism (PE) and deep venous thrombosis (DVT) are part of the same complex and common process (i.e. venous thromboembolism), which has a wide spectrum of clinical presentations and consequences, ranging from no symptoms or sequelae to a lethal outcome. DVT usually starts in the deep calf veins and propagates superiorly, and it is associated with symptomatic PE in about one-third of untreated patients. DVT is commonly confused with other conditions such as cellulitis, leg edema, and chronic venous insufficiency, and PE is also notoriously difficult to diagnose.
![Page 19: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/19.jpg)
You Decide• As the content expert, it’s up to you to
decide what is most important.• Let’s choose this:
PE is also notoriously difficult to diagnose
• Write a direct question based on the paragraph.
![Page 20: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/20.jpg)
The QuestionWhich of the following statements best explains the difficulties in diagnosing lower extremity DVT?
Answer:Is clinically similar to other conditions of the lower extremities.
![Page 21: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/21.jpg)
Wrong Answers• Also called distractors, are hard to write.• Try using these:
– Common misconceptions you’ve seen;– Misapply a principle (i.e. add when you should
subtract);– Ways in which things are related (cause or
coincidence?).
![Page 22: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/22.jpg)
Which of the following statements best explains the difficulties in diagnosing lower extremity DVT?
a. Is a distinctly different process than PE.b. Occurs rarely in cases of symptomatic PE.c. Is clinically similar to other conditions of the
lower extremities.*d. Usually originates in difficult to image deep
veins of the thigh.
![Page 23: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/23.jpg)
Other Rules/Hints• Just as in life, a fact is seldom absolutely
true or absolutely false. No T/F questions!• “All / None of the Above” is essentially T/F
– Not allowed.• Avoid any negative question
– Which is false?– Which is incorrect?
![Page 24: Standards for Question Writing](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56816110550346895dd06b1e/html5/thumbnails/24.jpg)
Other Rules/Hints• Stem is short and options long – a poor item!
– As much information as possible in the stem and keep the options short
• Avoid absolute terms. – Words like “always” or “never” are rarely true in life
and so are almost never the right answer on a test. • Avoid word repeats (word that appears in both
stem and response); points to a correct answer.• Language in the options must be parallel (i.e. all
diagnoses, or all anatomic entities).