Writing Selection Items

Multiple Choice

Angela,

Amber,

Cristian,

Jason,

Leanne

Assessment of Student Achievement Third custom edition for the University of Alberta. Chapter 4

Introduction

Learning Outcomes: 1. Describe the characteristics of

multiple-choice items.

2. Distinguish the strengths and

limitations of multiple-choice items.

3. Distinguish between well worded and

poorly worded multiple-choice items.

4. Identify and correct faults in bad

questions.

5. Match multiple-choice questions with

intended learning outcomes.

6. Construct multiple-choice questions

that are worded properly, appropriate

for learning outcomes and free of

mistakes.

Nature of Multiple Choice Items

They consist on a stem, which

presents a problem situation

and several alternatives

(options or choices), which

provide possible solutions to the

problem. Example: Which one of the following items types is an example of a supply-type test item? ……… An example of a supply-test item is the:

Uses of Multiple-Choice Items

Knowledge of: • Terminology

• Specific facts

• Conventions

• Trends and Sequences

• Classifications and Categories

• Criteria

• Methodology

• Principles and Generalizations

• Theories and Structures

Comprehension Items Measure at the lowest level of

understanding. To determine

whether the students have grasped

the meaning of the material without

required them to apply it.

Example: Outcome: Identify an example of a term Which of the following statements contain a specific determiner? a. America is a continent b. America was discovered in 1492 c. America has some big industries ***** d. America’s population is increasing

Application Items

Application of various aspects of

knowledge: facts, concepts, principles,

rules, methods, theories.

Ex.

• Which of the following methods is best for ___________?

• Which principle would be best for solving __________?

• What is the most probable effect of __________?

Multiple-Choice Items Strengths: 1. Learning outcomes from simple to complex can ne measured.

2. Highly structured and clear tasks are provided.

3. A broad sample of achievement can be measured.

4. Incorrect alternatives provide diagnostic information.

5. Scores are less influenced by guessing than true-false items.

6. Scoring is easy, objective, and reliable.

Limitations: 1. Constructing good items is time consuming.

2. It is frequently difficult to find plausible distractors.

3. This item is ineffective for measuring some types of

problem solving and the ability to organize and

express ideas.

4. Score can influenced by reading ability.

Rules for writing Multiple Choice Items

Nothing in the content or

structure of the item should

prevent an informed student

from responding correctly.

Nothing in the content or

structure of the item should

enable an uninformed student

to select the correct answer.

Rule # 1 Design each item to

measure an

important learning

outcome.

Include novelty!

Rule # 2 Ex.

Poor – A table of specifications: a. Indicates how a test will be used to improve

learning.

b. Provides a more balanced sampling of content ***

c. Arranges the instructional objectives in order of the

importance.

d. Specifies the method of scoring to be used on a

test.

Better- What is the main advantage of using a

table of specifications when preparing an

achievement test? a. It reduces the amount of time required

b. It improves the sampling of content ***

c. It makes the construction of test items easier

d. It increases the objectivity of the test

Present a single clearly

formulated problem in the

stem of the item.

Rule # 3 State the stem of the item in

simple, clear language.

Ex.

Poor – The paucity of plausible, but incorrect,

statements that can be related to a central idea

poses a problem when constructing which of the following types of test items?

Better – The lack of plausible, but incorrect,

alternatives will cause the greatest difficulty when constructing:

Rule # 4 Put as much of the

wording as possible

in the stem of the

item. Ex.

Poor – In objective testing, the term

objective: a. Refers to the method of identifying the

learning outcomes

b. Refers ti the method of selecting the

test content

c. Refers to the method of presenting the

problem

d. Refers to the method of scoring the

answer ****

Better – In objective testing, the term objective refers to the method of: a. Identifying the learning outcomes

b. Select the test content

c. Presenting the problem

d. Scoring the answers ****

Rule # 5 State the stem of the item in positive

form, wherever possible

To avoid test maker frustration, keep in mind… Which of the following is a category in the taxonomy of the cognitive domain? a. Comprehension *** b. (distracter needed) c. (distracter needed) d. (distracter needed)

Which of the following is not a category in the taxonomy of the cognitive domain? a. Comprehension b. Application c. Analysis d. (answer needed) ***

Rule # 6 Emphasize negative wording

whenever it is used in the

stem of an item.

Ex.

Poor – Which one of the following is not a desirable

practice when preparing

multiple-choice items?

Better – All of the following are

desirable practices when

preparing multiple-choice

items EXCEPT:

Rule # 7 Make certain that the intended answer is

correct or clearly best.

Ex.

Poor – What is the purpose of

classroom testing?

Better – One purpose of

classroom testing is: Or

The main purpose of classroom

testing is:

Rule # 8

Ex.

Poor – The recall of factual information

can be measured best with a :

a)matching item

b)multiple-choice item

c)short-answer item***

d)essay question

Better – The recall of factual information

can be measured best with:

a)matching items

b)multiple-choice items

c)short-answer items***

d)essay questions

Make all alternatives grammatically consistent with

the stem of the item and parallel in form.

Rule # 9 Avoid verbal clues that

might enable students

to select the correct

answer or to eliminate

an incorrect alternative

Consider:

a. Similarity of wording in both the stem and the correct answer is

one of the most obvious clues.

b. Setting the correct answer in text book language

c. Stating the correct answer in greater detail

d. Including absolute terms in the distracters (always, never…)

e. Including two responses that have the same meaning

Rule # 10 Make the distractors plausible and

attractive to the uninformed. Ex.

Poor – Obtaining a dependable ranking of

students is of major concern when using:

a)norm-referenced summative test***

b)behavior descriptions

c)checklists

d)questionnaires

Better – Obtaining a dependable ranking of

students is of major concern when using: a)norm-referenced summative tests***

b)Teacher-made diagnostic tests

c)mastery achievement tests

d)criterion-referenced formative tests

Rule # 11 Ex.

Poor – One advantage of multiple-choice items over essay

questions is that they:

a. Measure more complex outcomes

b. Depend more on recall

c. Require less time to score

d. Provide for a more extensive sampling of course content ***

Better – One advantage of multiple choice items over essay

questions is that they:

a. Provide for the measurement of more complex learning

outcomes

b. Place greater emphasis on the recall of factual information

c. Require less time for test preparing and scoring

d. Provide for a more extensive sampling of course content ***

Vary the relative length of

the correct answer to

eliminate length as a clue

Rule # 12

Avoid using the alternative “all of the

above,” and use “none of the above”

with extreme caution. Ex.

Poor – Which of the following is a category in the

taxonomy of the cognitive domain? a)critical thinking

b)scientific thinking

c)reasoning ability

d)none of the above***

Rule # 13 Vary the position of

the correct answer in

a random manner

Open a book! 1=A 3=B 5=C 7=D 9=E

Rule # 14

Control the difficulty of the

item either by varying the

problem in the stem or by

changing the alternatives.

Rule # 15

Make certain each item is independent of the other items in the test.

Rule # 16 Use an efficient

item format.

Rule # 17 Follow the normal rules of

grammar

Rule # 18

Break (or bend) any

of these rules if it will

improve the

effectiveness of the

item.

Multiple-choice Checklist

1. Is this type of item appropriate for measuring the intended learning

outcomes?

2. Does this item task match the learning task to be measured?

3. Does the stem of the item present a single, clearly formulated problem?

4. Is the stem stated in simple, clear language?

5. Is the stem worded so that there is no repetition of material in the

alternatives?

6. Is the stem stated in positive form wherever possible?

Multiple-choice Checklist

7. If negative wording is used in the stem, is it emphasized (underlining

or caps)?

8. Is the intended answer correct or clearly best?

9. Are all alternatives grammatically consistent with the stem and

parallel in form?

10.Are the alternatives free from verbal clues to the correct answer?

11.Are the distractors plausible and attractive to the uninformed?

12.To eliminate length as a clue, is the relative length of the correct

answer varied?

13.Has the alternative “all of the above” been avoided and “none of

the above” used only when appropriate?

14. Is the position of the correct answer varied so that there is no

detectable pattern?

15.Does the item format and grammar usage provide for efficient test

taking?