Writing Multiple Choice Questions

and Assessing Short Answer Responses

Alice AspinallCarlinda D’Alimonte

Writing Multiple-Choice Questions to

Measure Higher-Level Thinking Skills

This portion of the slide show is adapted by C. D’Alimonte from a Presentation by: Jana L. Scott

Missouri Assessment Program, University of MO- Columbia

2

Rationale for Training • EQAO Test Questions• Poor performance of students • Incongruence • We know that students are capable of

passing the test, but need help and practice.• Working on a student’s ability to answer

multiple-choice questions and short answer responses will help to make sure he or she passes.

3

What is the OSSLT• A test of students’ writing and reading abilities

they must pass in order to graduate.• Students read several types of text (narrative,

informational, and graphical) and answer multiple-choice and short answer questions on them.

• Students write several pieces for different purposes (news report, supported opinion, summary paragraph)

Our Focus

•MCQ’s

•Open Response: Short Written Response to Reading Passages

MCQ’s:Three Things to Consider

1. Student’s Process2. Structure of the Question3. Cognitive Process Involved in

Response6

Process

Three-Step Process Students Should Follow

• Before they attempt any of these, read the passage for basic understanding.

Step One

Step Two

Read the first question.

Do they know the answer?

If yes, read through all the choices and make sure they select the the most correct answer. Do not choose the first correct answer they see.

Step Three

If they do not know the answer right away:

- read the four choices- cross out obviously wrong answers- highlight key words in the question and scan the

text for those words to try to find the right answer

Be aware that some answers may seem correct, but there is only one best answer.

Next: • Go back to any questions that they circled

and left unanswered.

• If time permits, repeat the steps to answering multiple choice questions and review the passage.

• Make a selection - always. Never leave an answer blank.

Repeat this three-step process with each question in the passage.

Review – Answering MC Questions

Structure

Structure:How the Question is Constructed

- Use questions instead of incomplete statements- The student should be able to answer the question

before reading the possible answers or have a clear, specific idea of what is being asked

- Use plausible distracters- Keep options at a similar length- Be grammatically correct- Avoid clues to the answer- Use only “most correct” answer- Do not use “all of the above,” “none of the above,” or

negative questions or answers- Give clear instructions

Cognitive Process

Multiple Choice Questions

There are different types. Sometimes you

1. find the answer right in the text.2. need to infer – that is, read between

the lines or extend beyond what is written in the passage.

18

Making an Inference Means…..

•Drawing conclusions•Making deductions •Making conjectures•Speculating •Making presumptions

19

Cognitive Process

Emphasize Higher-Level Thinking – Think of Bloom’s Taxonomy

MCQ’s Can Require or Ask For

21

GeneralizationsSummarizations ConclusionsJustify Motives or Reasons Judgments or PredictionsShowing Relevance or SignificanceJustification/Evaluation of Methods, Procedures, Solutions, etc.

SupportOpinions Related to Authorial Choices/Text Features ApplicationComparison Interpretation/Explaining Cause-Effect Relationships

Generalizations:A General Thought, Idea or Premise Arrived at as a Result of or Based on Specific Information

___________________________1. Which word best describes the main character? the

central conflict? the initial problem?2. What is most likely the theme of the story?

a. Nice guys often finish lastb. Kindness is not always its own rewardc. Kindness is its own rewardd. Time really does heal

3. How can you tell that more discoveries are likely to be made in _________?

4. Based on the data/chart, what might one generalize?5. Which equation best reflects the input/output

information contained in the chart?

Summarizations:A Statement Which Specifies the Main Idea or

Main Points___________________________

1. Which is the best summary of the article/paragraph/excerpt?

a. There are many types of jobs and many people looking for work. It is important to find the best person suited to the job.

b. Unemployment is on the rise. There are many unemployed people who are applying for different types of jobs.

c. Job openings are on the rise. It is important that the unemployed work hard to try to find a job.

d. Unemployment is on the rise. Job openings are on the decline due to many economic factors

2. Which statement best summarizes the results of Experiment 2?

3. Which statement best summarizes the impact of the Truman Doctrine/_________Treaty/etc. upon world affairs?

Conclusions:A Judgement Formed After Critical Thought

___________________________

1. What might the reader conclude based on John’s actions?2. John put some water in a graduated cylinder and wrote down

the height of the water. Then, he placed a rock into the cylinder and wrote down the height of the water again. John subtracted the number of milliliters of the water without the rock from the number of milliliters of water with the rock. What did John do?

a. Found the volume of the rockb. Found the mass of the rockc. Found the amount of pressure needed to displace waterd. Found the size of rock needed to displace water

3. Based on the data in the chart/graph/data set, what might one conclude?

4. Based on the results of the problem, what might one conclude?

Justifying Motives or Reasons:Consider Actions, Ideas, Principles, etc.

___________________________

1. What is the most likely reason Tom walked out of the meeting?a. He was afraid of being confronted about his previous actionsb. He did not feel prepared and needed time to consider how he would handle

his presentationc. He was angry about what had happened in his boss’s office that morningd. He remembered that he needed to make an urgent phone call

2. Which of the following best justifies Fred Scott’s decision to sue his employer?

3. A student recorded the dimensions and location of a shadow cast by a tree at sunrise. Every hour throughout the day, until sundown, the student repeated this same process. At the end of the day, the student compared the dimensions and the locations. What might the student be trying to find out?

4. Which statement best describes the role a circle plays in being able to find the area of a minor segment?

Making Judgements/Predictions:Determinations Related to What Might Happen or Occur Next

___________________________

1. Based on the information given, which statement best describes what might happen to John’s annual expense account if he takes in two more roommates for six months (December to May)?

2. If the US had decided to remain uninvolved in WWII, what would most likely have happened?

a. There would have been little change in the outcome of the warb. The war would have lasted much longerc. Japan would have invaded Germanyd. Germany would have invaded Japan

3. Based on the diagram showing how John wired the electrical circuit, which statement best predicts what will most likely happen when John pushes the ‘ON’ switch?

4. Based on the data showing the amount of sunspot activity over the course of the past ten years, which statement best describes how future communications (radio, television, telephone, and satellite signals) might be affected?

Showing Relevance or Significance:Relate to the Matter Under

Consideration or Importance of Something___________________________

1. What does the robe symbolize in the passage?a. Loveb. Greedc. Charityd. Hope

2. Given a data set, what role would the data play in the creation of a testable question?

3. What role did the “control group” play in the scientific investigation/experimentation/inquiry?

4. Why is rewriting the problem using the reciprocal necessary when dividing fractions?

Requiring Justification/Evaluation:Examination of Methods, Procedures, Solutions

___________________________

1. Which of the following is the best procedure to use to determine the amount of heat released when a substance is burned in oxygen?

2. A student was asked the following question: ‘Briefly list and explain the various steps in creating and implementing a research plan.’ The student’s answer was this: ‘A research plan should contain ________.’ How would you judge this student’s answer?

3. Which of the following best describes the effectiveness of FDR’s New Deal Plan as a basis to improve the depressed US economy?

a. Highly effectiveb. Somewhat effectivec. Somewhat ineffectived. Highly ineffective

Providing Support:Reasoning or Evidence Behind Ideas

___________________________

1. Which statement best supports the idea that…?2. Which of the character’s actions best supports the idea of John’s

impulsiveness?3. Which of the following hypotheses about the relationship

between temperature and the density of a solid is best supported by the data in Figure 4? As the temperature of a solid increases, what does the density of the solid do?

a. Increases onlyb. Decreases onlyc. Increases, then decreasesd. Decreases, then increases

4. What evidence best supports the idea that Truman’s decision to drop the atomic bomb led to an early end to World War II?

Asking for Opinions on Authorial Choices/Text Features:

Ideas About the Writer’s Purpose, Techniques or Use of Format

___________________________

1. What is the most likely reason the author uses slang/ a diary format/subtitles/italics/graphs/etc?

2. What purpose is best served by the semicolon/colon/brackets/bold/dash/quotation marks/etc. in paragraph 5?

3. What is the most likely reason that the author included paragraph 4 in this passage?

a. To provide a counter argumentb. To provide more background informationc. To show that she is being faird. To create doubt in the reader’s mind

Requiring Application:The Use of Previously Learned Information

___________________________

1. Which set of questions would be best to use to guide research about effective agricultural techniques used in the 19th century?

2. Which of the following BEST describes the change that occurs when a batter hits a baseball?

3. The table shows the price of different quantities of medium-sized apples at Tom’s Corner Grocery Store. What is the least amount of money needed to buy exactly 20 medium-sized apples if the bags must be sold intact and there is no tax charged?

# of Apples Bag of 1 Bag of 6 Bag of 12 Price $.30 $1.20 $2.10

a. $3.60b. $3.90c. $4.20d. $5.50

Making Comparisons:___________________________

1. To what popular children’s story reviewed in class might the theme of this story/poem best be compared?

a. Little Red Riding Hoodb. Cinderellac. The Lion Kingd. Chicken Little

2. To what might _________ best be compared?

Requiring Interpreting/Explaining Cause and Effect Relationships:

Describing Outcomes or Impact of Events or Situations ___________________________

1. Why does investing money in the stock market pose a risk for an investor?

a. There is a high level of corruption among investorsb. The stock market is volatilec. History has shown that stock prices do not rise over long periods of timed. History has shown that stock prices decrease over long periods of time

2. Which of the following best describes the effect of _________on _________?

3. If the nominal gross national product (GNP) increases at a rate of 10% per year and the GNP deflator increases at 8% per year, what happens to the GNP?

4. Use the line graph to answer the question. If the vertical scale started at 0 and each line was labeled by whole numbers up to 10, what effect would it have on the data displayed in the graph?

5. Which statement best describes the impact the opening of the Panama Canal had upon the US economy?

Your Task • Work in a small group to write two higher-level

multiple-choice questions that will measure a part of any two objectives.

• Use the Planning Chart to write your questions and determine which higher order thinking task you are tapping.

• Refer to the sample questions and read any clarifications.

• Write structurally solid questions that force the student to use higher order thinking .

Worksheet

35

Example

36

Which of the following best describes the reason it is important to follow the directions listed in the passage?

a. To be able to accurately complete income tax forms

b. To be able to learn to file taxes before deadline

c. To ensure the government will not do an audit

d. To ensure the correct information is on all forms

X

X

Share Your Favorite Question Tell ….• What higher order thinking skills you

are tapping, • your question and answer choices.

37

Assessing Short-Answer (Open-Response)

Responses

Types of Short-Answer Questions

1. Answer the question using evidence from the text.

2. Answer the question using evidence from the text and your own ideas.

Steps to Answering Short-Answer (Open Response) Questions

After Reading the TextStep One: Categorize the Question (evidence from text only or both

evidence from text and own ideas)Step Two: Plan the AnswerStep Three: Write the AnswerStep Four: Check Conventions (grammar, punctuation, spelling, capitalization)

Step One: Categorizing The Question

Short Answer Question One

Explain why the final sentence is an effective conclusion to the selection. Use relevant and specific information from the selection to support your answer.

Use information in this selection and your own ideas to explain whether or not Ben is a good waiter.

Short Answer Question Two

Process

What was the same/different about your approach to the two questions?

Step Two: Planning The Answer

Step Two: Planning The Answer

• Highlight the parts of the question that tell what’s expected.

• Underline the words in the question that you will use to form your answer.

• Rough out the answer. Use point form or highlight the examples from the text you want to include. Use the rough notes section for this.

• Use point form to jot down connections to your own ideas (if required).

Question Type One

ExplainExplain why the why the final sentencefinal sentence is an is an effective conclusion to the selection. effective conclusion to the selection. UseUse relevant and specific informationrelevant and specific information from the selection to support your from the selection to support your answer.answer.

Question Type Two

Use information in the selection and your own ideas to explain whether or not Ben is a good waiter.

Highlight Details

Making Connections When Required

• Jot down answers to these questions as you try to make connections to your own ideas:

– Text to self connections (Have I been in a situation like this before? What was important? What did I learn? What was the same/different?);

– Text to media (Have I read, seen or heard about this before? What did I learn? Was important, etc);

– Text to world (Do I know someone who has been in a situation like this? What was important what did the person learn.).

Step Three: Writing the answer

Question Type One

ExplainExplain why the why the final sentencefinal sentence is an is an effective conclusion to the selection. effective conclusion to the selection. UseUse relevant and specific informationrelevant and specific information from the selection to support your from the selection to support your answer.answer.

Open Response Type One

• Answer

• Evidence

Answer the Question

• Question stem – Use key words from the question.

• Main idea.

For example: The final sentence is an effective conclusion

to the selection because it provides us with the information we need to understand the behaviour of all the characters.

Evidence from the Text

• Find the examples in the text to support the main idea.

For example: It explains why Ali and Dana insisted on

having a table in Ben’s section and later responded with rude comments. It also explains why Ben asks, “That’s it? You waited twenty minutes in the cold to order one plate of fries?”

Answer and Evidence

The final sentence is an effective conclusion to the selection because it provides us with the information we need to understand the behaviour of all the characters. It explains why Ali and Dana insisted on having a table in Ben’s section and later responded with rude comments. It also explains why Ben asks, “That’s it? You waited twenty minutes in the cold to order one plate of fries?”

Question Type Two

Use information in the selection and your own ideas to explain whether or not Ben is a good waiter.

Open Response Type Two

• Answer

• Evidence

• Connections

Answer the Question • Question stem – Use key words from the

question.• Main idea.

For example: Ben is a good waiter because he is polite and

treats his siblings as regular customers even though they are teasing him.

Evidence from the Text

• Find the examples in the text to support the main idea.

For example: Ben used a polite manner when he spoke to

them. He also apologized that they had to wait twenty minutes for a table.

Connections• Connecting the text to media, to the world, to

self• What does the reader already know that has

a connection to the text?

For example: Good waiters don’t argue with customers and

are always polite. They believe the customer is always right.

Answer, Evidence and

Connections Ben is a good waiter because he is polite and

treats his siblings as regular customers even though they are teasing him. Ben used a polite manner when he spoke to them. He also apologized that they had to wait twenty minutes for a table. Good waiters don’t argue with customers and are always polite. They believe the customer is always right.

Keys to Answering OR Questions

Students need to

• know and use strategies

• make connections when asked