oscar vergara chihlee institute of technology july 28, 2014
TRANSCRIPT
Test-Making StrategiesWorkshop
Oscar VergaraChihlee Institute of Technology
July 28, 2014
Lecturer at CIT since 2012
More than 10 years ESL/EFL teaching experience
Co-author of a variety of EFL textbooks & test data banks
Certified IELTS examiner
About Me
Some Guiding Principles
Multiple Choice Test Questions
Workshop Activity: Suggestions for Creating Good Tests
Q & A
Today’s Content
Clarity of questions & instructions
Reliability & Validity
Fairness
Some Guiding Principles
Instructions
Clearly state what is required Use simple language / unambiguous
Questions / Stems Content or structure should not prevent an
informed student from answering correctly Don’t include distracting or unnecessary details
Clarity
Do the exam questions measure what they
purport to measure? Do the questions test your course goals?
Does the exam accurately reflect the achievement of what you intended to teach? Classes may differ, so different versions may be
needed.
Reliability & Validity
Provide clear expectations about student
performance Provide examples / practice with mock tests or
past tests Students should know expectations of how their
grade on tests reflects their skills (not extraneous factors)
Fairness
Difficult to achieve due to:
Demands on time
Grading resources
Require many versions
Guiding Principles
Taxonomy of Educational Objectives
Specifies different abilities and behaviors related to thinking processes
Contains 6 distinct categories
Guiding Principles: Bloom’s Taxonomy
Knowledge
Comprehension
Application
Analysis
SynthesisEvaluation
- Rote memorization- Recall
- Interpret information
- Test questions on facts, rules &
principles
- Apply concepts to situations
- Distinguish or differentiate between
ideas
- Formulate or modify ideas
- Assess, criticize, justify
Knowledge: Remember
Knowledge of terms and concepts Recall of information
Comprehension: Understand Comprehension including translating,
summarizing, demonstrating, discussing Application: Apply
Apply what was learned Use problem-solving methods
Guiding Principles
Please work in pairs or small groups
Do only part A
Discuss
Worksheet
Made up of a single question called a stem
Many possible choices with one correct answer
Several incorrect answers called distractors Distractors are plausible but not possible
Multiple choice
Match your learning goals
Appropriate level of difficulty
Be aware of common errors
How to Write Good Multiple Choice
Stems
Please work in groups of 3 or 4
Do part B
There is no single correct answer; only note what you think could be improved
Worksheet: Group Activity
1. More than one possibility; change (d) to
different word formation2. Double negative is confusing3. Answers should all be of a similar length; (d)
is too long and obvious4. Too complex; time-consuming; frustrates
some test-takers; item value
Worksheet Part A: Suggestions
5. Grammar can give away answer; always use
a(n) as necessary6. Negatives should be emphasized; ie – NOT,
EXCEPT, etc.7. Too many blanks; item value; misspelled or
nonsensical/ non-existent words8. Avoid absolutes including All, None or more
than one answer
Worksheet Part A: Suggestions
Multiple Choice: Summary
What to avoid in the stem
Long complex sentences
Negatives / Double negatives
Unintentional clues
What to use in the stem
Your own words (if possible)
Single idea & clearly formulated question
Multiple Choice: Summary
What to avoid in the choices
Statements too close to being correct
Completely implausible answers
Absolute answers (ie – All of the above)
What to use in the choices
Plausible & homogeneous distractors
Same option lengths True statements that
do not answer the question
Answers distributed evenly
Composed only of statements with two
possible answers Assess familiarity with course content and
general misconceptions Test a range of broad concepts and can
quickly respond Easy to grade, but time-consuming to create
True / False
True / False
What to avoid
Negatives / Double negatives
Long complex sentences
Ambiguous or trivial material
What to use
Your own words 50/50 or 60/40 in
favor of false (students more likely to answer true)
One idea per item
Contains equal number of stems and choices
Assess recognition and recall Important if acquisition of detailed knowledge is
a learning goal
Easy to grade, but students may require more time than equal number of m/c or t/f
Matching
Matching
What to avoid
Long stems and options
Heterogeneous content (ie – testing grammar and vocabulary)
Implausible responses
What to use
Short responses; 10 to 15 items per page
Clear directions
Ordered choices (ie – alphabetical or chronological)
Q & A
Bloom, Benjamin S. Taxonomy of Educational
Objectives (1956). Published by Allyn and Bacon, Boston, MA. Copyright (c) 1984 by Pearson Education.
Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer, Richard E.; Pintrich, Paul R.; Raths, James; Wittrock, Merlin C. (2000). Anderson, Lorin W.; Krathwohl, David R., eds. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Allyn and Bacon. ISBN 978-0-8013-1903-7.
References
