test construction edited

TEST CONSTRUCTION

Presented by:Arnel O. Rivera

http://www.slideshare.net/ArnelSSI

The Art of Effective EvaluationPresented to the

Faculty and Students of UPHSD – Molino Campus

EMPTY YOUR CUP

An Old Buddhist Tale

Bruce Lee, the legendary martial artist, once told a story about a highly-educated man who went to a Zen teacher to ask and acquire knowledge about Zen.

As the Zen teacher began to explain things, the man would frequently interrupt with remarks like, "Oh, yes, we have that too…”

Finally the Zen teacher stopped talking. He began to serve tea. He poured tea into the man’s cup.

He poured tea into the cup until it was full, and then kept pouring until the cup overflowed. "Enough!" the man once more interrupted, "No more can go into the cup!"

“Indeed, I see," answered the Zen teacher, "If you do not first empty the cup, how can you taste my cup of tea?"

A full cup cannot accept anything more. Similarly, a person who believes that he had learnt a lot cannot learn anything else & will stagnate quickly and not move to higher levels.

WWW.SIMILIMA.COM

ATTITUDE DETERMINES YOUR ALTITUDE. A SMALL CHANGE IN ATTITUDE MAKES A BIG DIFFERENCE

Why is there a need to study TEST CONSTRUCTION?

13% of students who got low grades in exams are caused by faulty test questions.

WORLDWATCHThe Philadelphia Trumpet

August 2005

It is estimated that 90% of all test questions asked in the US are of

“Low level” - knowledge and comprehension.

(Wilen, W.W., 1992)

“Low level” doesn’t mean easy:Write an essay explaining the decline

and fall of the Roman Empire incorporating at least five of the seven causes discussed in class from the writings of Gibbon and Toynbee

“High level” doesn’t mean hard:Which movie did you like more, WALL-

E or Cars? Why?

Sample Test Question:

Outline: Principles in Test

Construction Steps in

Preparing Test Questions

Preparing Multiple Choice Questions

Preparing True or False Questions

Preparing Matching Type Questions

Preparing Sentence Completion Questions

Preparing Essay Questions

Other types of Test Questions

Wrap-up/Things to Remember

“The evaluation of pupils’ progress is a major aspect of the teacher’s job.“

Evaluating Educational Outcomes

(Oriondo & Antonio)

Explain the message of the comic strip.

Our students were trained to memorize information.

After they have memorized facts, then what?

Old belief about teaching .....

Education is a process where the notes of the teachers or the contents in the books are transfered to the notebooks of the students, without understanding them. (Wiggins, 2009)

In your lesson plans, did you ever bother to ask this question:

• How do you know if you have achieved your desired result?

“A test is a sample of behavior, products, answers, or performances from a particular domain” (Carrington, 1994)

“… it's a systematic method of eliciting performance which is intended to be the basis for some sort of decision making" (Hughes, 1989).

“A test will predict performance levels, and the learner will somehow reconstruct its parts in meaningful situations when necessary” (McCann, 2000)

“ Testing is generally concerned with turning performance into numbers.” (Baxter, 1997)

Guidelines for Test Construction

What is testing?

Inform learners and teachers of the strengths and

Motivate learners to review or consolidate specific material

Guide the planning/development of the ongoing teaching

Determine if the objectives

Encourage improvement

Create a sense of accomplishment

Guidelines for Test Construction

What are tests for?

weaknesses of the process

process

have been achieved

Encourage

improvement!

BARRIERS IN TEST CONSTRUCTION

Ms. Alanganin –

Mr. Highfalutin –

Ms. Madaldal –

Ms. Magulo –

Ms. Malabo –

Mr. Pulpol –

Ms. Foringer –

Ms. Colonial Mentality –

confusing statementsdifficult vocabularyexcessive wordinesscomplex sentence

structureunclear instructionsunclear illustrative

materialslinguistically bound

culturally bound words

Characteristics of Good Tests

Validity – the extent to which the test measures what it intends to measure

Reliability – the consistency with which a test measures what it is supposed to measure

Usability – the test can be administered with ease, clarity and uniformity

Scorability – easy to score

Interpretability – test results can be properly interpreted and is a major basis in making sound educational decisions

Economical – the test can be reused without compromising the validity and reliability

Other Things to Consider

“To be able to prepare a good test, one has to have a mastery of the subject matter, knowledge of the pupils to be tested, skill in verbal expression and the use of the different test format”

Evaluating Educational Outcomes(Oriondo & Antonio)

1. Multiple Choice

2. True or False

3. Matching Type

4. Fill-in the blanks (Sentence Completion)

5. Essay

5 Most Commonly used Test Format

Source: Turn-out of Test Questions in SSI (2003-2007)

General Steps in Test Construction

TEST ANALYZE

SUBMISSION

PRODUCE A T.O.S.

Table of Specifications (TOS)

A two way chart that relates the learning outcomes to the course contentIt enables the teacher to prepare a test containing a representative sample of student behavior in each of the areas tested.

Tips in Preparing the Table of Specifications (TOS)

Don’t make it overly detailed.

It's best to identify major ideas and skills rather than specific details.

Use a cognitive taxonomy that is most appropriate to your discipline, including non-specific skills like communication skills or graphic skills or computational skills if such are important to your evaluation of the answer.

Weigh the appropriateness of the distribution of checks against the students' level, the importance of the test, the amount of time available.

MATCH the question level appropriate to the level of thinking skills

Examples of Student Activities and Verbs for Bloom’s Cognitive Levels (Jacobs & Chase, 1992:19)

Bloom’s Cognitive Level

Student Activity Words to Use in Item Stem

Knowledge Remembering facts, terms, concepts, definitions, principles

Define, list, state, identify, label, name, who?, when?, where?, what?

Comprehension Explaining/interpreting the meaning of material

Explain, predict, interpret, infer, summarize, convert, translate, account for, give example, paraphrase

Application Using a concept or principle to solve a problem

Apply, solve, show, make use of, modify, demonstrate, compute

Examples of Student Activities and Verbs for Bloom’s Cognitive Levels (Jacobs & Chase, 1992:19)Bloom’s Cognitive

LevelStudent Activity Words to Use in Item

Analysis Breaking material down into its component parts to see interrelationships/ hierarchy of ideas

Differentiate, compare/contrast, distinguish ____from ____, how does ____relate to ___, why does ____work

Synthesis Producing something new or original from component parts

Design, construct, develop, formulate, imagine, create, change, write a poem or short story

Evaluation Making a judgment based on a pre-established set of criteria

Appraise, evaluate, justify, judge, which would be better?

The following array shows the most common questions types used at various cognitive levels.

Factual Knowledge Application Analysis and Evaluation

Multiple ChoiceTrue/FalseMatching TypeS. CompletionShort Answer/RRT

Multiple ChoiceShort AnswerProblemsEssay

Multiple Choice

General Rules in Writing Test Questions

Number test questions continuously.Keep your test question in each test group uniform.Make your layout presentable.Do not put too many test questions in one test group. T or F: 10 – 15 questions Multiple Choice: max. of 30 questions Matching type: 5 questions per test group Others: 5 – 10 questions

Some additional guidelines to consider when writing items are described below:

1. Avoid humorous items. Classroom testing is very important and humorous items may cause students to either not take the exam seriously, become confused or anxious.

2. Items should measure one’s knowledge of the item context not their level of interest.

3. Write items to measure what students know, not what they do not know. (Cohen & Wallack)

Multiple Choice Test

When checking the stems for correctness:

Ensure that the stem asks a clear question.

Reading level is appropriate to the students

The stem is grammatically correct.

Negatively stated stems are discouraged.

What to Look for on Multiple Choice Tests

Example:

What is the effect of releasing a ball in positive gravity?

a) It will fall “down.” correct

b) It will retain its mass. true but unrelated

c) It will rise. false but related

d) Its shape will change. false and unrelated

Anatomy of a PerfectMultiple Choice Tests

Multiple Choice Questions1. Use negatively stated stems sparingly and

when using negatives such as NOT, underline or bold the print.

2. Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer.

3. Only one option should be correct or clearly best.

Multiple Choice Questions:

4. All options should be homogenous and nearly equal in length.

5. The stem (question) should contain only one main idea.

6. Keep all options either singular or plural.

7. Have four or five responses per stem (question).

Multiple Choice Questions:

7. When using incomplete statements place the blank space at the end.

8. When possible organize the responses.9. Reduce wordiness.10.When writing distracters, think of

incorrect responses that students might make.

Examples1. Sheldon developed a highly controversial

theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work?a. He was influenced too much by the Freudian psychoanalysis.b. His rating of physique and temperament were not independent.c. He failed to use empirical approach.d. His research sample was improperly selected.

Examples

Better: (Eliminate excessive wording and irrelevant information)

1. Which of the following is a criticism of Sheldon's theory of personality?

Examples

1. The receptors for the vestibular sense are locateda. in the fovea.b. in the brain.c. in the middle ear.d. in the inner ear.

Examples

Better: (Include in the stem any word(s) that might otherwise be repeated in each option.)

1. The receptors for the vestibular senses are located in the _______.a. foveab. brainc. middle eard. inner ear

True or False

Each statement is clearly true or clearly false.Trivial details should not make a statement false.Statements are written concisely without more elaboration than necessary.Statements are NOT quoted exactly from text.

What to Look for onTrue/False Tests

Give emphasis on the use of quantitative terms than qualitative terms.Avoid using of specific determiners which usually gives a clue to the answer. False = all, always, never, every,

none, only True = generally, sometimes, usually,

maybe, oftenDiscourage the use of negative statements.Whenever a controversial statement is used, the authority should be quoted.Discourage the use of pattern for answers.

Tips in Making True/False Tests

Examples:

____ 1. Repetition always strengthens the tendency for a response to occur.

(Using "always" usually means the answer is false.)

Find the errors, and/or problems with the following true-false tests.

Examples:

_____ 2. The process of extinction is seldom immediate but extends over a number of trials.

(Words like "seldom" usually indicate a true statement.)

Examples:_____ 3. The mean, median, and mode are

measures of central tendency, whereas the standard deviation and range are measures of variability.

(Express a single idea in each statement.) e.g.“The mean and standard deviation are measures of central tendency.”

Matching Type

Parts of the Matching Type Test(Vertical Type)

Column A

(Premise)

Column B

(Response)

Parts of the Matching Type Test(Horizontal Type)

(Response)

(Premise)

The list of responses should be relatively short.Response options should be arranged alphabetically or numerically.Directions clearly indicate the basis for matching.

Can responses be used more than once?

Where will you place your answer? Can students infer relationships or

are they based on real word logic?

What to Look for on Matching Type Tests

Position of matches should be varied. Avoid using patterns.

The choices of each matching set should be on one page

There are more responses than premises in a single set if responses cannot be used more than once.

The premises are homogeneous as well as the responses and are grouped as one item. Example:

Set A: Provinces in Region I Set B: Provinces in CAR

If responses can be used more than once, it should be proportional to the number of premises (3:5 or 4:10)

Examples:

Directions: Match the following.1. Food A. Primary reinforcer2. Psychoanalysis B. Sigmund Freud3. B.F. Skinner C. Operant conditioning4. Standard deviation D. Measure of

variability5. Schizophrenia E. Hallucinations

Examples:Better: (Use homogenous material in matching items, and if responses are not to be used more than once, include more responses than stimuli.)Match the theories in Column A with their proponents in Column B. Write the letter of the correct answer.

Column A Column B___ 1. Psychodynamic Theory A. Albert Bandura___ 2. Trait Theory B. B.F. Skinner___ 3. Behaviorism C. Carl Rogers___ 4. Humanism D. Gordon Allport___ 5. Social Learning Theory E. Karn Horney

F. Raymond CattellG. Sigmund Freud

Sentence Completion / Fill-in the Blanks

Only significant words are omitted.

When omitting words, enough clues are left so that the student who knows the correct answer can supply the correct response.

Ensure that grammatical clues are avoided.

What to Look for on Sentence Completion Tests

Blanks are at the end of the statement.

The length of the responses are limited to single words or short phrases.

Questions are not lifted as verbatim quotes from text.

What to Look for on Sentence Completion Tests

Examples:

1. An animal with six legs is called _________.

The item is so indefinite.It can be completed with answers such as bee, mosquito or any other insect

Better:

1. Animals with six legs are called ___________.

Examples:

1.The __________ is the answer in _____.

Too many key words are omitted. Lines are not in equal length.

Better:

1. The product is the answers in _________.

Examples:

1. If a mango weighs 250 grams, 10 mangoes would weigh ______.

There are two possible answers – 250 grams and .25 kilos.

Better:1. If a mango weighs 250 grams, 10 mangoes would weigh ____ grams.

Essay / Short Answer Test

Types of Essay Items:

Extended response type The test may be answered by the examinee in whatever manner he wants Example: Do you think teachers

should be allowed to work abroad as domestic helpers? Explain your answer.

Two Types of Essay Items:Restricted response type

The test limits the examinees response may be answered by the examinee’s responses in terms of length, content, style or organization. Example: Give and explain three

reasons why the government should or should not allow teachers to work abroad as domestic helpers.

The task is clearly defined. The students are given an idea on the scope and direction you intended for the answer to take. The question starts with a description of the required behavior to put them in the correct mind frame.

E.g. “Compare” or “Analyze”

What to Look for on Essay Tests

The questions are written in the linguistic level appropriate to the students.

Questions require a student to demonstrate command of background information, not simply repeating information.

Questions regarding a student’s opinion on a certain issue should focus not on the opinion but on the way it is presented and argued.

A larger number of shorter, more specific questions are better, than, one or two longer questions.

Proposed Criteria in Grading Essay Test

Ideas (20%)

Weight of Evidence Presented (40%)

Correct Usage (20%)

Logical Conclusions drawn from the evidence (20%)

Example:What is wrong with this question?

Describe asthma?

Better: (Clearly explain what is expected of the student.)

Describe asthma. Include in your answer :a. the pathophysiologic features of asthmab. the clinical manifestations associated

with an asthma episodec. the management of an asthma episode.

(10 points)

Example:What is wrong with this question?Who is better, Rizal or Bonifacio?

Better: ( The students are given an idea on the scope and direction you intended for the answer to take.)Compare and contrast the method used by Rizal and Bonifacio in promoting nationalism. (5 points)

test construction edited

test measures

types of test

order test

test constructionms

test construction draft

sample test question

used test format1

causedby faulty test

Education

test construction villa

test construction manual

test construction 2

police test edited

overview of test construction

test construction day 2

criteria on test construction

test construction tony coloma

module 6 test construction &evaluation. lesson’s focus...

test construction: the elements

august test f4 2010 edited

construction edited new 27-03-2013

compiler construction by: muhammad nadeem edited by: m....

test well edited malfunction3

final test av2 - 2011 edited

testing and test construction

test construction and interpretation

cy1400 11 april edited test

test construction

test well edited malfunction2