Carolina Salazar17.02.2014

VALIDITY, RELIABLILITY, WASHBACK

The test measures what it is intended to measure.

VALIDITY

CONTENT VALIDITY

Content Validity

TEST

skills structures

concepts

RELEVANT!!!

Judging Content Validity

SPECIFICATION

S

ACTUAL TEST

Importance of Content Validity

Content Validity

Construct

Validity

General Validity

Content Validity

WASHBACK

CRITERION-RELATED VALIDITY

Criterion-related Validity

Test result

s

Assessment

resultsDO THEY AGREE?

Kinds of Criterion-related Validity

C-R V.

Concurrent

Test = criterion

Predictive

Test Criterion

EXAMPLES?

Examples of Criterion-related Validity

One of the objectives of a Spanish course is to measure speaking (especially pronunciation of vowels) and writing (especially comments on readings). Testing these abilities in detail could take up to 2 hours per student (too impractical), but only 20 minutes can be devoted to each student. Can a twenty-minute session give an accurate estimate of the student’s ability in these functions?

After the students finish the 20-min test, the teacher decides to choose a sample of the students taking the evaluation to subject them to the full 2-hour one. Then he compares the results.

Examples of Criterion-related Validity

A student takes a placement test for English. He receives the results and is placed in 3rd level. How well did the test predict the most appropriate class for the student? Will he succeed and pass?

SCORING VALIDITY

Scoring Validity

READING-COMPREHENSIO

N TEST

FACE VALIDITY

Face Validity

HOW TO MAKE TESTS MORE VALID?

Tips for more valid tests

1. Set specifications

2. Use direct testing

3. Relate the scoring to what you are testing

The test is repeatable.

RELIABILITY

Reliability

Reliability ExemplifiedSTUDENT

SCORE (A)

SCORE (B)

T 80 52

U 61 78

V 98 59

W 43 68

X 52 66

Y 40 98

Z 28 50

STUDENT

SCORE (A)

SCORE (B)

A 91 97

B 88 83

C 46 50

D 54 52

E 99 99

F 66 70

G 27 25

• 100-item test• SCORE A:

Monday• SCORE B:

Tuesday

The Reliability Coefficient

1. . . . .0

Role of Reliability

…THE MORE IMPORTANT THE DECISIONS, THE GREATER

RELIABILITY WE MUST DEMAND.

How do we arrive to the R.C.?

R.C.

Test-retest

Alternate forms

Split-half

SCORER RELIABILITY

Scorer Reliability

HOW TO MAKE TESTS MORE RELIABLE?

Tips for more reliable tests

1. Use enough items2. Do not give too much freedom3. Write unambiguous items4. Provide clear instructions5. Ensure that tests are legible6. Make candidates familiar with formats/techniques7. Provide non-distracting and uniform conditions8. Use items that allow objective scoring9. Provide a detailed scoring key10.Train scorers11.Agree upon acceptable responses and scores before

scoring12.Identify candidates by number13.Employ multiple, independent scoring

Reliability and Validity

A reliable test is not always valid and viceversa.

http://www.youtube.com/watch?v=5lG3R6-4T3k

WHY?

Effects of the test on students

WASHBACK

Kinds of Washback

POSITIVE WASHBACK

Encourages studentsClarifies concepts

…

NEGATIVE WASHBACK

Makes students nervous, anxious

Makes teachers/ students ignore «unimportant» topics

…http://tapchi.vnu.edu.vn/nn_4_09/b.8.pdf

Tips for achieving beneficial washback

1. Test the activities whose development you want to encourage

2. Sample widely and unpredictably3. Use direct testing4. Make testing criterion-referenced5. Base achievement tests on objectives6. Ensure the understanding of the test7. Provide assistance if necessary

Carolina Salazar17.02.2014

VALIDITY, RELIABLILITY,WASHBACK