PS397PS397Psychological Psychological

Measurement & TestingMeasurement & Testing

Educational Testing Service (ETS)Educational Testing Service (ETS)

Administers more than 11 million Administers more than 11 million tests annually in 181 countriestests annually in 181 countries

Tests include:Tests include:• SATSAT• GREGRE

For the Love of Learning:Report of the Royal Commission on Learning,

Ontario, 1995 Various forms of assessment have shown that those who

are poor, members of some minority groups, or who are female perform less well than their knowledge or skills would warrant. Some communities complain that their students have been negatively streamed because of biased assessments. For example, more than a decade ago, a York University symposium on racial and ethnic relations in city school boards was told by Marcela Duran that

“we were able to institute an experimental program, in co-operation with the Jamaican-Canadian Association, in which 100 West Indian children who had been placed in vocational schools were re-assessed, using different testing instruments. According to this process, 90 of these students were found to have been wrongly placed.”

Our students are tested to an extent that is unprecedented in American history and unparalleled anywhere in the world. Politicians and businesspeople, determined to get tough with students and teachers, have increased the pressure to raise standardized test scores. Unfortunately, the effort to do so typically comes at the expense of more meaningful forms of learning.

That disturbing conclusion emerges from Alfie Kohn's devastating new indictment of standardized testing. Drawing from the latest research, he concisely explains just how little test results really tell us and just how harmful a test-driven curriculum can be.

Alfie Kohn’s Critique of TestingAlfie Kohn’s Critique of Testing high scores often signify relatively superficial thinking a school that improves its test results may well have lowered its

standards to do so far from helping to "close the gap," the use of standardized testing

is most damaging for low-income and minority students as much as 90 percent of the variations in test scores among

schools or states have nothing to do with the quality of instruction far more meaningful measures of student learning - or school

quality - are available.

Kohn's central message is that standardized tests are "not like the weather, something to which we must resign ourselves. . . . They are not a force of nature but a force of politics-and political decisions can be questioned, challenged, and ultimately reversed."

The EQAOThe EQAO(Education Quality & Accountability Office)(Education Quality & Accountability Office)

Created in 1996 to oversee the provincial Created in 1996 to oversee the provincial testing program instituted at that timetesting program instituted at that time

All grade 3 & 6 students tested in reading, All grade 3 & 6 students tested in reading, writing & mathwriting & math

All grade 9 students tested in mathematicsAll grade 9 students tested in mathematics All grade 10 students tested in literacyAll grade 10 students tested in literacy Current budget: $50 million annually, over Current budget: $50 million annually, over

$250 million over the last five years$250 million over the last five years

Ontario English Catholic Teachers Ontario English Catholic Teachers Association, March 2002Association, March 2002

How many schools would $250 How many schools would $250 million build? How many repairs million build? How many repairs would it accomplish on existing would it accomplish on existing buildings? How many library books buildings? How many library books would it buy? Could every child have would it buy? Could every child have their own copy of their textbooks? their own copy of their textbooks? How many assistants could there be How many assistants could there be for children with special needs?for children with special needs?

Other CriticismsOther Criticisms

EQAO does not publish detailed EQAO does not publish detailed technical reports along with the technical reports along with the resultsresults

The precise statistical procedures The precise statistical procedures that were being used, the degree of that were being used, the degree of reliability of markers evidenced in reliability of markers evidenced in their scoring, and the confidence their scoring, and the confidence intervals of the scores, remain intervals of the scores, remain undisclosedundisclosed

Course ObjectivesCourse Objectives Help you understand some of the fundamental

principles underlying psychological measurement and testing.

Provide you with experience in constructing and evaluating psychological tests and measures.

Familiarize you with many of the more commonly used psychological tests.

Acquaint you with some of the important ethical and political issues relating to psychological testing.

TextText

Kaplan, R.M. & Sacuzzo, D.P. (2005). Psychological testing: Principles, applications, and issues (sixth edition). Pacific Grove, CA: Brooks/Cole.

Additional readings may be assigned for some of the classes.

Course RequirementsCourse Requirements

Brief assignments & class Brief assignments & class participation (10%)participation (10%)

Mid-term test (30%)Mid-term test (30%) Test construction project (20%)Test construction project (20%) Final exam (40%)Final exam (40%)

Lectures & Course ComponentsLectures & Course Components

The course is divided into 2 parts:The course is divided into 2 parts:• Part I (up until the mid-term)Part I (up until the mid-term)

Principles of testing, psychometrics & test-Principles of testing, psychometrics & test-related statistics, reliability, validity, test related statistics, reliability, validity, test construction techniquesconstruction techniques

• Part II (after the mid-term)Part II (after the mid-term) Applications & the ethics & politics of testingApplications & the ethics & politics of testing Review of major tests of intelligence, Review of major tests of intelligence,

vocational interest, personality, etc.vocational interest, personality, etc.

Lectures & Class NotesLectures & Class Notes

For Part I of the course, I will not be For Part I of the course, I will not be using PowerPoint to present the using PowerPoint to present the course materialcourse material

For Part II of the course, I will be For Part II of the course, I will be using PowerPoint, and the notes will using PowerPoint, and the notes will be available from my web-site be available from my web-site

Key DatesKey DatesFebruary 8 Test construction session

February 15 Mid-term test

February 19-23 Reading week

March 1 Group-constructed tests due for printing

March 8 Test administration session

March 29 Test report due

April 9 - 27 Exam period

Testing: Key TermsTesting: Key Terms Test

• a measurement device or technique used to quantify behaviour or aid in the understanding & prediction of behaviour

Item • a specific stimulus to which a person responds overtly;

this response can be scored or evaluated (for example, classified, graded on a scale or counted); the specific questions or problems that make up a test

Psychological Test • a set of items designed to measure characteristics of

human beings that pertain to behaviour (overt or observable, or covert, such as feelings, thoughts & attitudes)

Key Terms (continued)Key Terms (continued)

individual tests • given to one person at a time

group tests • administered to more than one person

at a time

Types of TestsTypes of Tests

Ability TestsAbility Tests (measure skills in (measure skills in terms of speed, accuracy)terms of speed, accuracy)• AchievementAchievement (measures previous (measures previous

learning)learning)• AptitudeAptitude (measures potential for (measures potential for

acquiring a specific skill)acquiring a specific skill)• IntelligenceIntelligence (measures potential to solve (measures potential to solve

problems, adapt to changing problems, adapt to changing circumstances, profit from experience)circumstances, profit from experience)

Types of Tests (continued)Types of Tests (continued)

Personality testsPersonality tests (measure typical (measure typical behaviour – traits, temperaments, behaviour – traits, temperaments, dispositions)dispositions)• Structured (objective)Structured (objective) – provides a self- – provides a self-

report statement to which a person report statement to which a person responds “true” or “false”, “agree” or responds “true” or “false”, “agree” or “disagree”“disagree”

• ProjectiveProjective – provides an ambiguous test – provides an ambiguous test stimulus; response requirements are stimulus; response requirements are unclearunclear

Individual DifferencesIndividual Differences

Most tests are designed to discover Most tests are designed to discover differences among individuals differences among individuals (“individual differences”)(“individual differences”)

Basic assumption of testing is that Basic assumption of testing is that differences in test results reflect real differences in test results reflect real differences in individualsdifferences in individuals

E.g., individuals who score higher on E.g., individuals who score higher on an IQ test assumed to be more an IQ test assumed to be more intelligentintelligent

A Brief History of TestingA Brief History of Testing

Chinese civil service Chinese civil service tests (1368 to 1644 tests (1368 to 1644 C.E.)C.E.)

Darwin, Galton & Darwin, Galton & individual differences individual differences (mid to late 1800’s)(mid to late 1800’s)

Binet & the Binet & the intelligence test intelligence test (1905)(1905)

Stanford Achievement Stanford Achievement Test (1923)Test (1923)

Sir Francis Galton

Alfred Binet

History of Testing (cont’d)History of Testing (cont’d)

Personality tests Personality tests (1930’s & 1940’s)(1930’s & 1940’s)

George Gallup & George Gallup & the opinion poll the opinion poll (1935)(1935)

Hermann Rorschach (1884-1922)

George Gallup

(1901-1984)