1/27 cresst/university of colorado at boulder a brief history of test-based accountability lorrie a....

1/27CRESST/University of Colorado at Boulder

A Brief History ofTest-Based Accountability

Lorrie A. Shepard

CRESST ConferenceUCLA, Los Angeles, CA

January 22-23, 2007

If you choose to use this title slide, simply delete the previous slide(the one-line title version). This will be slide 1 of your presentation.

School of EducationUniversity of Colorado at Boulder


Accountability Chronology

Title I, ESEA, 1965 National Assessment of Educational Progress (NAEP),

1969 SAT Test Score Decline, 1963, 1977 Minimum Competency Testing A Nation at Risk, 1983 Excellence Movement and Basic Skills Testing, 1980s Standards-Based Reform, 1990s NCLB and the Standards Movement Today

The section titles here should be reflected in the blue section dividers throughout your presentation.


Recurring Themes (Then & Now)

Title I, ESEA 1965• Quid Pro Quo: Federal dollars spent in

exchange for evidence of program effectiveness.

• Robert Kennedy: Evaluation data to be used by parents as a “whip” or a “spur” to leverage changes in ineffective schools.

• Education seen favorably, its benefits to be extended to poor and minority children.

When pasting text from another document, do the following:1. Highlight the text you want to replace2. Go to the EDIT menu and select PASTE SPECIAL3. Select “Paste as: UNFORMATTED TEXT”


National Assessment of Educational Progress(NAEP), 1969

• Initially: an independent and neutral monitor; a census-like data system like health statistics; designed independent of political jurisdictions.

• Over time: increasingly politicized, scoring simplified and interpreted; jurisdictions identified and compared.

• The use of NAEP to leverage reform has been resisted to protect validity of the data.



SAT Test Score Decline, 1963, 1977

Along with economic downturn, Vietnam War, disillusionment of the late 1960s,…

The SAT test score decline contributed to a political climate that became hypercritical of public education.



Four Decades of Test-Driven Reform

• Minimum Competency Testing, 1970s• Excellence Movement and Basic Skills

Testing, 1980s• Standards-Based Reform, 1990s• NCLB version of Standards-Based

Reform



Reform Constants

• Tests and test results used as the primary drivers of reform.

• Tests used both to induce and to measure change.

• Policy intended to “incentivize” change with substance of change determined locally.



Reform Revisions

• Challenge and quality of test content raised (and then lowered).

• Performance levels raised (and then lowered again).

• Stakes raised, and raised again.• The need for capacity-building

recognized and pursued only by some.



Challenge of test content raised

“`Minimum Competency’

examinations (now required in 37 states)

fall short of what is needed, as the

`minimum’ tends to become the

`maximum’ thus lowering educational

standards for all.” (p. 20)

To resize an image correctly, click on it once so that its “handles” appear. While holding down SHIFT, drag one of the corner handles.This will keep the image from being distorted while resizing.


Challenge of test content raised

“Standards must reflect high expectations, not

expectations of minimal competency. To make

national standards meaningful, it is important that the Nation be able to measure progress toward

them. New forms of assessment – tests worth

teaching to – are envisioned.”

The National Council on Education Standards and

Testing. 1992.

To resize an image correctly, click on it once so that its “handles” appear. While holding down SHIFT, drag one of the corner handles.This will keep the image from being distorted while resizing.


Standards Movement Today

Competing ModelsCacophonous standardsGood and ill effects

The section titles here should be reflected in the blue section dividers throughout your presentation.


Standards Movement Today

Competing models (Elmore & Rothman, 1999)

• The “basic” standards-based reform model/High-stakes incentives model

Vs.• Expanded, capacity-building model/

Teaching and learning, cognitive science model



Competing Models: (Elmore & Rothman)How Standards Produce Higher Learning


School delivery standards

”Standards and assessments must be accompanied by policies that provide access for all students to high quality resources, including appropriate instructional materials and well-prepared teachers.”

NCEST, 1992


Opportunity to learn

“It is unfair to hold students accountable for what and how well they are learning unless they are provided with opportunity to learn.”

“Without adequate support, higher standards will further victimize students already harmed by gross inequities in the educational system.”

NAE, 1995


Cacophonous Standards

• Minimum competency standards coexist with world-class standards.

• Public doesn’t know which is which.



Cacophonous Standards

Interpreting Different “Proficiency Standards”in Comparison to National Norms

NormativePercentiles

____ 75

____ 50

____ 25

____ 10

Examples of Varying“Proficiency Standards”

National Assessment (NAEP)CSAP 4th-grade Writing

Grade LevelNationally Normed TestsCTBS, ITBS

Texas High School ExitTest (Reading)

Minimum CompetencyStandard

Historical Periods

1990sStandards-BasedReforms

1980sBasic SkillsReforms

1970sMinimumCompetencyReforms


Good and ill effects

• Accountability pressure has both increased and decreased achievement.

• Teaching the test is widespread leading to test-score inflation or increased achievement depending on the quality of the test.

• Accountability may have negative side effects, e.g., deprofessionalization, and exodus of teachers from low-scoring schools.



Figure 4 Trends in Percent Meeting Standard in Texas and Maryland:

Grade 8 Mathematics

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Year (1994 to 2001)

Perc

ent M

eetin

g St

anda

rd

TXMD


Figure 5 Trends in Percentage of Students Proficient or Above on State NAEP

Grade 8 Mathematics for Maryland and Texas (!990 through 2000)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11Year (1990 to 2000)

Perc

ent P

rofic

ient

or A

bove

MDTX


Difference Between White and Black Mean Scores inStandard Deviation Units on NAEP and TAAS

1.11.00.90.80.70.60.50.40.30.20.10.0

Effectsize

4th GradeReading

4th GradeMath

8th GradeMath

From Klein, 2000


Advice for Policymakers

Ensure the quality of assessment content.

• The best way to prevent limited learning from test-driven instruction is to build assessments that represent the full content domain, with a rich variety of assessment formats.



Advice for Policymakers

Evaluate the validity and impact of assessment programs.

• Tests are short-cut tools. Real validity studies require independent measures of student learning.

• To evaluate the effect of high-stakes testing on school improvement, motivational and moral dimensions of schooling must be studied as well as achievement.



To adjust the slide numbering, do the following:1. Go to the VIEW menu, MASTER, and select SLIDE MASTER2. In the lower right, change the number 27 to your number of slides3. Do not change the <#> character. It generates the auto-numbers.

1/27 cresst/university of colorado at boulder a brief history of test-based accountability lorrie a....

Documents