A Brief History of the NRSA Brief History of the NRS

A Brief History of the NRS

American Institutes for ResearchFebruary 2005

A Brief History of the NRSA Brief History of the NRS2

Wave 1: Development of the NRS Pre-NRS: Emergence of accountability Development process

Wave 2: Implementation Refinement of measures and procedures (1999–

2001) State implementation(2000–2003) Training and T/A on requirements, data quality,

data use Wave 3: Focus on Improvement

OverviewOverview

A Brief History of the NRSA Brief History of the NRS3

General trend toward accountability for education and employment programs (late 1980s—early 1990s)

Quality Indicators (1991–92) Required by National Literacy Act Federal (model) indicators State indicators

Emergence of AccountabilityEmergence of Accountability

A Brief History of the NRSA Brief History of the NRS4

Assessed local program performance in: program planning recruitment retention student learning professional development curriculum and instruction support services

Quality IndicatorsQuality Indicators

A Brief History of the NRSA Brief History of the NRS5

Primarily qualitative measures, focused on program process—what a program does

Less emphasis on student outcomes Lacked uniform, quantitative measures

No standards No rewards/incentives

Not fully implemented in states—voluntary

Quality Indicator ApproachQuality Indicator Approach

A Brief History of the NRSA Brief History of the NRS6

1993: Government Performance and Results Act (GPRA)—required all Federal agencies to develop indicators of program performance to demonstrate they are meeting their goals.

1995: Adult education program threatened with elimination into workforce development block grant system. Lack of data on program effectiveness cited.

1996: State directors and U.S. Department of Education develop a framework for a system of program accountability.

1997: National stakeholder meeting to validate framework and identify outcome measures.

Emergence of NRSEmergence of NRS

A Brief History of the NRSA Brief History of the NRS7

Included all state directors Formal process to identify goals of

adult education Identified measures for each goal Endorsed development of formal

accountability system, leading to first NRS development project

1997 Development Meeting1997 Development Meeting

A Brief History of the NRSA Brief History of the NRS8

Educational improvement and lifelong learning

Enhancement of employability Improve civic participation Improve literacy and education within

the family

Goals and Measures Goals and Measures IdentifiedIdentified

A Brief History of the NRSA Brief History of the NRS9

Collaborative process of DAEL, state directors, other stakeholders

Broad group identified basic program goals and measures

Advisory board refined measures and procedures

Key decisions brought to all state directors before final resolution

NRS Development Project: NRS Development Project: 1997–981997–98

A Brief History of the NRSA Brief History of the NRS10

Goals—reflect program purposes

Measures—tied to goals

Performance Standards—establish level of performance on measures

Rewards and/or Sanctions—for exceeding or falling below standards

Key Components of Key Components of Accountability SystemsAccountability Systems

A Brief History of the NRSA Brief History of the NRS11

Board included state directors, literacy volunteer agencies, experts in accountability

Concluded there were too many measures, some difficult to collect

Applied criteria for inclusion

Refinement by Advisory Refinement by Advisory BoardBoard(1997–98)(1997–98)

A Brief History of the NRSA Brief History of the NRS12

Meet WIA requirements Maintain state and local control,

flexibility to meet learner needs Minimize burden Accommodate methodology to produce

valid and reliable national data Definitions and procedures compatible

with other agencies

Criteria for Including Criteria for Including MeasuresMeasures

A Brief History of the NRSA Brief History of the NRS13

Required four types of measures—from NRS development work

Requires measures tied to state performance standards.

Rewards for states exceeding standards

Allows states sanctions—states must “consider” local performance.

WIA PassageWIA Passage

A Brief History of the NRSA Brief History of the NRS14

Include other measures that state directors had identified and considered Could not be required due to WIA Also concern on increased burden,

difficulty in collecting and defining Optional tables created, with possible

formal inclusion if widespread state use

Optional Measures and Optional Measures and TablesTables

A Brief History of the NRSA Brief History of the NRS15

Transitional year 1999 Guidelines published, national training,

state technical assistance Performance standard setting process Refinements to measures and

procedures Definitions, procedures, tables refined Data quality checklist Data quality guide and training

Wave 2: ImplementationWave 2: Implementation

A Brief History of the NRSA Brief History of the NRS16

Implement new polices and procedures: WIA reauthorization

Continue to refine NRS procedures and measures

Stronger focus on state monitoring of local programs

Use data for program administration and improvement

Wave 3: ImprovementWave 3: Improvement