Accessing and ReportingState Student

Achievement Data for GPRA Purposes

Amy A. Germuth, Ph.D.

Compass Consulting Group, LLC

2

Overview of Session The GPRA Performance System

Overview of the Government Performance and Results Act (GPRA)

MSP GPRA Performance Indicators

Acquiring Student Achievement Data Potential issues related to using student achievement data Acquiring student achievement data

Reporting Student Achievement Data Goals and Issues

3

The GPRA Performance Indicator System

The Government Performance and Results Act of 1993 requires federally funded agencies to develop and implement an accountability system based on performance measurement.

Agencies must:

1. state clearly what they intend to accomplish,

2. identify the resources required, and

3. periodically report their progress.

4

The GPRA Performance Indicator System

Expected outcomes: Increased accountability for the expenditures of public

funds. Improved congressional decision-making through

more objective information on the effectiveness of federal programs.

Promotion of a new governmental focus on results, service delivery and customer satisfaction.

Above excerpted from: Demonstrating Results, An Introduction to Government Performance and Results Act, Spring 1999

5

Annual Performance Report

Information on program’s outcomes on GPRA indicators are collected via the Annual Performance Report (APR).

Project impacts, as assessed via individual project evaluations and use of control/comparison groups are also collected via the APR.

Together this information is used by Congress in determining future funding and thus it is critical that both are reported in as consistent a manner as possible.

6

MSP GPRA Performance Indicators

Under GPRA, the Department of Education must account for the impact of ED funded activities, usually obtained via grantees’ evaluations.

The Education Department is also required to develop performance measures that focus on what’s happening within and among programs. Thus for MSPs the following two GPRA performance indicators have been adopted:

The percentage of students who score at the basic level or above in state assessments of mathematics or science.

The percentage of students who score at the proficient level or above in state assessments of mathematics or science.

7

GPRA Data Needs From Grantees

For GPRA purposes MSP projects are asked to report student achievement data aggregated at the project level.Please note - most projects will want data aggregated at the teacher/class level and possibly student-level data linked to teachers for their own evaluation needs.

For GPRA, only the most recent year of data is reported, so new data is reported each year

This means GPRA measures are not longitudinal measures and thus do not assess change / improvement over time; instead they provide a snapshot of the MSP program at a certain time point.

8

End Overview of GPRA

Questions?

9

Issues with Use of Student Achievement Data

Available state testing data for math and science based on NCLB requirements:

Currently all states should have an annual mathematics assessment for grades 3 through 8 and one given at least once in grades 10 through 12.

By the 2007-08 school year an annual assessment in science should be given at least once in grades 3 through 5, once in grades 6 through 9, and once in grades 10 through 12.

10

Issues with Use of Student Achievement Data

Family Educational Rights and Privacy Act – 1974

Requires that schools obtain written permission from students/parents before releasing individual educational records.

Aggregate (statistical) data that contains no personally identifiable information about students does not require written permission from students/parents.

The information all MSPs will be required to submit does not fall under FERPA and thus does not require parental permission.

11

Issues with Use of Student Achievement Data

Anonymity vs. confidentiality

Anonymity – person’s identity not known to data user Confidentiality – person’s identity known to data

user but when publicly reported no identifiers included

The information all MSPs will be required to submit does not require that the data users know the teacher’s identity, thus anonymity is maintained.

12

Data To Request On All Participating Teachers

1. Total number students taught in main subject (math or science) summed across all classes and grade levels.

2. Total number of students of participating teacher who were tested on state standardized test in math/science (summed across classes/grades)

3. Total number of students of participating teacher scoring at the basic level or above (summed across classes/grades)

4. Total number of students of participating teacher scoring at proficient or above (summed across classes/grades)

NOTE: Math and science should be reported separately!

13

Example Mr. Jones is a participating teacher and teaches the following:

7th grade math: 30 students, all tested, 28 at or above basic, 24 at or above proficient

8th grade math: 31 students, 29 tested, 21 at or above basic, 17 at or above proficient

8th grade elective - science and civilization: 32 students

The following numbers would be calculated for Mr. Jones (and similarly all participating teachers):Number taught: 30+31 = 61Number tested: 30+29 = 59Number at or above basic: 28+21 = 49Number at or above proficient: 24+17= 41

These numbers would be summed across all participating teachers, schools and districts to represent project-level data, which is what will be reported on the APR.

14

Notes on Data Requests

There are various ways in which the districts might provide this data to grantees, and thus how grantees would need to transform that data into what needs to be reported. In some cases districts are only able to report actual scores. In

these instances grantees will need to convert scores to performance levels.

In other cases districts might provide the grantee with data aggregated to the classroom level. In these instances grantees will have to do the aggregation to provide project-level data.

15

Removing Roadblocks

It is important to know: what you need to ask for, in what form you should request it, whom to ask for it, and when to ask for it.

Always keep in mind that obtainment of data for MSP GPRA indicators is both legal and required.

16

Acquiring Student Data

Identify who has access to the data needed (should be someone in district).

Inform stakeholders and require maintenance of anonymity and confidentiality of data.

For GPRA purposes only assessment data that cannot be linked to individual teachers or students is being requested. (However, for individual project evaluations this data may need to be linked…)

17

Acquiring Student Data Find out district requirements for requesting data,

including: Whom specifically to request data from Timelines for requesting data Form(s) in which data are available When data may no longer be available!

Want aggregated data in electronic form not linked to individual teachers Realize this may require a long lead time so request data

early Ensure cooperation during proposal design phase May want to create a MOU to ensure access to data

18

Selecting Teachers On Which To Aggregate Data

Create a list of all teachers who participated in MSP at any time, regardless of when they participated.

May want to also request same data for a comparison group of matched teachers who did not participate in the MSP project.

Request data without IDs.

Sum data across teachers, classes, schools, and districts to report project-level data

19

Notes on Selecting Teachers

In some instances districts have had to have lists of students’ names in order to access their achievement data as it wasn’t available by teacher.

20

End Acquiring Student Achievement Data

What have been your experiences trying to obtain student achievement data?

What roadblocks have you faced?

How have you overcome them?

21

Reporting Student Achievement Data

Goal: To report uniform student achievement data for all participating MSP teachers in the project.

Reporting Issues Data availability Dealing with missing data

- report on all teachers even if test data missing or not available

- make note of number of teachers and students for whom data is missing

22

What Should be Reported

Information aggregated to the project level, including: Number of districts Number of schools Number of MSP teachers Number of students of those MSP teachers Number of students with student assessment data in

the relevant subject (math or science) Number of students who scored at basic or above Number of students who scored at proficient or above

23

Wrap-Up

What we covered: Understanding GPRA, PART, and the APR and how they

relate to each other. Reviewing the MSP performance indicators and what

data must be reported. Issues around obtaining student achievement data:

Knowing in what form to request the data Knowing from whom, when, and how to get student achievement

data

How to report student achievement results

24

Wrap-Up Continued

Questions?