measuring what matters: creating a longitudinal data system to improve student achievement

8/14/2019 Measuring What Matters: Creating a Longitudinal Data System To Improve Student Achievement

1/20

Using Data To Improve Student Achievement

Creating a LongitudinalData System

American Diploma Project Network

in partnership with


2/20

Does your state collect the data you need toanswer these questions?

How many students drop out or are otherwiseunaccounted for after 8th grade?

Which schools produce the strongest academicgrowth for their students?

What achievement levels in middle school indicathat a student is on track to succeed in rigorouscourses in high school?

What high school performance indicators (e.g.,enrollment in rigorous courses or performance onstate tests) are the best predictors of studentssuccess in college or the workplace?

What percentage of high school graduates who gon to college take remedial courses?


3/20

A rapidly changing global economy and concerns

about our ability to create a competitive workforcehave focused national attention on the educationsystems of Americas states, highlighting their criticalrole in ensuring a prosperous future for our country.As a result, American schools are expected to perform better than ever before in preparing all students to meetrigorous educational requirements for postsecondaryeducation and the workplace.

Educators and policymakers are beginning to recog-

nize the value of better information as an essential toolfor improving schools. They understand that whenstates collect the most relevant data and are able tomatch individual student records over time, they cananswer the questions that are at the core of educationaleffectiveness. Longitudinal data (data gathered on thesame student from year to year) make it possible to:

follow students academic progress as they movefrom grade to grade;

determine the value-added and efficiencies of specific schools and programs;

identify consistently high-performing schools sothat educators and the public can learn from bestpractices;

evaluate the effect of teacher preparation andtraining programs on student achievement; and

focus school systems on preparing a higher percent-age of students to succeed in rigorous high schoolcourses, college and challenging jobs.

Policymakers and educators need longitudinaldata systems capable of providing timely, valid andrelevant data. Access to these data gives teachersthe information they need to tailor instruction tohelp each student improve, gives administrators theresources and information to effectively and efficiently

manage, and enables policymakers to evaluate which

policy initiatives show the best evidence of increasingstudent achievement.

Essential Elements of a State Longitudinal Data System

Although each states education system is unique, 10elements are essential in a longitudinal data system:

1. A unique statewide student identifier.

2. Student-level enrollment, demographic andprogram participation information.

3. The ability to match individual students testrecords from year to year to measure academicgrowth.

4. Information on untested students.

5. A teacher identifier system with the ability tomatch teachers to students.

6. Student-level transcript information, including

information on courses completed and gradesearned.

7. Student-level college readiness test scores.

8. Student-level graduation and dropout data.

9. The ability to match student records between thepre-K12 and postsecondary systems.

10. A state data audit system assessing data quality,

validity and reliability.In building a statewide data system with each of these components, the state should ensure that studentrecords can be easily transferred, student privacy is pro-tected, data definitions and requirements are clear to allconcerned, and the data system is organized in waysthat facilitate data use and user-friendly reporting.

Introduction


4/20

A unique statewide student identifier is a single, non-duplicated number that is assigned to and remainswith a student throughout his or her pre-K12 career.A student who leaves the state and returns should beassigned his or her original number.

A student identifier will allow the state to follow theprogress of each student over time, from prekinder-garten though grade 12, and across campuses or dis-tricts within the state while ensuring his or her privacy.It also makes it possible to identify information abouta single student across various data sources (e.g.,enrollment, program participation, demographicsand assessment) to evaluate the relationship betweenprogram participation and performance and to study

student mobility patterns and evaluate the effect of mobility on performance.

A statewide student identifier can help policymak-

ers and educators know, among other things:

The academic value-added of a school or program.

The achievement levels in early grades that indicate

that a student is on track to succeed in subsequentgrades.

The test scores in early grades that should bethresholds for intervention.

The 10 Essential Elements in Detail

D

mU

D

m

A

m

1 A Unique Statewide Student Identifier 36 states report having this element1

Accurate information on student enrollment, demo-graphics and program participation (e.g., studentparticipation in special education or the free andreduced-price lunch program, which is the mostcommon indicator of student poverty status) is essen-tial to evaluate the effects of schools and programs on

student achievement and to assess the impact of stu-dent mobility and continuous enrollment on learning.

Correct student demographic and program participationinformation also is critical for the accurate disaggrega-tion of test scores, assuming that a student identifier canconnect these data to test results. For example, test scores

2 Student-Level Enrollment, Demographic andParticipation Information 38 states report having this element

Action Steps for Policymakers*

Every student in the state is assigned a unique student ideis consistent throughout his or her public education in the

1 In August 2005, the National Center for Educational Accountability, with support fromthe Bill & Melinda Gates Foundation and The Broad Foundation, administered a survey of all 50 states and the District of Columbia to learn how many states already have the 10essential elements in place. A similar survey was conducted in 2003 and 2004. Forty-eight

states and the District of Columbia responded to the survey, and results can be viewed atwww.DataQualityCampaign.org.

*See Appendix for examples of State Education Agency Actions associated with eachessential element.


5/20

cannot be correctly disaggregated for special education,

English language learner, or free and reduced-pricelunch students unless accurate information on thesestudents is connected to the test database.

Attendance data also can be an important indicator of student motivation and the likelihood that studentswill drop out. Collecting information on the percent-age of days that students attend school (and specificclasses if a state collects attendance at that level) in agiven six- or nine-week period and matching those

data to results on subsequent statewide assessmentsor end-of-course exams can help identify studentswho fail those exams even though their attendancerates are high.

With student-level enrollment, demographic and

program participation information, policymakers

and educators will know:

The extent to which free and reduced-price lunchenrollment drops off in high school, and how thatmight affect measures of each high schools povertyrate.

How the percentage of minority students in gifted

and talented programs compares with that of whitestudents.

The rate at which English language learners areentering the state for the first time in high school,and how are they doing on the states high schoolexams.

A statewide database of individual student results onstate exams and state-mandated local exams should be maintained with the ability to disaggregate theresults by individual item and objective. This willallow the state to provide good diagnostic informationto teachers. Though most states do have annual testrecords for individual students, only some of these

states have created the ability to match records forindividual students across time and with otherdatabases (e.g., enrollment, course completion andgraduation databases).

Combining a student-level statewide test databasewith the ability to match individual student recordsover time allows states to monitor student academic

3 The Ability To Match Individual StudentsTest Rfrom Year to Year To Measure Academic Growth 32 states report having this element

Action Steps for Policymakers

Enrollment, demographic and program participation data arecollected and permanently stored at the state education agencyso they can be matched across years. At a minimum, these data arcollected at least one time during the year other than when thestate assessment is administered. Ideally, states will collect thisinformation continually or at multiple times during the year.

Attendance data are collected and permanently stored at the stateeducation agency. At a minimum, these data are collected over sixor nine-week increments.


6/20

growth and provide valuable diagnostic information

from the students academic history to teachers andprincipals. Having this information stored in astatewide database makes it easier for the informationto follow the student as he or she changes grades,schools or districts.

With the ability to match individual students test

scores from year to year, policymakers and educators

will know (by grade and subject):

The percentage of students who were below profi-cient last year but who met the states proficiencystandard this year.

Whether or not proficient and advanced students

are achieving at least a years growth every year.

D

mU

D

m

A

m


Student-level test data are collected and permanently sto

The state develops a method (such as the use of a statewiidentifier) to match individual test records across years.

The test database is made available for research and progevaluation activities (e.g., measure of year-to-year studenacademic growth).


Student-level records for untested students are maintainecontain information about which subject the student wasin and why.

Too often students fall through the cracks in educa-tional systems. To address this problem, the No ChildLeft Behind Act of 2001 requires states to keep track of the number and percentage of students who do nottake the state tests. However, states need to go onestep further to find out why these students are nottested and then match their records to separate enroll-ment and program participation databases. Thismakes it possible to identify patterns associated with

specific student populations (e.g., special educationstudents or English language learners) and ensure thatall students are held to high expectations.

With information on untested students, policymakers


Which students were not tested by grade andsubject and why.

Whether there are trends over time in the numberand percentage of untested students from each stu-dent group (e.g., English language learners, specialeducation students, different ethnic groups).

Whether or not particular schools and districts haveexcessive absences on test day or questionable pat-terns of absences and exemptions across years(these measures can be used in a states data auditsystem to ensure data quality).

4 Information on Untested Students 25 states report having this element


7/20

Many states collect data on teacher education andcertification, but matching teachers to students byclassroom and subject is critical to understanding theconnection between teacher training and qualifica-tions and student academic growth. As with students,teachers should be given a unique statewide identifierthat follows them over time. This makes it possible todetermine which students and which courses are being taught by teachers with different levels andtypes of preparation or certification and which formsof teacher training and certification have the greatesteffect on students academic growth in the classroom.Such a match makes it possible to evaluate the effec-tiveness of teacher preparation programs, including

traditional and alternative certification programs, based on students academic progress. Combining thisinformation with student demographic informationalso allows states to determine the experience levelof the teachers teaching low-income or special needsstudents.

With a teacher identifier and the ability to connect

teacher and student data, policymakers and educa-tors will know:

The teacher preparation programs that produce

graduates whose students have the strongestacademic growth.

How the experience levels of the teachers in the dis-tricts high-poverty schools compare with those of teachers in the schools serving affluent students, andhow these experience levels are related to the aca-demic growth of the students in their classrooms.

The relationship between the performance of thedistricts low-income students on the state mathexam and teacher preparation in that subject.


Every public education teacher in the state is assigned a uniqueteacher identifier that is consistent throughout his or her teachingcareer in the state.

Teacher preparation information is maintained by and/or sharedwith the state education agency.

Teacher data can be matched to data on students in each of theteachersclasses.

5 A Teacher Identifier System with the Ability To Teachers to Students 13 states report having this element


8/20

Many states are encouraging students, particularlylow-income and minority students, to take rigorouscourses in high school so that they are better preparedfor success in postsecondary education and the jobmarket. In most states, however, course-taking data

are not collected at the state level, making it impossible to monitor the effect of these policies. To fill in themissing information, states should collect student-level transcript information from middle and highschools, including courses taken and grades earned.These data can be aggregated to the school and dis-trict levels to track patterns in overall student course-taking and the relationship between those courses andstudent readiness for college and work.

Because titles for the same courses may not be consis-tent, and some titles may not match the materialtaught in the classroom, states should explore creatinga statewide course classification system (numberingand course description/naming). This system should be based on the state standards that are covered in thecourse. For example, the title Algebra I could beapplied to courses designed to cover certain content;courses that cover only part of this content wouldreceive a different title.

To make sure that students are actually learning thecontent implied by the course titles (e.g., studentsactually learn algebra in courses titled Algebra I),information on students course completion andgrades should be linked to other measures of learning,

such as scores on state end-of-course tests and collegereadiness exams, and to data on students need forremediation in college.

With student-level transcript information, including

courses completed and grades earned, policymakers


The number and percentage of students who areenrolling in and completing rigorous courses inhigh school, disaggregated by ethnicity and incomestatus.

The middle schools that are doing the best job of preparing students for rigorous courses in highschool.

Whether students in more rigorous courses in highschool have been more successful in college or inthe workplace.

Whether there is evidence of grade inflation (e.g.,students with the same test scores receiving dra-matically higher grades in the same course in cer-tain schools or districts).

D

mU

D

m

A

m

6 Student-Level Transcript Information, IncludInformation on Courses Completed and Grad 7 states report having this element


A standardized course classification (numbering and coution/naming) system is developed and used by the state.

Course completion records and grades for all courses takmiddle and high school are collected for all students.


9/20

To ensure that students make a successful transitionfrom high school to postsecondary education, it isimportant for states to collect and report student per-formance data on college admissions, placement andreadiness tests. Student performance on SAT, SAT II,

ACT, Advanced Placement (AP) and InternationalBaccalaureate (IB) exams is an important indicator of students college readiness; states should collect andreport these data annually. Currently, only sevenstates maintain this information from year to year atthe student level.

Some states are even building college readiness testsinto their statewide assessment systems so that allstudents not just a self-selected group are tested

on these skills. This will provide an even clearer pic-ture of how well schools are preparing students forpostsecondary education.

Matching students college readiness test informationfrom high school with the same students test scoresin middle school makes it possible to analyze theeffectiveness of high schools for students with varyinglevels of preparation.

With student-level college readiness test scores,policymakers and educators will know:

How participation rates and scores on SAT, ACT,AP and IB exams change over time for low-incomeand minority students.

The percentage of students who met the proficiencystandard on the state 8th grade test who also takeAP or IB courses in high school and pass the corre-sponding exams.

The percentage of low-income students who metthe proficiency standard on the state high schooltest who also take the SAT and ACT exams andearn a score that indicates college readiness on

those exams.

7 Student-Level College Readiness Test Scores 7 states report having this element


Student-level college-ready assessment data, includingSAT, SAT II,ACT, AP, IB and end-of-course test data, are collected andmaintained for all students who take the tests.


10/20

A majority of states currently collect annual recordson individual graduates and dropouts. But a newNational Governors Association (NGA) compactsigned by 48 states aims to create a more valid,reliable and consistent graduation rate that tracks

students from 9th to 12th grade. Based on NationalCenter for Educational Accountability analyses, only14 states currently can calculate the graduation ratedefined in the NGA compact. 1

The calculation of accurate graduation rates alsorequires being able to accurately account for whathappens to students who leave public education. Forexample, states must be able to distinguish betweenstudents who drop out or get a GED and students

who transfer to another school.States can engage in the following activities tomaintain accurate records on students who are not

accounted for:

Maintain a statewide enrollment database.

Match student records over time.

Merge this information with student records oncompleted diplomas and GEDs.

Establish a statewide coding system for the variousreasons why students leave a school and clearguidelines on how those codes should be applied.

Provide training and hold school districts account-able for using the statewide coding system.

Spot-check or audit districts that are likely to haveproblems.

With accurate graduation and dropout data in placeand the ability to match records to other databases,

policymakers and educators will know:

When and why students leave the states publiceducation system.

The percentage of first-time 9th graders in a givenyear who graduate from high school within four,five or six years.

The schools and school systems that are doing the best job of reducing the dropout rate.

The characteristics of high school dropouts andwhether or not there are early warning signs thatschools can look for in elementary and middleschool.

D

mU

D

m

A

m

8 Student-Level Graduation and Dropout Data 34 states report having this element


Graduation and dropout data are collected at the student all students in grades 712.

A standardized system for identifying why a student leavpublic education system (e.g., moves to another district, schooled or leaves the state) is developed and used.

Consequences are applied to school districts that do a poaccounting for missing students.1 In early 2005, t he National Governors Association (NGA) convened a task force charged

with helping states develop a high-quality, comparable high school graduation measure.The result was a compact signed by 48 governors to develop a standard, four-year,adjusted-cohort graduation rate. To calculate the agreed-upon rate, states need a way totrack students over time. According to responses to the National Center for EducationalAccountability survey of state longitudinal data systems, only 14 states have the necessaryelements (numbers 1, 2, 8 and 10) in place to calculate the graduation rate defined by theNGA graduation compact.


11/20

As states and school systems work to align expecta-tions in high school with the demands of postsec-ondary education, they need better data on studentsuccess when they leave the pre-K12 system andenter college. Most states today do not have data

systems that enable this two-way communication.There are often two separate data systems, and theyrarely can exchange information.

States must move toward a more integrated system.Postsecondary institutions should provide annualfeedback reports to individual high schools on thesuccess of their graduates in their first year of college-credit coursework. These data, reported in aggregateto protect student privacy, would give high schools

valuable information for improving the rigor and effec-tiveness of high school curricula and instruction.

It is advantageous for the pre-K12 and public post-secondary systems to agree on a common data set toinclude in the high school transcript state assessmentdata that may be used in the placement process. Theapplication to public colleges should include a requestfor the unique statewide student identifier so that stu-dent records can be electronically exchanged among

high schools, community colleges and four-year col-leges. Although federal privacy laws place somerestrictions on the exchange of individual records,they do not prohibit states from sharing studentrecords. Several states have worked out ways to makethis exchange possible while protecting student privacyand remaining within the bounds of federal law.

Just as pre-K12 information needs to be collectedin a central database so that it does not have to berequested from each district, so too should analysts beable to retrieve student-level postsecondary recordscentrally without having to request those records from

each institution.

With the ability to match student records between

pre-K12 and postsecondary systems, policymakers


The percentage of each districts high school gradu-ates who enrolled in college within 15 months aftergraduation.

The percentage of last years graduates from each

high school or school district who needed remedia-tion in college, and how this percentage varied bystudent income and ethnicity.

The percentage of students who met the proficiencystandard on the state high school test and stillneeded remediation in the same subject in college.

How students ability to stay in and complete col-lege is related to their high school courses, gradesand test scores.

9 The Ability To Match Student Records between Pre-K12 and Postsecondary Systems 12 states report having this element


A mechanism for sharing data between the pre-K12 and postsec-ondary systems is developed and used.

The state investigates the possibility of developing and maintainina central repository for all of these data for all the states publicpostsecondary institutions.


12/20

The decisions made in education are only as goodas the data on which they are based. A poorly organ-ized data management system, limited staff andunclear rules for data entry all lead to less-than-validinformation. Invalid or careless reporting by some

schools and districts is a problem in a number of states one that is likely to continue in the absenceof checks on the quality and accuracy of the datasubmitted by schools and districts. Without a well-designed and well-implemented state data auditsystem, the public cannot have confidence in thequality of the information coming out of the statespublic education system.

At the outset, states need to ensure that the data ele-

ments they request are clearly and unambiguouslydefined, as are any rules or interpretations concerningthe entry or reporting of those data. Similarly, statesshould provide materials or professional develop-ment opportunities for school and/or district staff to ensure that they understand state rules, regula-tions, definitions and protocols. States also need awell-developed system to identify data submitted by school districts that are likely to be in error, tospot-check other information on a random basis andto conduct site visits as necessary to audit the validityof the data. In addition, states must be prepared toimpose consequences on districts for submittingincomplete or incorrect information.

With a robust data audit system, policymakers andeducators will know:

Whether or not the disaggregated student informa-tion used to rate schools for Adequate YearlyProgress (AYP) is valid.

The districts that do the best job of accuratelyreporting their dropout data.

Whether or not districts are reporting valid numbers of untested students and reasons for nottesting the students.

The amount and type of data quality problemsidentified by districts and how those problems are being addressed.

0

D

mU

D

m

A

m

10 A State Data Audit System Assessing Data QValidity and Reliability 19 states report having this element


The state adopts and clearly communicates a set of data dand standards to all entities submitting data to the state.

A thorough data audit system is developed and used to edata received by the state education agency are accurate the data definitions and standards adopted by the state.

Thorough training and professional development on dataand standards are provided to all education units in the st


13/20

In addition to the 10 essential elements, states need to

ensure that they take into account the following fun-damental concepts in the construction of their longitu-dinal systems.

Privacy Protection: One of the critical concepts thatshould underscore the development of any longitudi-nal data system is preserving student privacy. Animportant distinction needs to be made betweenapplying a unique student identifier and makingpersonally identifiable information available, for

example. It is possible to share data that are unique toindividual students but that do not allow for the iden-tification of that student. It also is critical to put inplace encryption and data security protocols to securethe transmission or transaction of data between andamong systems. States should ensure that they bringprivacy considerations into the development of eachrepository and the exploration of each protocol orreport.

Data Architecture: Data architecture defines how dataare coded, stored, managed and used. Good dataarchitecture is essential for an effective data system.Many states are in the process of improving their dataarchitecture so that they can clearly communicate withall entities with which they share and from whichthey receive data. Districts need to know specificallyhow data elements are defined (e.g., what a dropoutis), how they should be formatted, and how and whenthe data should be transferred to the state educationagency. Without these standard definitions and dic-tionaries, state education agencies will have an

extremely difficult time making sense of the data

received from their districts. With standards in placethat are used by everyone, staffing resources and pro-cessing or cycle time can be greatly reduced, data can be made available to users when they need them, andreports can be based on clear and common definitions.

Data Warehousing: Many states are in the process of designing and building or upgrading their data ware-houses. Policymakers and educators need data sys-tems that not only link student records over time and

across databases but also make it easy for users toquery those databases and produce standard or cus-tomized reports.

A data warehouse is, at the least, a repository of dataconcerning students in the public education system;ideally, it also would include information about edu-cational facilities and curriculum and staff involved ininstructional activities, as well as district and schoolfinances. The warehouse should ensure student and

teacher confidentiality, allow longitudinal analyses,and include analytical capabilities for its users.Examples of the capabilities that should be availablein a data warehouse include, but are not limited to,trend analyses; tracking of students over time andacross campuses and/or districts; queries designedand conducted by different users (with different levelsof access to detailed data, depending on user classifi-cation); and standard summary reports at the campus,district or state level for policymakers and educators.The key to effective data warehousing is the timelyand efficient use and reporting of data.

Fundamentals in Designing State Longitudinal Data System


14/20

Interoperability: Data interoperability entails the

ability of different software systems from differentvendors to share information without the need forcustomized programming or data manipulation bythe end user. Interoperability reduces reporting bur-den, redundancy of data collection, and staff time andresources. It allows for better, faster and clearerreporting of data. It depends on systems having com-mon data standards and definitions (as addressed inelement 10 above). Organizations such as the SchoolsInteroperability Framework Association work toensure the creation of platform-independent, vendor-neutral open standards that can be used by educatorsand vendors to design and implement interoperabledata systems.

Portability: Data portability is the ability to exchangestudent transcript information electronically acrossdistricts and between pre-K12 and postsecondaryinstitutions within a state and across states. Portabilityhas at least three advantages: it makes valuable diag-nostic information from the academic records of stu-dents who move to a new state available to theirteachers in a timely manner; it reduces the time andcost of transferring students high school course tran-scripts; and it increases the ability of states to distin-guish students who transfer to a school in a new statefrom dropouts. The large interstate movement of stu-dents in the wake of Hurricane Katrina made thevalue of such a system obvious. Data portability is

supported by the implementation of interoperable sys-tems, but it requires states that use these systems tohave a set of common definitions or protocols.

Professional Development around Data Processes

and Use: Building a longitudinal data system requiresnot only the adoption of key elements outlined in thispaper but also the ongoing professional developmentof the people charged with collecting, storing, analyz-ing and using the data produced through the newdata system. The local school person who inputscourse grades needs to understand fully how his/herwork fits into the broader data system, the principalneeds to understand how data can effect daily schoolmanagement both facilities and academic decisions and policymakers need to understand how theirdecisions are limited or expanded based on the qualityof the data available. For these changes in culture andmanagement to occur, states need to make it a priorityto rethink and possibly reorganize how education datais managed throughout the system, increase trainingand professional development for staff both man-agers and users and assist all employees andstakeholders of the state education system to be active

consumers of the longitudinal data system.Researcher Access: Research using longitudinal stu-dent data can be an invaluable guide for improvingschools and helping educators learn what works.These data are essential to determining the value-added of schools, programs and specific interventions.States are developing ways to make student-level dataavailable to researchers while protecting the privacyof student records under the Family Education Rights

and Privacy Act. Because state education agencies andlocal school districts usually do not have the resourcesto conduct this research themselves, providing accessto the data to outside researchers with appropriateprivacy protections allows critical research to be doneat no cost to the state or to school districts.

2

D

mU

D

m

A

m


15/20

These 10 elements and design fundamentals are essen-

tial but not sufficient. States need to plan for a seriesof next-generation improvements in fact, somestates are already working on them. In the future, datasystems can make it possible to do the following:

Connect school performance to spending: Building alongitudinal data system will go a long way towarddetermining how much value schools and districts areadding to student outcomes. This system will helpidentify which school systems do particularly well

and might serve as models for lower-performingpeers. However, the question that such identification begs is why are these school systems so effective?How do these school systems allocate their resources time, staff and money to achieve their success?How can less successful school systems reallocatetheir existing resources or allocate new resources toachieve the same success?

The existing financial data reported within states are

limited. For example, most systems do not report beyond the district level, making it impossible to tellhow resources are allocated within individual schools.To really harness the power of a longitudinal data sys-tem, states need to collect financial data at the schoollevel, and ultimately at the program level, so they canmatch initiatives or interventions with any subsequentchanges in outcomes. In fact, another complete set of essentials would be required to build a high-qualitydata system that links resource allocation with thelongitudinal data system described in this paper.

That said, states could use existing financial data to better understand how the functional resource alloca-tions of the most successful schools (or their districts)

compare to those that are not as successful. For exam-

ple, most states could use existing financial data toanswer the following questions:

Are the most successful schools more likely to belocated in districts that allocate more of theirmoney toward instruction than are less successfulschools?

Do districts with schools that have increased theirsuccess with student cohorts over time spend moremoney on staff development than other districts?

Do patterns of improvement in student outcomescorrespond to changes in overall spending levels orspecific spending allocations?

How does the overall spending level of districtswith the most successful schools compare with theamount spent by less successful districts?

These are questions that policymakers grapple withregularly in almost every state. Creating a comprehen-

sive data system that links longitudinal student out-come data to financial data at the district level andultimately, the school or even classroom level canhelp decisionmakers get to the next level and figureout not only what works but also what it takes todeliver what works.

Connect school performance to employment: Thepre-K12 education system is a precursor to employ-ment either directly (straight from high school) or

indirectly (after postsecondary education). Educatorsand policymakers need to know whether schools arepreparing students for long-term success in the work-place. Obtaining this information requires matching

Future Directions of State Data Systems


16/20

the pre-K12 and postsecondary academic records and

employment records of individual students. Becauseemployment records use social security numbers(SSNs), the educational data systems must be able tocollect students SSNs under suitable privacy protec-tions to make the matching of school and employmentrecords possible. This requires some states to lift

restrictions on the collection of social security num-

bers by the pre-K12 and postsecondary systems.

States also should consider incorporating into theireducation data system as needed records from othersocial service agencies that have information relevantto students health and safety.

4

D

mU

D

m

A

m

Without data, youre just another person with anopinion. This aphorism underscores the importanceof making educational decisions at every level basedon valid and reliable information.

Only a handful of states have data systems with mostof the 10 essential elements, and no state had all 10elements as of 2005. (To see how your state stacks up,

visit www.DataQualityCampaign.org.) The foundingpartner organizations of the Data Quality Campaignare eager to support states in the development andmaintenance of data systems that include all 10 essen-tial elements. The partners challenge every state to

put these systems in place within three years. Thecampaign also commits to providing tools, resourcesand support as states develop quality longitudinaldata systems.

There has never been a more unique opportunity orurgent need for every state to create and embrace alongitudinal data system. As a nation, we must take

advantage of this unique moment and work togetherto ensure that states have the data foundation andinfrastructure they need to support and enrich thehard work under way to strengthen our schools andimprove student achievement.

Call to Action


17/20

Appendix: State Education Agency Actions

This paper has dealt primarily with actions and issues with global state policy

actions related to each of the 10 essential elements. Each of those, however,translates to multiple specific actions that need to occur at the state educationagency level. Examples of specific state education agency actions associatedwith each element include:

1. A Unique Statewide Student Identifier

The state assigns each student a unique statewide student identifier thatcan be used to match records accurately across databases and years.

The state develops procedures to ensure that two identifiers of the sametype are not assigned to the same student (e.g., when the student moves,she/he keeps the same identifier) and that two students are not assigned

the same identifier.

The state will assign an identifier that will follow each student from kinder-garten (or prekindergarten when applicable) through 12th grade as he orshe moves across campuses or districts and leaves and re-enters the statespublic education system.

2. Student-Level Enrollment, Demographic and ProgramParticipation Information

The state collects information at least annually on each students:

campus of enrollment

grade level gender ethnicity economic status English language learner status participation in bilingual or English as a Second Language program special education status migrant status Title I status gifted and talented status

At least one enrollment data collection takes place in the fall.

At least one data collection for each of these items occurs at a differenttime from when the state test is administered.

The information is stored permanently in a state database. (If the informa-tion is used to populate the test database and then discarded, the stateloses track of enrollment over time.)

The state collects student attendance data either daily or over a smallperiod of time (e.g., over a six- or nine-week period) that includes at aminimum campus of attendance, number of days absent and number of days present.

3. The Ability To Match Individual StudentsTest Records from Year

to Year To Measure Academic GrowthThe state updates its student test database with demographic and programparticipation information collected earlier in the school year rather thanupdating at the time of the test administration. (This assumes a studentidentifier is available to connect the two databases.)

The state collects and permanently stores information on each studentstest score in each subject for year-to-year comparisons. The informationmay be disaggregated by skill or skill area for each student (e.g., readingcomprehension/ability to identify the main idea).

The state makes the data available and/or uses the statewide database to

conduct research and program evaluation activities (e.g., the measuremeof year-to-year student academic growth).

At the beginning of the year, the state makes available to each teacher tescore information on state exams that can be broken out by specific skillareas within each subject for each of the teachers students.

4. Information on Untested Students

The state maintains a record for each untested student in a tested grade,including information on why the student was not tested.

The state matches information on untested students to demographic, pro-

gram participation and attendance information.The state conducts analyses of patterns among untested students acrosscampuses and districts.

5. A Teacher Identifier System with the Ability To Match Teachers toStudents

The state assigns each teacher a unique statewide identifier that can beused to match records accurately across databases and years.

The state develops procedures to ensure that two identifiers of the sametype are not assigned to the same teacher and that two teachers are notassigned the same identifier.

The state collects information on each teachers college major, graduateschool degrees by degree type and subject, types of certification or credetial, certification exam scores, salary, and experience.

The state collects data from each school district that match each teacher the students taught in each of the teachers classes, by teacher and studenidentifier.


18/20

6. Student-Level Transcript Information, Including Information on

Courses Completed and Grades EarnedThe state adopts or develops and maintains an ongoing electronic courseclassification system (including standard course numbers, titles anddescriptions).

The state collects individual course completion records for all courses takenin middle and high school. These include:

courses taken during the regular fall or spring semesters, courses taken in summer school, courses taken in middle school for high school credit (e.g., Algebra I), courses taken at local colleges for dual credit,

credits transferred from private high schools or home school, and credit received for distance learning.

The state also collects the grade the student earned in each course and thestudents overall grade point average.

Individual student records in the course completion database can be con-nected to the same studentsrecords in the enrollment, demographic, pro-gram participation and test databases.

7. Student-Level College Readiness Test Scores

The state acquires from the College Board, ACT and the International

Baccalaureate Organization and permanently stores student-level resultsby individual exam for each of the following: SAT, SAT II Subject Tests,ACT, AP, IB.

The state is able to connect the student-level test data mentioned above tothe enrollment, demographic, program participation and test databases.

8. Student-Level Graduation and Dropout Data

The state collects and stores graduation and dropout data at the student level.

The state collects and stores student-level graduation data by diploma type(e.g., Recommended Graduation Plan, New York Regents Diploma).

For students in grades 712 who were enrolled in one year, not enrolledthe next year and did not graduate, the state collects information fromlocal school districts on where each departing student went. The evidenceon departing students can be used to determine whether students gradu-ated; dropped out; transferred to another school, district or state; earned aGED; or are missing (they cannot be located, and no evidence exists onwhere they went).

The state makes every effort to track reported dropouts back to otherschools in the state via the enrollment, test and/or attendance databases.

The state has standards for the types of evidence that may be u

determine where departing students went.The state has standards for the percentage of departing studentschool districts should be able to locate.

The state applies consequences to school districts that do a pooaccounting for missing students (e.g., lower accountability rati

9. The Ability To Match Student Records between Pre-K1Postsecondary Systems

The state works with the postsecondary system to match studenrecords among all institutions of the states pre-K12 and publieducation systems.

The postsecondary information to be matched includes, but is nto, student records on:

enrollment course completion graduation degrees and certificates received performance on mandated state tests administered by thpostsecondary system

10. A State Data Audit System Assessing Data Quality, Vali

ReliabilityThe state develops a clear set of data standards and definitions to all data received by the state education agency.

The state provides training on these data standards to local schopersonnel.

The state performs statistical checks on data submitted by scho

The state has criteria established for determining when data subschool districts is likely to be in error.

The state has a system for investigating the accuracy of data th

by the statistical checks.The state has a system for occasionally spot-checking the accuin cases that are not flagged by statistical checks.

The state has a system of selecting districts for on-site audits aon-site audits in the selected districts.

The state imposes consequences on school districts that do a pocollecting and submitting accurate and complete information.

6

D

mU

D

m

A

m


19/20


20/20

Visit the Data Quality Campaign Web siteat www.DataQualityCampaign.org formore information about the:

10 essential elements and the statepolicy actions required to establish,maintain and use a quality longitudi-nal data system;

results of NCEAs 2005 survey thatshow where your state stands on the10 essential elements; and

tools, materials, meetings and infor-

mation that can aid states and interest-

ed organizations seeking to ensureincreased quality, accessibility and useof data.

This white paper was produced by theData Quality Campaign and made possible with the financial support of theAchieve, Inc., American Diploma Project.It is based on work originally written byChrys Dougherty, Ph.D., NCEA directorof research, and Nancy J. Smith, Ph.D.,

NCEA associate director of research anddevelopment.

Data Quality CampaignThe Data Quality Campaign is a national,collaborative effort to encourage and sup-port state policymakers to

improve the collection, availability anduse of high-quality education data and

implement state longitudinal data sys-tems to improve student achievement.

The campaign provides tools andresources that states can use as theydevelop quality longitudinal data systemsand also will serve as a national forumfor reducing duplication of effort and pro-moting greater coordination and consen-sus among like-minded organizations.

Founding partners in the Data Quality

Campaign include:

Achieve, Inc.Alliance for Excellent EducationCouncil of Chief State School OfficersThe Education TrustNational Center for EducationalAccountabilityNational Center for Higher EducationManagement SystemsNational Governors AssociationCenter for Best PracticesSchools Interoperability Framework AssociationStandard & Poors School EvaluationServicesState Higher Education ExecutiveOfficers

The campaign is managed by theNational Center for EducationalAccountability and supported by the Bill& Melinda Gates Foundation.

For more information, visit theData Quality campaign Web site at

American Diploma Project NetworFormed out of the 2005 NationalEducation Summit on High Schools, theAmerican Diploma Project (ADP)Network is a coalition of 22 states, led by Achieve, Inc., that have committedto align K12 standards, assessments,curriculum and accountability with thedemands of postsecondary education andwork. ADP states are taking four actions:

Aligning high school standards withthe knowledge and skills required for

success after high school.Requiring all high school graduates totake challenging courses that actuallyprepare them for life after high school.

Streamlining the assessment system sothat the tests students take in highschool also can serve as readiness testsfor college and work.

Holding high schools accountable for

graduating students who are ready forcollege or careers, and holding post-secondary institutions accountable forstudents success once enrolled.

For more information, visit AchievesWeb site at www.achieve.org.

American Diploma Project Network

Find Out More

measuring what matters: creating a longitudinal data system to improve student achievement

Documents