understanding the 2011-2012 accountability report & report card timothy johnson david scalzo...

Understanding the 2011-2012 Accountability Report & Report Card

Timothy JohnsonDavid Scalzo

Erie 1 BOCES Data Analysis Team. All rights reserved.

Before We StartYou will need to gather some resources from:

NYSED site

SEDDAS Portal

Level 2 Data Warehouse


Before We Start…

2010-11 AOR and CIR for your District and Schoolshttps://reportcards.nysed.gov/counties.php?year=2011

2011-2012 Accountability Report and Report Card for your District and Schools

https://reportcards.nysed.gov/counties.php?year=2012 SEDDAS – IRSP Portal – ESEA 2010-2011 Revised Performance Indices

Report – released July 2012http://portal.nysed.gov/portal/page/pref/PortalApp

Effective Annual Measurable Objectiveshttp://www.p12.nysed.gov/irs/accountability/amos/2012/EAMOs-for-AYPDeterminationBased-on-2011-12Results.pdf

Ira Swartz Memo from January 2013http://www.p12.nysed.gov/accountability/documents/FieldMemoReminderofKeyChanges-Revised.pdf

Accountability Determinations FAQ April 2013 documenthttp://www.p12.nysed.gov/accountability/documents/2011-12AccountabilityDeterminationsFAQ.pdf


Resources

https://reportcards.nysed.gov/counties.php?year=2011

https://reportcards.nysed.gov/counties.php?year=2012

http://portal.nysed.gov/portal/page/pref/PortalApp

http://www.p12.nysed.gov/irs/accountability/amos/2012/EAMOs-for-AYPDeterminationBased-on-2011-12Results.pdf



http://www.p12.nysed.gov/accountability/documents/FieldMemoReminderofKeyChanges-Revised.pdf

http://www.p12.nysed.gov/accountability/documents/FieldMemoReminderofKeyChanges-Revised.pdf

http://www.p12.nysed.gov/accountability/documents/2011-12AccountabilityDeterminationsFAQ.pdf

http://www.p12.nysed.gov/accountability/documents/2011-12AccountabilityDeterminationsFAQ.pdf

Level 2 Verification Reports - https://dataview.wnyric.org

SIRs 101 - HS Accountability Reports

SIRs 102 - Elementary/Middle Level Accountability Reports

SIRs 307 – Annual Dropout and Non-Completer Report

SIRs 201 – Total Cohort Summary Report

SIRs 202 – Total Cohort Assessment Summary Report


Resources

https://dataview.wnyric.org/



Changes in NYS School Year End Reports

Pre 2011-2012 2011-2012 and Forward


Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts


AOR – Accountability & Overview Report

District Summary Elementary/Middle Level ELA, Math, Science participation and

performance Secondary Level ELA, Math, Science participation and

performance Graduation Rate



Breakdown of school accountability in a district


Shows grade by grade breakdown of all subject grade level performance on state assessments


CIR – Comprehensive Information Report

Regents Exams Regents Competency Tests NYSESLAT NAEP Cohort Performance after four years NYSAA High School Information

High School Completers Non-completers Post-Secondary Plans.


Adequate Yearly Progress and Performance Indices Elementary/Middle Level

ELA AYP-Participation-Performance Math AYP-Participation-Performance Science AYP-Participation-Performance

Secondary Level ELA AYP-Participation-Performance Math AYP-Participation-Performance

Un-weighted Combined ELA and Math Performance Indicators Graduation Rate AYP 4 year Graduation-Rate Total Cohort 5 year Graduation-Rate Total Cohort


The New Accountability Report…

Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts


The New Report Card Report…

Contains brief description of NYSTP Assessments, NYSAA Assessment, NYSESLAT Tests, Secondary Level Cohorts

Grade by grade breakdown of all subject grade level performance on state assessments for 3-8 Assessments

Statewide NAEP Results, Participation Rates for NAEP, LEP, Students with Disabilities

Total Cohort Results after 4 Years of Instruction in Secondary ELA, Mathematics, Global History and Geography, US History

and Government, and Science. NYSAA results, Regents Exams,

Regents Competency Tests, NYSESLAT Erie 1 BOCES Data Analysis Team. All rights reserved.

High School Completers

High School Non-Completers

Post-Secondary Plans of 2011-2012 Completers


The New Report Card Report…

Easy Part First…

New Report Card is a repackaging of reports we have seen before.

Profile Section = AOR District Profile Student Performance = AOR Overview of District

Performance minus Summary plus CIS Assessment Breakdown

Student Outcomes = CIS High School Information


Hard Part – Accountability Report

In order to understand this report you must have an understanding of the changes to accountability as a result of the ESEA Waiver. The Ira Swartz memo from January 2013 summarizes this with three attachmentsAttachment A – Revised Method for Calculating PIAttachment B – How NYS Reset Annual

Measurable Objectives (AMO)Attachment C – 4 year and 5 year Graduation

Rates and Adequate Yearly Progress (AYP)Erie 1 BOCES Data Analysis Team. All rights reserved.

Methodology for Calculation of PI at the Elementary/Middle Level

Look at Attachment A of Ira Swartz January 2013 memo PI for ELA and Math now gives districts credit for

students in Level 1 and Level 2 who are on track to proficiency

PI for calculating 4 & 8 Science remains the same because there is no way to project a track to proficiency without grade level assessments between the 4th and 8th grade tests.

Note– this will improve a district’s chances of making AYP for the Elem/Middle Level because of the formula changes.


Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012

PI = ([2(Count at Level 1 On Track) + Count at Level 2 Not On Track + 2(Count at Level 2 On Track) + 2(Count at Level 3) + 2(Count at Level 4)] ÷ [Count of Tested Students]) × 100

PI = ([2(0) + (146) + 2(4)+2(235)+2(121)]/[521] )x 100

PI = ([0 +146+8+470+242]/[521]) x 100

PI = (866/521) x 100

PI = 1.66x 100 = 166

Sample PI Calculation


Performance LevelOn Track to Proficiency

Number of Students Multipier Total

1 (Below Standards) No 15 0 0

1 (Below Standards) Yes 0 200 0

2 (Meeting Basic Standards) No 146 100 14600

2 (Meeting Basic Standards) Yes 4 200 800

3 (Meeting Proficiency Standards) NA 235 200 47000

4 (Exceeding Proficiency Standards) NA 121 200 24200

Totals 521 86600

PI = 86600/521=166

Another Way to Look at PI…


Methodology for Calculation of PI at the Secondary Level

The formula for calculating the PI at the Secondary Level remains the same.

The rigor increases because the cut points for the Levels change (and are different) for ELA and Math.


Erie 1 BOCES Data Team. All rights reserved.

Safe Harbor is an alternative way to meet performance criterion for accountability groups whose PI’s are less than their EAMOs.

The formula for 2011-12 is: 2010 -11 PI + (200 – 2010-11 PI)x0.102010-11 PI’s which were reissued, are available in the ISRP portal.

In the above example, the 2010-11 revised PI was 136Note: Revised PI obtained from the IRSP Portal – ESEA 2010-2011 Revised Performance Indices Report

SH = 136 + (200-136)x0.10 = 136 + 6.4 = 142.4 = 142

Safe Harbor

138


If the SH Target matches the EAMO, it is because the formula would have yielded a result greater than the EAMO, as in the example below.

The 2010-11 PI was 133

SH = 133 + (200-133)x0.10 = 133 +6.7 = 139.7> EAMO of 128

Note: Safe Harbor will no longer require schools and districts to meet the third academic indicator requirement (Science or Graduation Rate).

Safe Harbor

The formula to calculate high school PI remains the same as follows: ((Level 2 + Level 3 + Level 4 + Level 3 + Level 4)/Cohort)*100 Districts get half credit for Level 2s and full credit for Level 3s & 4s.

Note: Under the new scales there will be more 2s and fewer 3s and 4s.

This lowers the PIs for most districts.

Note: Under this methodology districts receive “no credit” in the PI

for students who score between 55-64 on a Regents exam or Pass a

RCT exam.


Cut Point Chart

Reset of the Annual Measurable Objectives (AMO)

Look at Attachment B of Ira Swartz January 2013 memo

Because of ESEA Waiver AMO’s were reset to reflect student growth.

These EAMOs (Effective Annual Measurable Objectives) are by group size – this is where the EAMO values came from on the Accountability Charts.

The charts found at:http://www.p12.nysed.gov/irs/accountability/amos/2012/EAMOs-for-AYPDeterminationBased-on-2011-12Results.pdf




4 Year and 5 Year Cohort Graduation Rates and AYP

The numbers in the total cohort groups change based on the following: One day enrollment changes the numbers from

previous report cards. These numbers used to be based on 5 month enrollment.

Incarcerated youth will not be counted as transfers. Ungraded SWD will be entered into a Cohort based

upon when they first entered a 9th grade program.


4 year and 5 year Graduation Rate to make AYP

Its 4 year graduation rate cohort meets or exceeds the graduation goal; or its 4 year graduation rate cohort meets or exceeds the 4 year progress target.

Its 5 year extended graduation rate cohort meets or exceeds the graduation goal; or its 5 year extended graduation rate cohort meets or exceeds the 5 year progress target.


Things to watch out for!

Elementary/Middle Level

Participation charts are based on Students enrolled during the test admin period.

Performance charts are based on Tested Students Enrolled on BEDS day


Things to watch out for!

Secondary

Participation charts are based on 12th Graders with valid test scores

Performance charts are based on Cohort groups



Unweighted combined (or average) PI’s for ELA and Math for the building or district, by subgroups.

In this example, (163+177+170+136)/4 = 161.5 = 162

Unweighted Combined ELA and Math PI’s

Level 2 Charts can Help with Discrepancies

Level 2 charts have different ways of displaying data that help in resolving data discrepancies.

Level 2 charts also allow you to drill down to see individual students. When you see a student’s name it can help to determine if that student falls into one of the three reasons that may move students between cohorts.


Reasons for Data Surprises• Elem/MS PI formula changes caused some districts to make AYP and

others not to make AYP – students not on track to proficiency made the difference between districts making AYP at Elem/MS Level

• High School Levels range changes made it more difficult for student groups to make AYP at HS level

• Differentiated accountability changed the EAMO targets because they are based on the NCLB subgroups (buckets)

• Changes in Grad % for four and five year were caused by rule changes for placing students in cohorts

• Changing from 5 month enrollment to 1 day enrollment caused numbers of students districts were held accountable for to rise

• Ungraded students who received IEP Diplomas were added to cohort numbers



School that is not a Priority or Focus School that: a) Has large gaps in student achievement among subgroups or students orb) Has failed to make AYP for three consecutive years with same subgroup on same measure orc) Is located in a non-Focus district but is among the lowest in the state for the performance of one or more subgroups and for which the school is not showing progress

Contact us for Clarification

Timothy [email protected]

David [email protected]


mailto:[email protected]

mailto:[email protected]

understanding the 2011-2012 accountability report & report card timothy johnson david scalzo...

Documents

boces data analysis

new report card report

new accountability report

data warehouse erie

district erie

completers erie

nyseslat erie

forward erie