1 Getting Up to Speed on Value-Added - An Accountability Perspective Presentation by the Ohio Department of Education 2 Agenda 1)Value-Added and the Accountability System 2)2007 Overall Value-Added Results 3)Analysis of Individual Districts/Schools 4)AYP Growth Model 5)Data Availability 3 Value-Added and the Accountability System 4 Former Accountability System Architecture Indicators Met (District #s in parentheses) Performance Index Score Change in PI Score AYP Status Excellent 94% - 100% (29 or 30) oror 100 to 120 andand Met/Not Met Effective 75% % (23 to 28) oror 90 to 99.9 andand Met/Not Met Continuous Improvement 0% % (0 to 22) oror 0 to 89.9 PI Score Improvement Criteria Met andand Met 50% % (15 to 22) oror 80 to 89.9 oror andand Not Met Academic Watch 31% % (10 to 14) oror 70 to 79.9 oror andand Not Met Academic Emergency 0% % (9 or fewer) andand 0 to 69.9 andand Not Met 5 Up to 2007, Performance Index Growth has been used Starting in 2008, Value-Added data will be used for districts and schools with tested grades 4-8 Change in Improvement Measure: Implementing Value-Added 6 Ohios Value-Added Measure Measures the contribution of a school or district to the progress of its students on test scores Requires a Scaled Score metric Current Alternate Assessments are not measured on a Scaled Score 7 Ohios Value-Added Measure Scores are measured in Normal Curve Equivalent gains 0 gain represents the typical or expected gain Value-Added is based on the 2006 07 distribution of scores Scores also use a measure of precision (1 Standard Error) to help describe the Value-Added classification 8 There are three classification bands using gain score and 1 Standard Error + Above expected growth (Green) Met expected growth (one year of growth in one year of time) (Yellow) - Below expected growth (Red) Ohios Value-Added Measure 9 Scores calculated for: Grades (4-8) Subjects (Reading and Math only) Grade and subject composites School composite District composite Only District and School composite scores will be used for ratings Ohios Value-Added Measure Meet ( ) Exceed (+) Meet ( ) Not Meet (-) Gain Score, Standard Error and Classifications Relative to Value-Added Standard Exceed (+) 11 Value-Added Will Affect Ratings first year VA can change LRC designation Reward / sanction Reward enough growth Penalize insufficient growth 12 Impact on Designation Above expected gain is rewarded Each rating category is rewarded based on above expected gain, including an Excellent with Distinction rating At least two years of above expected gain 13 Below expected gain results in lower ratings Each rating category is impacted by below expected gain Rating is lowered if you have three years of below expected gain Academic Emergency will not be lowered Impact on Designation New Accountability System Architecture Including VA Impact Indicators Met Performanc e Index Score AYP Status Preliminary Designatio n 94% - 100% oror 100 to 120 andand Met or Not Met Excellent 75% % oror 90 to 99.9 andand Met or Not Met Effective 0% % oror 0 to 89.9 andand Met Continuous Improvemen t 50% % oror 80 to 89.9 andand Not Met 31% % oror 70 to 79.9 andand Not Met Academic Watch 0% % andand 0 to 69.9 andand Not Met Academic Emergency Did the Preliminary Designatio n increase or decrease based on the AYP Status? IF YES STOP HERE No additional change to the designatio n can occur based on the value added calculation IF NO CONTINU E Value- added MAY affect a designatio n when it has not been changed by the AYP Status Preliminary Designatio n Amount of growth using value-added calculation Final Designation Excellentandand Above expected growth for at least 2 consecutive years Excellent with Distinction Below expected growth for at least three consecutive years Effective Otherwise no effect on ratingExcellent Effectiveandand Above expected growth for at least 2 consecutive years Excellent Below expected growth for at least three consecutive years Continuous Improvement Otherwise no effect on ratingEffective Continuous Improvement andand Above expected growth for at least 2 consecutive years Effective Below expected growth for at least three consecutive years Academic Watch Otherwise no effect on ratingContinuous Improvement Academic Watch andand Above expected growth for at least 2 consecutive years Continuous Improvement Below expected growth for at least three consecutive years Academic Emergency Otherwise no effect on ratingAcademic Watch Academic Emergency andand Above expected growth for at least 2 consecutive years Academic Watch Otherwise no effect on ratingAcademic Emergency 15 Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA Above Expected in 2007 and 2008 Final Rating ? 16 Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA Above Expected in 2007 and 2008 Final Rating - Excellent 17 Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA Above Expected in 2007 and 2008 Final Rating - ? 18 Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA Above Expected in 2007 and 2008 Final Rating - Continuous Improvement (Why?) 19 Proposed Report Card Graphic Grade Reading Mathematics Subject Composite Total Composite* + + Grade Composite *Used in LRC rating Overall Value-Added Results 21 Value-Added 2007 Data Scores based on four years of data: Grades 4-8 results (Reading, Math, Writing, Science and Social Studies) Grades 3-7 results (Reading and Math in addition to Grade 4 Writing) Grade 3 results (Reading and Math); Grades 4-5 results (Reading only) Grade 3 results (Reading only) 22 Achievement Test Results: NCE Means based on 2007 results Math Grade 23 Relationship of Gains to Grade Level 24 Achievement Test Results: NCE Means based on 2007 results Reading Grade 25 Relationship of Gains to Grade Level 26 Value-Added 2007 Composite Results 27 Relationship of LRC Designation to District Typology *Note: Urban 21 are a subset of All Urban category 28 Relationship of Composite VA Gains to District Typology *Note: Urban 21 are a subset of All Urban category 29 Relationship of VA Gains to Achievement 30 Relationship of VA Gains to Math Achievement 31 Relationship of VA Gains to Reading Achievement 32 Performance (Status) by % Poverty 33 Performance (VA Gain) by % Poverty 34 Performance (Status) by % Minority 35 Performance (VA Gain) by % Minority 36 Analysis of Individual Districts/Schools 37 Value-Added Gains and Performance Index High Performance Low Value-Added 38 Value-Added Gains and Performance Index Average Performance Low Value-Added 39 Value-Added Gains and Performance Index Very Low Performance and Value-Added 40 Value-Added Gains and Performance Index - Very Low Performance Green VA But is it enough? 41 Value-Added Gains and Performance Index Green- But will it be enough to improve? 42 Value-Added Gains and Performance Index Low Performance High Value-Added 43 Value-Added Gains and Performance Index High Performance High Value-Added 44 Value-Added Gains and Performance Index High Performance High Value-Added 45 Within District Variability School level VA Gains vs. Performance 46 AYP Growth Model 47 Changes in 2008 New Goals Making minimum N uniform Include a Growth Model criterion 48 New Goals Reading Reading Math Math Elementary Grade 371.2%77.0%60.6%68.5% Grade 468.3%74.6%67.1%73.7% Grade 568.3%74.6%49.6%59.7% Middle Grade 675.8%80.6%55.1%64.1% Grade 768.6%74.9%47.3%57.8% Grade 873.8%79.0%47.5%58.0% HighOGT (Grade 10)71.8%77.4%60.0%68.0% 49 Uniform Minimum N Size Minimum size for evaluation was: 30 for all groups except students with disabilities 45 for students with disabilities Minimum N size starting with : 30 for all groups Minimum N size change for report card data to meet federal requirement 50 Up to , districts and schools could meet AYP achievement in one of three ways: 1)By meeting or exceeding all AYP targets; 2)By meeting or exceeding AYP targets with a two- year average of previous and current years reported data; 3)Via the AYP safe harbor provision district/school achieves a 10% reduction in the percentage of non-proficient students from the previous year and also meets graduation or attendance rate goal. Meeting Adequate Yearly Progress 51 Growth Model and AYP 4 th way of meeting AYP Individual student projections Projected out to first year of next school (3 years maximum) Includes next school effects Only counts Full Academic Year students in computation 52 Growth Model and AYP AYP% for subgroup = Numerator (count subgroup students only): Sum of 3 rd graders proficient or better + Sum of on track students (those projected to reach proficiency) + Sum of other proficient students with no growth computation. 53 Growth Model and AYP AYP% for subgroup = Denominator: Total number of full academic year students in combined tested grades (including 3 rd where applicable) 54 Meeting AYP Status Only Percent of Students Proficient A D B C MEET AYP MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP MISS AYP Magnitude of Individual Student Gains 55 Meeting AYP Growth Only Percent of Students Proficient A D B C MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP Magnitude of Individual Student Gains 56 Meeting AYP Status and Growth Percent of Students Proficient A D B C MEET AYP MISS AYP Students On-Track to Proficiency Students Not Moving To Proficiency MEET AYP Magnitude of Individual Student Gains 57 AYP Growth vs. Value-Added Gains AYP Growth _______________ Student Help meet AYP OAT Projected gain Students on-track to proficiency Value-Added Gains _______________ School / District LRC rating OAT Composite mean gain Value schools add to students starting points Level Use Data Source Measure Purpose 58 Value-Added Data Availability 59 School and District Value-Added Data Publicly available ODE home page Accountability Menu Value Added and click on Value-Added Data and Reports Data Available through Power User Reports (iLRC) in Ratings folder Data limited to Gain Scores, Standard Errors, and VA N - by subject and grades level and composites 60 Value-Added Data Availability EVAAS Diagnostic Reports Hosted on SAS web site at: https://ohiova.sas.com/evaas/login.jsp https://ohiova.sas.com/evaas/login.jsp Login and password required District superintendent has administrative authority 61 Value-Added Data Availability EVAAS Diagnostic Reports Reports Are interactive Contain one year of data Provide student information by SSID List gain scores only for students who have contiguous grade scores in a subject - Includes only full academic year students (WKC) 62 In the Works: Value-Added Tools for Teachers and Parents Explaining the Value-Added measure in the Accountability System to educators Practical knowledge for teachers to talk to parents about value-added Disk format available before school year 63 Getting Up to Speed on Value-Added - An Accountability Perspective Questions?