accountability 2013 Interpreting Your 2013 Accountability Report Its Like Learning To Read All Over Again Ervin Knezek John Fessenden TEPSA Webinar | September 5, 2013 goals 2013 accountability Construction Clarity Of calculation... and application Connection Of indexes What measures/data are included? To instructional processes and systems TEPSA Webinars (May 10 and 17) Today Ongoing the vibe Your Questions 5 the BIG picture p. 21p. 22 Accountability | Overall Design Four Performance Indexes Designed to meet 4 statutory goals 1.Improving student achievement at all levels in core subjects 2.Ensuring progress of all students toward Advanced Academic Performance (STAAR Level III) 3.Closing Performance Gaps among groups in Advanced Academic Performance (STAAR Level III) 4.Closing gaps among groups in % of students graduating under Recommended or Advanced high school programs Index 1 Index 2 Index 3 Index 4 p. 22 2013 Accountability Achievement All Students Only Single percentage combined over ALL Subject Areas Level II (Phase 1) STAAR Met Std Grade 11 TAKS Progress 10 student groups All Students 7 Race/Ethnicity SpEd ELL Minimum size 10 for All Students 25 for other groups By Subject Area Rdg & Math in 2013* Based on PROGRESS 1 point for % Met 2 points for % Exceeded Closing Gaps 3 student groups Eco Dis and 2 lowest Race/Ethnicity groups from prior year By Subject Area (all subjects) 1 point for % Level II (Phase 1) Level III to be added (2014) Postsecondary 4-year or 5-year Graduation Rate (or Annual Dropout Rate) 10 student groups % RHSP/DAP 8 student groups (not SpEd or ELL ) STAAR Postsecondary Readiness Level II Final (2014) Index 1 Student Achievement Index 2 Student Progress Index 3 Closing Performance Gaps Index 4 Postsecondary Readiness Target = 50Target = 5 th %ile Target = 55Target = 75 Target = ESMSHSDist p. 21 Interpreting our Reports Its like learning to read all over again! 2013 accountability... PartSection# Pages 1Accountability Summary1 2Index Calculations and Data Tables12 3Distinction Designations (Campuses Only)4 (8/29/13) 4System Safeguards3 (8/29/13) Total Pages20 (8/29/13)... a report in 4 parts p. 1 p p p Index 1 Student Achievement Index 1 Student Achievement p. 2 p. 3-4 Index 1: Student Achievement Index 1 Student Achievement p Additional Student Groups reported 7 Race/ Ethnicity SpEd Eco Dis ELL Used for System Safeguards p. 2 Calculate Overall Pass Rate For All Students 1.Add # Passed ( Phase-in 1 Level II) each subject area (R + M + W + S + SS) = Add # Tests taken each subject area (R + M + W + S + SS) = 1,386 3.Divide # Passed by # Tests to determine % Passed 720 1,386 = 51.9%, which ROUNDS to 52% 4.Assign 1 point for each % Passed = Index 1 Score 52% = 52 POINTS Overall pass rate IS NOT average pass rate Average pass rate would equal: (57% + 60% + 37% + 47% + 27%) 5 = 47% Index 1 Score 57%60% 37%47%27% p. 3-4 p. 2 Where did these numbers come from? Pages 3-4! % at Final Level II (13%) % at Level III (3%) [Instructional Planning] Disaggregated Data There are a LOT of numbers on pages 3 and 4! I thought Index 1 ONLY evaluated the All Students group! True! BUT... the data are included for System Safeguards p. 3 System Safeguard Low Performance in a single cell for a particular student group does NOT automatically cause the district/campus to miss Index 1 However, ANY student group meeting MSC with a pass rate below 50% MUST be addressed in the district/campus improvement plan MSC = 0 for All Students | 25 for all other Student Groups p. 13 So how do we find System Safeguards? 1.In each subject area, determine which student groups meet MSC MSC = 0 for ALL Students | 25 for all other student groups 2.Then determine if % Passing < 50% (at Phase-in Level II or above) for any group meeting MSC 3.If % Passing < 50% AND Student Group meets MSC, then Student Groups Performance MUST be addressed in improvement plan p. 13 p. 3 p. 13 Others? # Groups Missing Safeguard 8 # Groups Meeting Safeguard 9 p. 3 # Groups Meeting MSC Reading, Math, Writing 17 p. 4 Science? Social Studies? # Groups Meeting MSC Reading, Math, Writing 17 # Groups Missing Safeguard 8 # Groups Meeting Safeguard 9 Science, Social Studies 88 0 TOTAL p. 3 v p. 4 System Safeguards 25 student groups met MSC (across 5 subject areas) ONLY 9 Met Safeguard target of 50% 16 Missed Safeguard target (including EVERY group in Writing, Science, Social Studies) BUT campus MET Index 1 Target! All 16 Student Groups that Missed Safeguard target of 50% must be addressed in CIP System Safeguards Are summarized with Y/N indicators in Part 4 of the Campus 2013 Accountability Report (p. 18) p. 18p. 3 p. 19 System Safeguards Performance Rate data from p. 3-4 are duplicated on p. 19 (slightly reorganized) p. 19 System Safeguards Performance Rate data from p. 3-4 are duplicated on p. 19 (slightly reorganized) Participation Rate data are also provided Index 1 is summarized in the campus 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1 27 Your Questions Index 1 Student Groups? MSC? Safeguards? Index 2 Student Progress Index 2 Student Progress p. 5 p. 6-7 Index 2: Student Progress Index 2 Student Progress p. 22 Index 2: Student Progress Based on actual student progress (2012 to 2013) Index 2 Student Progress STAAR Level III focuses on progress toward... Each student is assigned a progress category based on his/her change in scale score in relation to progress expectations Did Not Meet Met Exceeded Index 2: Student Progress Index 2 Student Progress Index 2: Student Progress Subject areas evaluated for 2013 Reading Grade 4 through English II Math Grade 4 through Algebra I Writing English I to English II Index 2 Student Progress Performance Standard Met Progress Exceeded Progress Index 2 is based on a fraction where Numerator = # of points based on % of students who Met or Exceeded Progress 1 point for % Met 2 points for % Exceeded Denominator = Maximum Points Possible based on number of student groups meeting minimum size criteria (MSC) Maximum Points Possible = 200 points for each student group meeting MSC Index 2: Student Progress Index 2 Student Progress What is progress? Index 2: Student Progress Index 2 Student Progress With exceptions for high-performing (almost perfect) and low-performing (lower than guessing) students, the general rule is Met Progress = Level II Final (Higher Grade) Level II Final (Lower Grade) Exceeded Progress = Level III (Higher Grade) Level II Final (Lower Grade) p. 23 Subject and Grade Level/Course Items Points Level II Standard (Final Recom- mended) Level III Standard Grade 4 (Eng. & Span.) Grade 5 (Eng. & Span.) Met Progress? Level II 5 th Grade Level II 4 th Grade 1627 1599 = 28 p. 23 Exceeded Progress? Level III 5 th Grade Level II 4 th Grade 1710 1599 = 111 p. 23 Subject and Grade Level/Course Items Points Level II Standard (Final Recom- mended) Level III Standard Grade 4 (Eng. & Span.) Grade 5 (Eng. & Span.) p. 23 5 th Grade STAAR Math Progress (2013) 5 th Grade Math th Grade Math 2012 Scale Score Differential Progress Category Performance Level Raw Score Scale Score Performance Level Scale Score Alex III 48 (out of 50) 1847III 2016 (Perfect Score) -169 EXCEEDED PROGRESS Vinnie I261481I EXCEEDED PROGRESS Arthur II391627II DID NOT MEET PROGRESS Index 2 Simple Example All Students # Tests 500 # Met 300 # Exceeded 100 % Met 60% % Exceeded 20% 1 point for % Met OR Exceeded 80 1 point for % Exceeded 20 Weighted Progress Points 100 Maximum Possible Points 200 Index 2 Score (Weighted Progress Points Maximum Possible Points) 50 1 point for % Met 60 2 points for % Exceeded 40 Weighted Progress Points 100 Index 2 Data Reports Build back to front Start with page 7 Then page 6 Then page 5 p. 7 For each student group meeting MSC in each subject, determine: % Met or Exceeded Progress % Exceeded Progress From Index 1 (p. 2) REMINDERS (Why Index 2 numbers will not match Index 1): Only students with 2 years of matched data are included Progress starts at 4 th grade (no 3 rd graders receive a Progress Measure) ELL students are mostly excluded from Index 2 in 2013 STAAR Modified and STAAR Alternate are not included in Index 2 in 2013 Writing is ONLY evaluated for English I to English II MSC = 10 tests for All Students | 25 tests for other Student Groups p. 6 Data from page 7 are reproduced on page 6 In each subject area, for each Student Group meeting MSC Assign 1 point for each % Met OR Exceeded Assign another 1 point for each % Exceeded Total = Weighted Progress Rate for each Student Group meeting MSC Effectively 1 point for each % Met and 2 points for each % Exceeded Add Weighted Progress Rate in each subject for each student group meeting MSC Maximum Points = # Student Groups meeting MSC x p. 5 Data from page 6 are summarized on page 5 (remember: read back to front!) Weighted Progress Rate for each Student Group meeting MSC in each subject area is reported and summed Reading: = 216 points Math: = 209 points Sum the Total Points in Reading and the Total Points in Math = 425 Sum the Maximum Points in Reading and the Maximum Points in Math = 1200 425 1200 = 35.4, which rounds to SO Index 2 Score = 35 points Index 2 Score Index 2 is summarized in the campus 2013 Accountability Summary (p. 1) 2013 Accountability Summary p. 1 47 Your Questions Index 2 Progress Categories Points Data Index 3 Closing Performance Gaps p. 8 p. 9-11 Index 3: Closing Performance Gaps Index 3 Closing Performance Gaps Impacts every student group of which the ELL student is a member p. 24 Index 3: Closing Performance Gaps Index 3 emphasizes Level III (Advanced Academic Performance) for Performance Gap Groups BUT... STAAR Level III cannot be included in accountability until 2014, so for Measure = % of students in Performance Gap Groups passing state assessment Evaluated in EACH subject area (Reading, Math, Writing, Science, Social Studies) Then cumulated to a single score Index 3 Closing Performance Gaps Index 3: Closing Performance Gaps Index 3: Closing Performance Gaps? Index 3 Closing Performance Gaps All Eco Dis % Achieving Level III Index 3: Closing Performance Gaps Student Groups Identified for Evaluation in 2013 Economically Disadvantaged (no MSC) 2 lowest performing Race/Ethnicity groups meeting MSC ( 25 tests) across all subjects from Index 1 in 2012 The 2 Race/Ethnicity Groups are determined based on combined performance across all subjects (Index 1) So the Race/Ethnicity Groups analyzed in Index 3 are the same in all subjects For Index 3 evaluation, MSC for each Race/Ethnicity Group in each subject area = 25 tests taken by students in that Race/Ethnicity Group in that subject area If a Race/Ethnicity Group does not meet MSC in a subject area, then it is NOT evaluated in that subject area Index 3 Closing Performance Gaps p. 8 Summary Data page 8 How were the 2 Race/Ethnicity groups selected? From 2012 performance data p. 10 Summary Chart of 2012 Performance for the 7 Race/Ethnicity Groups Which Race/Ethnicity groups met MSC in 2012 ( 25 tests taken by students in that Student Group across all subject areas)? African American, Hispanic, White If 3 Race/Ethnicity groups meet MSC, then 2 lowest performing are used for Index 3 African American (44%) and Hispanic (51%) If only 2 Race/Ethnicity groups meet MSC, then only 1 Race/Ethnicity group is used for Index 3 (the lower performing group) If only 1 (or 0) Race/Ethnicity groups meet MSC, then Index 3 is based solely on Economically Disadvantaged Index 3 Data Reports Build back to front Start with pages 10 and 11 Then pages 8 and 9 Then top of page 8 p. 10 For each subject and each student group, build from the bottom row up Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above p. 11 Repeat for Science and Social Studies Total Tests # at Level II Phase 1 or above % at Level II Phase 1 or above Notice that there are fewer than 25 tests taken by African American students in Science and Social Studies A Student Group is NOT EVALUATED in a subject area UNLESS it meets MSC IN THE SUBJECT AREA ( 25 tests taken by students in that student group in that subject area) p. 8 These data come from pages p. 10 p. 8 In each subject area, for each Student Group meeting MSC: Assign 1 point for each % at Phase-in 1 Level II or above Then total Weighted Points Maximum Points = 100 x Number of Groups meeting MSC ++ ++ p. 9 Determine Weighted Performance Rate for Writing, Science, Social Studies