accountability: are you aware? kim gilson senior consultant data and accountability region 10 esc...
TRANSCRIPT
Accountability: Are You Aware?
Kim Gilson
Senior Consultant
Data and Accountability
Region 10 ESC
972-348-1480
Index 1: Who PassedIndex 2: Who Grew
• Accountability Reports Tell You:• Performance by Subject• Performance by Student Group
• Accountability Reports DON’T Tell You• Performance by Grade Level, Course,
Teacher…• Too many other things to list!
Sample State ReportsIndex 1
Index 2
Sample Safeguards Report
Student Groups and Minimum Size
• On State Accountability Reports, Student Group Performance is reported on Page 2 of Index 2 as Safeguards• Data is aggregated by subject across the
campus/district• For weak student group performance in
safeguards, break it down by grade level• Where are those students this year?
• Minimum size of 25 means potentially a handful at each grade level, resulting in campus shifts as students progress from one grade to the next
5th 6th 7th 8th
Student Group 2014
9 10 9 4
Student Group 2015
9 10 9
23
28
Phase-In Custom Report
How to Build It
Student Progress Measure Tip
State Report• Combines Students Who
Meet OR Exceeded Growth in the Calculation for 1 Point• Example: If 60% of
students Meet Progress and 30% Exceed Progress, then you will see this:
Aware Report• Separates Students Who
Meet Growth From those Who Exceed• Same Example: 60%
Students meet and 30% Exceed, then Aware will report this:
% Who Met Or Exceeded
90
% Who Exceeded 30
Weighted Score 120
% Met % Exceeds
60 30
Double the Exceeds Score to Give them Their 2nd Point in Index 2
Growth Measure Rule
• Students must test 2 consecutive years on the same version of the test• Example: 2013 STAAR M 2014
STAAR• No Growth Measure can
be calculated• Any student who took a STAAR M last
year will not have a Growth Measure this year
Phase-In Custom Report
Drill Down By Grade and Subject
• District View
How did each grade/subject contribute• Campus View
How did each campus contribute to each grade/subject• Course View
How does performance compare in an intervention course to regular to PAP/AP?
• Teacher View
How do students perform in various teachers’ classrooms?
Drilling Down Continued
• Reports from the Pull-Down Menu:• Reporting Category• Learning Standard• Individual Responses
Important!
• Regardless of what indicators you’re analyzing, Compare Student Group performance to ALL STUDENTS: Is it a “some students” issue or an “everyone” issue?
Other Interesting Views
• Learning Standard Description
Writing Composition Scores
How it’s Built
Other Tidbits…
• Use of the following Student Groups might help in PBMAS and Program Evaluation:
Index 3
• Index 3 Awards: • 1 Point for Students who Meet the
Standard• Index 1, Student Groups Reported
on Safeguards• 2 Points for Students Who Meet
Advanced Level III• Evaluated for Econ Dis and Up to 2
Student Groups• Greatest Opportunity to Raise Index 3?
• Advanced Level III Numerator!!!
Advanced Level III
• Use Your Aware Reports to Evaluate Advanced Level III Performance• Drill Down at Various Levels
• Campus, Grade, Course, Teacher, Reporting Category, Learning Standard, Individual Responses
• Compare Regular to Pre-AP Courses!!!
CRITICAL!!!
• State Reports DON’T Tell The Performance of ALL STUDENTS in Index 3.• Is it a “Some Students” Issue or an “All
Students” Issue?• Do Your 3 Student Groups Represent ALL
Students, or Only a Portion?
What Exactly Does Advanced Level III Mean?
• Students must score in the top 5-6 Raw Scores to Earn Advanced Level III
• Guiding Question:• Can my students successfully answer
the 5-6 most difficult questions on the test?
• We know how many questions the top performers got right, we need to ask….”Which ones did they miss????”
Analyzing Test Items For Students Scoring 70 or Greater
2 students who scored >70 on the test missed this question
Both chose answer C (incorrect)
These 2 students averaged a 70% on the test
How to Build It…
Index 4
• 4 Components• Graduation Rate• Diploma Plan• Post Secondary Indicators:
• ACT• SAT• TAKS (Compare 2013 College
Readiness to 2012)• FINAL LEVEL 2 IN 2+ Subjects!
Sample Index 4 Report for Post Secondary Readiness
This doesn’t tell you:Which Grades?Which Subjects?
How Can the Data Help Me?
• Begin thinking in terms of “How many questions can a student MISS and still earn Final Level 2?”
How Many Can I Miss?
ItemsPhase 1
Number You Can Miss
Phase 2 Number You
Can Miss
Phase 3 Number You
Can Miss
Final Level 2 Number You Can
Miss
Advanced Level 3 Number You Can Miss
Grade 3 40 19 16 12 10 5
Grade 4 44 21 17 13 10 6
Grade 5 46 21 18 14 11 6
Grade 6 48 23 19 14 12 6
Grade 7 50 24 20 16 13 8
Grade 8 52 26 22 17 14 8
English I 92 43 40 36 34 16
English II 93 43 40 37 34 14
Grade 5 44 18 15 11 9 4
Grade 8 54 24 21 17 14 8
Biology 54 34 30 25 21 9
Grade 8 52 26 22 17 14 9
U. S. History 68 40 35 29 24 13
Grade 4 44 21 19 16 14 7
Grade 7 72 33 29 24 21 11
STAAR Raw Score Conversion
Subject
Use the 70% Report to Identify the Hardest Questions on the Test!
• Have your students been challenged at that level consistently in class?
Conclusion
• The State Accountability System has many and varied components. Aware can help you understand the stories behind the numbers to positively impact 2015 ratings.
