summary of findings: reliability
DESCRIPTION
Summary of Findings: Reliability. Relationship between SPS Overall Percent Favorable and Overall Evaluation Rating . What about the survey elements? . Classroom Community. Classroom Management. Student Learning. Student-Centered Environment. Summary of Findings - Validity. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/1.jpg)
Summary of Findings: Reliability
Student-Level Reliability (α)Grades 3-5 Grades 6-12
Overall Reliability (all items) 0.94 0.96Student Learning 0.90 0.94Students-Centered Environment 0.86 0.90Classroom Community 0.80 0.86Classroom Management 0.75 0.80
Teacher-Level Reliability (α)
Grades 3-5 Grades 6-12Overall Reliability (all items) 0.97 0.98Student Learning 0.95 0.97Students-Centered Environment 0.94 0.96Classroom Community 0.90 0.94Classroom Management 0.90 0.91
![Page 2: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/2.jpg)
Relationship between SPS Overall Percent Favorable and Overall Evaluation Rating
![Page 3: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/3.jpg)
What about the survey elements? Classroom Community Classroom Management
Student-Centered Environment Student Learning
![Page 4: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/4.jpg)
Summary of Findings - Validity
• Analyzed 14,539 open-ended responses (3-5 and 6-12)
• We find that not only were the majority of students taking the survey seriously, but that many of the responses were specific and actionable in nature– 98.6% (N=14,341) were considered substantive – 66.3% (N=9,646) were coded as actionable – Moreover, although some subjects and grades were slightly
more likely to garner actionable feedback, in general actionable responses came from students in all grades and subjects
![Page 5: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/5.jpg)
What Students are Saying About Teachers
Overview of Student Feedback for Teachers
![Page 6: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/6.jpg)
Guiding principles for administration
• We know that teachers care about their practice and especially about their students. – They also may experience nervousness and anxiety about the
surveys, and district and school leaders must address those fears.
• By itself, a reliable and valid instrument does not ensure that teachers will receive good feedback.
• Messaging matters! – Engage stakeholders early and often– Make the process as transparent as possible
• Give stakeholders real decision-making power
![Page 7: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/7.jpg)
Engaging Stakeholders
• Forming a planning committee• Building educator investment• Building student understanding and comfort• Informing other stakeholders
![Page 8: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/8.jpg)
Key Decisions
• Use a vendor or manage administration internally?
• Online or paper/pencil survey administration?• When will the survey be administered? • How will students and teachers be sampled?• Include an open-ended question on the survey?• How will survey results be used?
![Page 9: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/9.jpg)
Key Decision: How will survey results be used?
• Consider a “hold-harmless” pilot year• As a formative tool
• Reflect on individual practice• Identify over-arching trends and create strategies to address
them. • As a part of an evaluation
• Used as an artifact for determining ratings for professional practices
• Included as one of several multiple measures • Sharing teacher-level results
![Page 10: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/10.jpg)
Contact Information
Colorado’s Student Perception Survey Website:www.colegacy.org/studentsurvey
Amy Farley, Director, Research and Impact720-502-4723
![Page 11: Summary of Findings: Reliability](https://reader035.vdocuments.mx/reader035/viewer/2022062310/568161c1550346895dd1a332/html5/thumbnails/11.jpg)
Questions?Elaine Allensworth, Consortium on Chicago School [email protected]
Amy Farley, Colorado Legacy [email protected]
Kendra Wilhelm, Denver Public [email protected]
@AYPF_Tweets#aypfevents