telling the whole story alternative accountability in portland, oregon prepared for the alternative...
TRANSCRIPT
Telling the Whole StoryAlternative Accountability in Portland, Oregon
Prepared for the Alternative Accountability Policy Forum
November 2015
About usChristopher Mazzeo, Ph.D.Director of Evidence Use and Policy Education NorthwestDirector, REL Northwest
Carla Gay, MSW Vice Principal, Reconnection ServicesMultiple Pathways to GraduationPortland Public Schools
Matt EideSenior AdvisorEducation Northwest
Goal
Share Portland’s experience, and lessons learned on the development of multiple
quantitative and qualitative accountability
metrics for alternative schools.
Guiding Question
How do you determine what quality looks like in an alternative program?
Discuss this question with a neighbor
Just enough context…
The District Perspective
Nagging question…How do we get accurate data to
represent the work of our alternative schools and programs?
But it was all below the line
The District Perspective
Confluence of Events in 2012 brought the nagging question above the line. . .
National Youth Taskforce
Oregonian Article Budget Cuts
The District Perspective
With the confluence of events, developing an accountability
framework for alternative education options became an above the line
priority…
The Alternative Schools’ Perspective
Will the metrics tell the whole story?Can traditional metrics accurately determine quality programming and achievement in alternative settings?
We don’t want your cookie cutter!Alternative programs in Portland are strong because of their diversity and specificity!
But we believe in quality too.
The REL Perspective
Help move the work forward
An issue of equity An
important issue in our
region
What we didand what we learned…
The Portland Approach
Quantitative
Site VisitsQualitative
How did we do this?
JFF pointed us in the direction of Denver and Chicago school districtsDenver helped us map our process for a district wide segmentation analysisChicago shared their accountability templateWe convened a group of stakeholders including policy, research and providersWe convinced REL Northwest to facilitate. . .
Your turn…
Use your dots to indicate which three metrics would be best to determine what quality looks like in an alternative program.
The Portland Framework
Put it here
METRIC DESCRIPTION
Academic ProgressSKILL GROWTH Percent of students who meet or exceed growth targets in
Reading and Math on either MAP or CASAS
CREDIT ATTAINMENT Percent of students who meet targets for the number of credits earned for length of enrollment or earn their maximum required credits while in school
Successful CompletionONE YEAR HS GRADUATION RATE Percent of students eligible for graduation who graduate within
one yearCOLLEGE READY GED ATTAINMENT RATE
Percent of students who meet/exceed GED target composite score
POSTSECONDARY READINESS Percent of students who meet/exceed target college readiness scores on COMPASS or ACT
School ConnectionAVERAGE DAILY ATTENDANCE Percent of days attended by students enrolled at the school
GROWTH IN ATTENDANCE Percent of students that show growth in their individual daily attendance rates compared to their individual daily attendance rate in the previous school year
ANNUAL RETENTION RATE Percent of students enrolled at an alternative school and retained from the point of enrollment to the end of the year
SCHOOL CLIMATE TBD
The Report Card
What we learnedHave a third party facilitate the process and analyze the data
Make sure to attend to technical and leadership considerations
Collaborative process
Include state policymakers in the process
Programs need access to real-time data
The Dashboard
What’s missing?
Are these metrics sufficient to reliably determine alternative program quality?
Discuss with your neighbor
Telling more of the story
through qualitative data
Qualitative ProcessThe Process
Seven step iterative process:
– needs sensing, instrument creation, instrument feedback collection, instrument revision, pilot testing, data analysis and reporting.
Piloted focus groups and interviews with three alternative programs
– Administrators, teachers, students, and parents
Qualitative Process
The InstrumentFocus group and interview questions aligned with PPS successful schools framework
The ResultsEvaluation briefs highlighting program context
Qualitative Process
Lessons Learned
Challenging to schedule visits and interviews
Aligning protocol with PPS Successful Schools Framework was useful but limited findings
Student focus groups should include alumni and students who are struggling
Qualitative Process
Lessons Learned
The qualitative review process is labor intensive and time consuming
Data triangulation and member check
De-islandize alternative education
Include as many stakeholders as possible
Questions?
