new mexico principal support network helping leaders use accountability data effectively

New Mexico Principal Support Network

Helping Leaders Use Accountability Data Effectively

Presented By:

Beata I. Thorstensen, NM Office of Education Accountability

Jan Sheinker, Sheinker Education Services

Judy Englehart, Aztec Municipal Schools

Tania Prokop, Aztec Municipal Schools

Presented At: CCSSO’s Education Leaders Conference,

September 12-14 2007

Hilton St. Louis at the Ballpark

NM Principal Support Network: Began in 2005 with 40 principals & assistant

principals in 8 districts. Expanded in 2006 to superintendents,

principals, assistant principals and district staff from 28 districts.

Expanding again in 2007 to 43 districts. Purpose: to provide comprehensive

assessment data, data analysis tools and professional development on data-based decision making.

Funded through the generous support of The Wallace Foundation.

What the PSN Does:

Provides a network of training and support for assistant principals, principals, superintendents and other school district leaders.

Focuses on the analysis, interpretation and use of high stakes accountability data.

Goal: to help educational leaders use data for school improvement, communication and advocacy.

PSN Curriculum:

Data Analysis and UseAnalyzing high stakes assessment data to

pinpoint areas for school improvement.Communicating data to school boards,

superintendents, teachers, students and the community.

Using evidence in conjunction with best practices for data-based decision making.

PSN Curriculum:

Data-Based Decision MakingProviding best practice information on the

facets of data-based decision making Informing comprehensive school improvement

plans utilizing New Mexico’s Educational Plan for Student Success (EPSS).

PSN Curriculum:

Connecting with PeersMembers work with peers both within their

district and outside of their district to: Analyze student data Build comprehensive school improvement plans Share promising practices for interventions

Outcomes of PSN: Data Use Used to communicate data with district staff,

teachers and the community. Used to facilitate visits with school improvement

teams from the Public Education Department. Used to inform the development of the statewide

data warehouse reporting system. Used to inform decisions about

instructional/curriculum interventions.

PSN Membership & Format

Superintendents, Principals, Assistant Principals and other district data staff.

Focus on Schools in Need of Improvement.

3-4 meetings per year.

Excel Pivot Tables for District & School Level

Data Analysis:

Assessing Student Performance Beyond the Report Card:

Allowing Drill-Down to Student

Level

Example of Student Level Drill-Down:

Example of Data Display: Simple Bar Chart

Logging into NM DBDM

Surfing NM DBDM

Start Anywhere

Purpose of DBDM for PSN

A tool for leaders (helping their schools): Multiple entry points for differentiated support Specific, real time explanations, examples, links,

school stories, and resources On-going job-embedded assistance Continuous monitoring, collaboration, and feedback Continuous data-based school improvement

system into routine practice

Establish a school improvement team What is a school improvement team?

How do we establish a school improvement team?

Who is on our team? What do we do?

How does the school improvement team make time to do its work?

Develop a hypothesis

What information does our school or district need to make decisions that will improve student achievement?

How is our school doing compared to the standard?

Develop a hypothesis 

What information does our school or district need to make decisions that will improve student achievement? Baseline – What do you already know about your school? •What learning strengths and weaknesses are evident in the school data? •Which subgroups of students are having difficulty learning? •What instructional changes might improve student learning in the areas of weakness? •What professional development is needed to improve student learning in the areas of weakness? •What materials and equipment are needed to support changes in instruction?

How is our school doing compared to the standard? Baseline – What do we think we know about how we are doing?•Overall school results. •Disaggregated by gender. •Disaggregated by disability status. •Disaggregated by ethnicity. •Disaggregated by English proficiency. •Disaggregated by income status. •Disaggregated by migrant status.  

Gather data to assess needs

What are the most useful sources of student data?

Why use multiple measures?

What are the most useful sources of direct student achievement data?

What are the most useful sources of indirect student achievement data?

What are the most useful sources of subgroup student achievement data?

What are the most useful sources of demographic data?

How do context variables impact the validity of our interpretation?

What do we have? What do we need?

Gathering Data Direct Measures

NMSBA– From Pivot Tables Other Assessments –DIBELS, Short-cycle, etc. Teacher Made Assessments

Indirect Measures Attendance and graduation rates Information About Curriculum

Demographics Ethnicity and race proportions Gender proportions Socio-economic percentages Language status proportions Disability status percentages Migrant status proportions

Other? Subgroups Class size Teacher training Student mobility

Use data How do we organize the data to help us answer important questions?

What do different sources tell us?

What do different displays tell us?

How do we display the data?

What patterns exist in the data?

How do we present data to the school and examine it?

What are the tests designed to measure?

Is there confirmation across data?

How should we present data and conclusions to the school community?

How do we formulate data-based goals?

Does our interpretation raise new questions?

What is our level of confidence in our interpretation?

Use data How do we organize the data to help us answer important questions?

What do different sources tell us?

What do different displays tell us?

How do we display the data?

What patterns exist in the data?

How do we present data to the school and examine it?

What are the tests designed to measure?

Is there confirmation across data?

How should we present data and conclusions to the school community?

How do we formulate data-based goals?

Does our interpretation raise new questions?

What is our level of confidence in our interpretation?

Formulating Data Based Goals

When preparing to set goals based on the data, schools clarify the results to determine the area of greatest concern for setting one or two important goals for improvement. The school: uses the information about differences in achievement across content

areas (reading, writing, mathematics) to pinpoint the goal for improvement

uses the clarification of specific standards (basic reading, reading comprehension, math computation, geometry, math problem solving) or benchmarks to help plan the strategies and interventions

The goals for improvement are: specific to the content area of greatest concern (reading, writing,

mathematics) sometimes related to strategies for improving specific standards

(reading for comprehension, use literature and media) or benchmarks (Reading for information, Reading strategies, Literature)

Use data How do we organize the data to help us answer important questions?

What do different sources tell us?

What do different displays tell us?

How do we display the data?

What patterns exist in the data?

How do we present data to the school and examine it?

What are the tests designed to measure?

Is there confirmation across data?

How should we present data and conclusions to the school community?

How do we formulate data-based goals?

Does our interpretation raise new questions?

What is our level of confidence in our interpretation?

Does our interpretation raise new questions?

Which specific standards or benchmarks are students farthest from achieving? Impact of benchmarks on overall standard

Which specific subgroups of students are failing to achieve the standards? Impact of a benchmark on scores for all subgroups

How have past changes affected student performance? Data over time

What is our level of confidence in our interpretation?

• Error of measurement. Distance from cutpoints. Reliability of scores/proficiency classification.

• Different scores on two reading tests. Differences in what is measured. Differences in how it is measured. Differences in degree of alignment with standards.

• Different results for state and local tests. Differences between SBA and short-cycle results. Differences is specificity. Differences in sample size.

Develop a data-based plan What must be considered when setting data-based goals?

How do we set data-based goals?

How can additional data help us identify the interventions we need?

How do we select interventions?

How do we select interventions for targeted subgroups?

How do we plan to include parents in interventions?

What staff development and support are necessary?

How does our plan impact our budget?

What is our time line?

What assignments are necessary?

Develop a Plan

Is the goal cleargoal clear to everyone? Do strategies/activities address specific data-based needsaddress specific data-based needs? Are the activities specific and sequentialactivities specific and sequential? Are persons responsible statedpersons responsible stated (who will do what, when, how)? Are resources specifically statedresources specifically stated? Are the selected intervention scientifically research intervention scientifically research

based/research provenbased/research proven? Are task and evaluation deadlines clearly statedevaluation deadlines clearly stated? Do specific professional developmentspecific professional development activities support full

implementation of the strategies and interventions? Are activities to involve parents and community specificparents and community specific to the

strategies and interventions to be implemented? Is evidence of completion specifically identifiedevidence of completion specifically identified?

Monitor progress and document success

How do we monitor implementation of the plan?

How do we use data to monitor progress toward our goals?

How do we know if we made the right decisions?

How do we use data to document success in meeting goals?

What should we report to the public?

Other information about purpose and use Explanations and search tools

overview “how to” guide for various audiences glossary of terms annotated bibliography of school

improvement publications key work search

Links to State customized version on-line data and school improvement

resources examples of school, district, and state

data use State school improvement documents

Password protected school accounts

for tracking progress on improvement implementation.

for on-time interactions with state support teams, district support personnel, and technical assistance providers.

Avoiding Pitfalls: Common Reasons Why Plans FailWhy Plans Fail

The plan is never fully implementednever fully implemented. Timelines are not metTimelines are not met for each activity. Interventions are not evidentInterventions are not evident in all

classrooms. Tasks are notTasks are not all completed on time. Resources are not acquiredResources are not acquired or deployed in

accordance to the plan. Next steps are not articulatedNext steps are not articulated.

New Mexico Principal Support Network

Data Based Decision Making Website found at: http://www.edvantia.org/dbdm

Office of Education AccountabilityNew Mexico Department of Finance & Administration

Contact Information: 505-476-1070www.nmdfa.state.nm.us