nh responds evaluation component

Pat Mueller David Merves October 6, 2008 NH RESPONDS Evaluation Component

Upload: tudor

Post on 25-Feb-2016




0 download


NH RESPONDS Evaluation Component. Pat Mueller David Merves October 6, 2008. Why We Collect Data/Evaluate?. Somebody said you had to Inform instruction School improvement Local, state, and federal accountability Public information Choose/set policy Marketing - PowerPoint PPT Presentation


Page 1: NH RESPONDS Evaluation Component

Pat MuellerDavid Merves

October 6, 2008

NH RESPONDSEvaluation Component

Page 2: NH RESPONDS Evaluation Component

Why We Collect Data/Evaluate?

Somebody said you had toInform instructionSchool improvementLocal, state, and federal accountabilityPublic informationChoose/set policyMarketingBecause that’s what all the cool kids are doing…

Page 3: NH RESPONDS Evaluation Component

State Personnel Development Improvement Grants (SIG/SPDGs)

SIG/SPDGs are measured against…

OSEP program performance measures

NH RESPONDS performance measures

Page 4: NH RESPONDS Evaluation Component

SPDG Program Performance Measures

% of personnel receiving professional development (PD) on scientific-or evidence-based instructional practices.

% of projects that have implemented PD/training activities that are aligned with improvement strategies in the State Performance Plan (SPP).

% of PD/training activities provided that are based on scientific-or evidence-based instructional/behavioral practices.

Page 5: NH RESPONDS Evaluation Component

SPDG Program Performance Measures

% of PD/ training activities that are sustained through on-going and comprehensive practices (e.g., mentoring, coaching).

% of SPDG projects that successfully replicate the use of scientifically based or evidence-based instructional/behavioral practice in schools

Page 6: NH RESPONDS Evaluation Component




Steps for Conducting an Evaluation

Page 7: NH RESPONDS Evaluation Component

1. Define the Criteria to Be Evaluated

Terminology Goals Long-Term Outcomes or Impact

Objectives Short-term & Intermediate Outcomes

Activities Outputs

Page 8: NH RESPONDS Evaluation Component

Two Types of Evaluation Standards

Process/Formative: Assesses ongoing project activities

Begins at program implementation and continues throughout program

Is the program being delivered as planned?

Is the program progressing towards its goals and objectives?

Page 9: NH RESPONDS Evaluation Component

Two Types of Evaluation Standards

Outcome/Summative: Assesses the program’s success and whether the program or initiative had an impact.

Compares the actual results to projected goals/objectives.

Typically used for decision making purposes

Important to look for unanticipated outcomes

Page 10: NH RESPONDS Evaluation Component

2. Logic Models

A conceptual model that links an initiative’s goals and objectives, with expected outputs and/or outcomes.

Numerous types of logic models.

There are many other methods of illustrating the conceptual framework of an initiative.

Page 11: NH RESPONDS Evaluation Component

3. Writing an Evaluation Plan

Components to include: Program goal/objectives

Evaluation questions

Performance indicators

Data collection procedures

Data analysis method

Person responsible


Page 12: NH RESPONDS Evaluation Component

4. Collecting Data

Process/formative data Amount and type of PD provided Satisfaction and utility of PD provided Products developed These data tend to be gathered by those providing PD

Outcome/Summative data Reduced office discipline referrals Reduced suspensions/expulsions Improved reading scores These data tend to be collected from the LEA or SEA

Page 13: NH RESPONDS Evaluation Component

NH RESPONDS Data Collection Tools

PD Activity Log completed by TA/PD providers

Minutes Leadership Team meetings Workgroup meetings School & District Improvement Team minutes/products

Surveys/Interviews/Focus groups Annual Participating Personnel Survey (March/April) Workshop surveys

Page 14: NH RESPONDS Evaluation Component

NH RESPONDS Data Collection Tools

Fidelity instruments (for PBIS & Literacy) Benchmarks of Quality

School-wide Evaluation Tool

Existing data Office Discipline Referrals

Suspension/expulsion data

Reading scores

Page 15: NH RESPONDS Evaluation Component

5. Analyzing Data & Understanding Results

SAU/School-Level Analysis of student performance to improve

instruction (i.e. reading scores)

Analysis of school-level data to improve safety and/or climate (i.e. SET)

Project/Grant Level Analysis of formative data for program improvement


Aggregate analysis of all outcome data to describe impact of NH RESPONDS

Page 16: NH RESPONDS Evaluation Component

6. Communicate the Findings (Reporting)

Provide on-going feedback to project management (what’s working/what’s not).

Provide on-going data to the NH Bureau related to completion of objectives and success of project efforts.

Provide annual report to the U.S. Department of Education related to completion of objectives and success of project efforts.

Page 17: NH RESPONDS Evaluation Component
Page 18: NH RESPONDS Evaluation Component

[email protected](802) 434-5607


Evergreen Educational Consulting