starboard communications & grants

STARBOARD COMMUNICATIONS & GRANTS

MEASURING YOUR PROGRAMS

Why spend time and effort?

Outcome Measurement

Outcome measurement involves: the identification of how the program benefits

clients (how – in what way); development of appropriate outcome indicators and

data collection procedures (who collects the data); data analysis to better understand organizational

achievements (why services benefit clients) ; User-friendly, regular reporting of the findings.

Outcome Measurement

Outcomes measures are important because: adds credibility to program/services; supports fundraising efforts; drives allocation of resources (staff & program

funding); builds case for support; necessary to secure grant funding; and helps link budget directly to outcomes.

Types of Evaluation

Formation or process evaluation: focus on information for program improvement, modification and management.

Summative or impact evaluation: focuses on determining program results and effectiveness (merit and worth). Serves the purpose of making major decisions about program: continuation, expansion, reduction, and funding.

How does an organization get started? What to measure? Think in terms of how clients/customers benefit

(in one sentence) examples: families secure safe housing. students learn a new skill. kids grow-up tobacco free.

Getting Started

Key Steps

Set-up: what programs to include; determine who will be involved in developing the

process and how; establish an overall schedule; and define the background, purpose and methodology

of the program.

Key Steps

Deciding what and how to measure: identify the program’s mission, objectives and clients; identify the outcomes of the program; select specific indicators to measure the outcomes;

select data sources and data collection procedures for each indicator;

identify key client and service characteristics to link to outcome information; pilot test the procedures, make modifications and implement.

Key Steps

Analyzing the data: examine the outcome data; report the findings; seek explanations for unusual or

unexpected findings.

Key Steps

Using the results: use the outcome information to improve services

and report findings to stakeholders and others; capture impact of outcomes in marketing material

and grant applications.

Linking program measures to outcomes (e.g. your program results).

Measuring Your Program

How do your clients/customers benefit? Show the value of the program:

e.g. 25% of students learned a subject. e.g. 50% of students improved their test

scores. e.g. 10% of adults decreased their smoking

frequency.

Benefits of Measurement

Benefits of Measurement

direct decision-making and accountability responsibilities; and

provide evidence that an organization’s resources have improved the lives of clients through program effectiveness, cost-effectiveness, efficiency, or quality.

Benefits of Measurement

Operate and manage those resources in a way that maximizes effectiveness in helping clients; and

Help managers and board members become learning organizations that constantly improve the effectiveness of their services, identifying specific steps and providing suggestions for examining and using the outcome information.

Define measurement objectives and outcomes; Integrate measurement into the program; Establish a cost/benefit analysis before beginning

the project.

Ongoing Program Measurement

How do I begin a cost benefit analysis? describe individual activities and elements of

each program; quantify activities in terms of time and

personnel (and/or resources used); list program functions (areas); and establish program budget categories for each

activity area within the overall budget.

Cost/Benefit Analysis

Cost/Benefit Examples:

Assigning quantifiable measures to program activities such as:

number of hours spent per client; categories of activities within a program; and/or client satisfaction in services provided.

Outcome Measurement Strategy:

identify broad outcome areas; assign programs to areas; create program outcome statement for each

area; define outcomes (goals & objectives); and use financial data to show #’s served,

resources used & links to outcomes.

"Overall Referral Satisfaction" J anuary - J une 2009

89.3 87.591.7 94.4 91.7

87.5 87.5

100.0 100.0

0.0

25.0

50.0

75.0

100.0

Programs

Mean

Sco

re

Outcomes

OUTCOMES FROMPERSEUS HOUSE, INC:RESIDENTIAL AND

EDUCATIONAL SERVICES

CICTP OutcomesSuccessful Completion of Program

Total N=763

YesN=514 67.4%

NoN=24932.6%

CICTP OutcomesRecidivism Rate

(Clients Who Successfully Completed Program)

Missing=104

YesN=6015.0%

NoN=35085.0%

Residential OutcomesHistory of Residential Placement(Clients Who Successfully Completed Program)

N=106

YesN=8176.4%

NoN=2523.6%

CICTP OutcomesRecidivism Rate

(Clients Who Successfully Completed Program)

Missing=104

YesN=6015.0%

NoN=35085.0%

CICTP OutcomesPrior History of Residential Placement

Missing=2

YesN= 32042.0%

No N=441 58.0%

CICTP OutcomesHistory of Residential Placement(Clients Who Successfully Completed Program)

N=320

YesN=23272.5%

NoN=8827.5%

CICTP OutcomesPrior History of Residential Placement

Recidivism Rates(Clients Who Successfully Completed Program)

Total N=232 Missing 29

NoN=16681.8%

YesN=3718.2%

CICTP OutcomesGrade Point Average

2.5

2.9

0

1

1

2

2

3

3

4

Admit Exit

N=511+0.4 Significant

CICTP OutcomesSocial Skills Scores

148.2 141.9

179.6164.1 160.1

167.9

0

20

40

60

80

100

120

140

160

180

200

Client Family Trainer

Pre Post

N=431+16.0 Significant

N=462+11.7 Significant

N=511+18.2 Significant

CICTP OutcomesAggression Testing Scores

56.753.3 52.2

54.250.5

52.553.451.3 49.5

52.348.2

50.5

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Total Physical Hostile Verbal Indirect Anger

Pre Post

N=171-2.0 Significant

N=171-3.3 Significant

N=171-2.7 Significant

N=171-2.3 Significant

N=171-2.0 Significant

N=171-1.9 Significant

Residential OutcomesSuccessful Completion of Program

Total N=217

YesN=17580.6%

No, N=4219.4%

Residential OutcomesRecidivism Rate

(Clients Who Successfully Completed Program)

Missing=87

YesN=9

10.2%

NoN=79

89.8%

Residential OutcomesPrior History of Residential Placement

YesN=10648.8%

No N=11151.2%

Residential OutcomesHistory of Residential Placement(Clients Who Successfully Completed Program)

N=106

YesN=8176.4%

NoN=2523.6%

Residential OutcomesHistory of Residential Placement

Recidivism RatesN=81 Missing=34

YesN=7

14.9%

NoN=4085.1%

Residential Outcomes Grade Point Average

1.9

2.7

0

1

1

2

2

3

3

Admit Exit

N=174+0.8 Significant

Residential Outcomes Social Skills Scores

169.5

143.7130.5

187.2172.5

164.1

0

20

40

60

80

100

120

140

160

180

200

Client Family Trainer

Pre Post

N=77+28.8 Significant

N=171+17.7 Significant

N=172+33.6 Significant

Residential OutcomesAggression Testing Scores

52.249.2

52.253.254.2 52.752.248.1 49.7

53.148.8 50.6

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Total Physical Hostile Verbal Indirect Anger

Pre Post

N=130-5.1 Significant

N=130-2.0 Significant

N=130-2.5 Significant

N=130-0.4 Not Significant

N=129-1.5 Not Significant

N=130+0.4 Not Significant

45

31

1112

31

23

10

4 0 0

34

51

0.0

10.0

20.0

30.0

40.0

50.0

60.0

Num

ber

of H

olds

AH RTF AH ITU PH RTF PH ITP Flo Crit Shelter

Time Period

"Therapeutic Holds"

July 1, 2003 - June 30, 2004 July 1, 2004- June 30, 2005

Sample using pre and post tests

Sample using numbers served:

Sample using time study:

STARBOARD COMMUNICATIONS & GRANTS

Starboard Communications & Grants offerscomprehensive communications, grant anddevelopment services specifically for the nonprofitand government sector.

Starboard Communications & Grants is a jointsocial enterprise between Perseus House, Inc.,and Safe Harbor Behavioral Health.

A social enterprise is an earned-incomebusiness or strategy undertaken by a nonprofitfor the purpose of generating revenue insupport each nonprofit’s social mission.

Starboard:

improves the capacity of nonprofit organizations to effectively seek & secure grant funding for operating and/or capital needs;

improves planning, marketing & communications to initiate greater public support; and

increases capacity of non-profit development functions.

Starboard:

Grants and research services. Marketing and development services. Outcomes measurement services.

Other grants: 7.3 million:

USDA telemedicine grant for visiting nurse agency. SAMHSA grant for improving mental health svcs. Robert Wood Johnson Foundation grant for

nursing outreach program. Ronald McDonald House Charities provided

neonatal equipment. Tax credits benefited capital campaign initiatives. Susan Komen Fdn. funded cancer education &

outreach.

Questions?

STARBOARD COMMUNICATIONS & GRANTS

1330 West 26th St.Erie, PA 16508814-451-2206www.starboardgrants.org