12/6/2013

Rebecca Rodriguez

For the Philanthropist 1

Outcome Measurement Determining & Selecting Outcomes

A General Framework

12/6/2013 2

Rebecca Rodriguez

For the Philanthropist

Agenda

Purpose & Importance Literature Review How To implement Outcomes Logic Models Data Collection Methods Recommendations References

12/6/2013 3

Retrieved from the United Way of

America web site:

http://www.liveunited.org/outcomes

Introduction

Outcome Measurement is the regular, systematic tracking of the extent to which program participants experience the benefits or changes intended.

- United Way of America

12/6/2013 4

Outcomes for Success! Organizational

Research Services, Inc. & Clegg and

Assoc., Inc.

The observance, tracking and analysis

of changes to participants in a program.

Measurable & meaningful.

Directly linked to program activities.

Useful for program improvement.

Digging Deeper…

12/6/2013 5

Outcomes for Success!; United Way of

America, Center for What Works

Importance of Outcome

Measurement

Accountability, Responsiveness, Efficiency

Helps staff make informed decisions.

To learn that your efforts are helping and

not harming.

Helps organization learn to tell “their story”

effectively.

Can support fundraising efforts.

12/6/2013 6

Outcome Measurement is NOT

Formal evaluation.

The end all and answer to all our

program questions/ problems.

A one-size- fits all solution.

A quick and easy fix.

12/6/2013 7

Literature Review

Plethora of information that falls into two

main areas:

- Importance of

- Implementation Process

Common elements:

- Complexity; general lack of understanding

- Sharing of best practices

12/6/2013 8

Center for What Works;

www.innonet.org; Outcomes for

Success!; Instituting an Outcomes

Assessment Effort

Implementation Steps

1. Build consensus – bring everyone to the table.

2. Start with a logic model; talk about theory of change.

Look at best practices already working!

3. Determine & select outcomes.

4. Determine data collection methods.

5. Collect, record, analyze data.

6. Reflect, talk about, brainstorm.

7. Make adjustments to program if necessary.

8. Share your findings.

12/6/2013 9 Outcomes - United Way of America

Skills

Status

Awareness

Behaviors

Attitudes

Knowledge

Outcomes to Measure

12/6/2013 10

Adapted from the Logic Model

Development Guide found at

www.kellogg.org

Logic Model Template

Planned Work Intended Results

Resources/Inputs Activities Outputs Outcomes Goal

12/6/2013 11

Personal Communication with The

Lewin Group

Time from referral to service – status

Provider quality – knowledge, behavior, skill, attitude

Assisting consumers planning in advance for long-term care – knowledge, skills, attitudes, behavior

Positioning AAAs as the single point of entry for long-term care – knowledge, awareness

What Area Agencies on Aging

Measure

12/6/2013 12 Adapted from Outcomes for Success!

Data Collection Methods

Reliability vs. Validity

0102030405060708090

Sur

veys

Inte

rviews

Obs

erva

tions

Case

Reco

rds

validity

reliability

12/6/2013 13

Rebecca Rodriguez

For thePhilanthropist

Elements for Success

Build consensus.

A strong desire to demonstrate impact.

Willingness to make change.

Be disciplined in collecting / analyzing

data.

Keep it simple.

12/6/2013 14

Recommendations

Proposed 18 - 24 month timeline:

Build consensus.

Assign an outcome measurement driver.

Start with a logic model.

Agree upon one/two outcome(s). Start simple!

Choose method of data collection: surveys or interviews

Collect, record, analyze data.

12/6/2013 15

Resources

www.whatworks.org

Collins, J. (2005). Good To Great And The Social Sectors. Boulder, CO: Jim Collins.

www.grantcraft.org

W.K. Kellogg Foundation

United Way of America. (2000). Agency Experiences with Outcome Measurement.

Virginia: United Way of America.

www.innonet.org (registration required)

Edna McConnell Foundation

Charting Impact – Initiative of Guidestar collaborative effort

American Evaluation Association