What Brought You Here?

Why You Should Measure Outcomes

What You Should Measure

How To Make It Happen

2

Why are you interested in this workshop?

What do you hope to take away today?

3

Program evaluation is carefully collecting and analyzing information about a program or some aspect of a program in order to make necessary decisions.

The type of evaluation you undertake to improve your program depends on what you want to learn.

Continual improvement is an unending journey. 4

Process Descriptions of what you do, how much you produce, and how you do it

OutcomesChanges that occur within the client, member or system as a result of your activities

5

6

External PressuresIncreased Public ScrutinyFunder AccountabilityEconomic Decline

Internal MotivationsContinuous ImprovementGreater Mission Focus

7

Direct service providers need to demonstrate results for the client or target populationAssociations must justify fees and retain membersSystems change nonprofits need to show changes in system capacity, investment or readinessFoundations must demonstrate impact across grants awarded or at the community level

8

9

Data can provide powerful evidence

Data can be objective and reduce bias

Balanced information provides the opportunity for independent interpretation

Data in existing tracking systems may be useful

10

Who are the possible audiences for your data?

What do they already know?

What do they want to know?

How will they use the data?

11

Possible Audience What Do They Already Know?

What Do They Need to Know?

How Will They Use the Data?

12

Exercise

Designing a good tool requires more time and attention than you may think.

Questions can tap into:• Characteristics• Opinions• Experiences• Knowledge• Attitudes• Behaviors

13

Process –activity/output/efficiencyNumber and type of volunteer hours provided, volunteers trained, workshops held, applications processed, etc.Example: Average contact time with participants

Outcomes – effectiveness/results/impactCan include change in clients’ level of awareness, knowledge gained, behaviors that have changed, etc. Example: Percentage of participants who demonstrate increased knowledge of effective parenting skills

14

What is a logic model? A logic model is a simple description of how your program, service, or initiative works that shows the linkages between:

Problem you are attempting to addressProgram componentsProgram activitiesOutcomes (both short- and long-term)

15

Useful for designing/developing your program or initiativeProvides a framework for evaluationProvides a focal point for stakeholders, requiring them to work together to identify the components of the model and to think about expected outcomes

16

17

Find a championUsually someone in leadershipIntegrate communication about the effort into operations

Select someone who can implement and propel the effort forward

Choose a person who has a compatible mindsetChoose a clear communicatorProvide professional development if neededGive authority that allows dedicated time

18

What outcomes will you measure?How will you define each outcome? What tools will you use to do so?Who is responsible for data collection?What are the timelines for doing so?What questions do you want to be able to answer at the end of the year?

19

SurveysPersonal interviewsFocus groupsOther data collections tools Existing databases

20

A quantitative research project in which a relatively large number of people are queried, each being asked a standard set of questions, posed in the same way each time.

21

To increase understandingTo collect specific, standardized data across respondentsTo collect data across many people as efficiently as possible To collect data quickly, even across multiple groupsOpen-ended quotes may add impact and credibility

22

An interview is the collection of data by asking people questions and following up or probing their answers.

23

To increase understandingAs an exploratory first step to creating quantitative toolsEnhance understanding of interesting findings which emerged from other processesReal-world quotes may add impact and credibilityTo collect specific, standardized data across respondents

24

A focus group is a group discussion. Participants are brought together in a neutral location for the specific purpose of discussing and issue or responding to ideas or materials of interest.

25

To increase understandingTo brainstorm or explore an idea you may wish to implementEnhance understanding of interesting findings which emerged from other processesTo generate ideas for possible solutions

26

ObservationsChecklists (process documentation)Attendance logsCase recordsExisting databases

27

Depends on the purpose – who can answer your questions?If not apparent, ask for guidance from key informants.If potential population is large, generate a list of candidates and key characteristics then randomly select.

28

Document your processes firstChoose the most simple solution that will get the job doneInclude leadership, data entry personnel and data analysts in these conversationsFocus on how data relates to reportingSoftware is not the silver bullet

29

Keep it simpleTake baby stepsBalance simplicity with precisionFocus on interpretationBlend methods if appropriate

30