hyde, justeen - evaluation, august 5, 2014

Justeen Hyde, PhDInstitute for Community Health

Harvard Medical School

If you don’t know where you are going, how will you know when you

get there?Project Planning and Evaluation

Workshop

Institute for Community Health

• Vision is to improve community health by uniting academic research with real world practice

• Utilize a participatory research and evaluation approach– Partnership and collaboration– Long-term commitment– Co-learning– Deep appreciation for diverse

experiences, practices, values, and resources

Institute for Community Health

Focus for Today• Overview of Evaluation– Purpose– Basic steps– Logic models and other visual depictions

Theory of Change

• Workshop your projects– Problem to be addressed– Underlying assumptions– Visual depiction of your Theory of

Change

What is Evaluation?

Evaluation is the systematic investigation of

merit, worth or significance of an object

In other words…

Addresses question of whether an initiative has been worthwhile in terms of delivering what was intended and expected?

What is Evaluation?

Can also…– Inform future program planning and

design– Provide lessons learned– Ensure transparency and accountability– Provide broader lessons learned about

good practice

Typical Participatory Evaluation Cycle

First step

• What is the problem you are trying to address?–What are your assumptions about why

this problem exists?–What are these assumptions based on?–Whose perspectives are

included/excluded?

The Importance of Assumptions

It’s a matter of reliability and validity

Access to and Retention in HIV Care in Metro Boston

• New HIV infections in the U.S. have remained stable for the last decade– Estimated 20% of people living with HIV

are unaware of their status– Estimated 20-30% of people with HIV

are not engaged in HIV care

• HIV is increasingly viewed as a chronic disease– Shift in thinking about appropriate

interventions for persons living with HIV

HIV as a Chronic Disease• Shift from preparing people to die to

helping people manage chronic disease

• Most people living with HIV in the U.S. are connected to care and on medication

• Focus on the 20-30% who are not managing their disease–What are the assumptions about this

group?

Possible Explanations• Limited access to quality care• Shame/stigma associated with HIV

infection• Problems with self management

associated with substance use or mental illness

• Fatalism• Others?

Importance of Assumptions

Project LEAP Assumptions• Poverty is a primary determinant of

health– HIV is one of many issues– Sense of overwhelm– Access to quality physical and mental

health care

• Social isolation associated with having a disease that is still highly stigmatized – Affects traditional sources of support– Suppression of identity to remain in

community

What is the Problem?

• Succinct statement about the problem your group is addressing

• What are your assumptions about why this problem exists?

• What informs these assumptions?• What perspectives may be missing?

Engage Stakeholders

Describe Program/Service/Intervention

Goals and Objectives

What is the Goal of the program, service, intervention, policy? – A broad statement that describes what you hope to achieve through your efforts

How will you achieve your goal? (Objectives) –Specific, measurable steps that can be taken to meet the goal

Be SMART

Examples of GoalsPoor

• To be healthier

• To change the world and make it a better place

SMARTer• To improve my health by

exercising at least 3 times per week and eating fruits and vegetables at every meal

• To cultivate international leaders who are committed to social change by creating an environment open to critical dialogue, providing hands on experience engaging in social problems, and building capacity to use different forms of media to communicate with

others.

Your group’s goals

• Write a succinct statement describing the goal of your project– Problem you seek to address– Target audience– Primary strategy or approach

Intended Effects

Outcomes• Specific changes in behavior,

knowledge, skills, status and level of functioning resulting from activities – Short-term outcomes – immediate

changes, often knowledge and attitudes– Intermediate outcomes – several months

to a few years, often skills, beliefs, behavior change

– Long-term outcomes – lasting changes that may take several years to realize

Logic Models• Graphic depictions of the relationship

between activities and intended effects– Inputs– Outputs– Outcomes– Impact

Workshop

Develop a logic model for your project• Inputs• Activities• Outputs• Outcomes– Short-term outcomes– Intermediate (if appropriate)– Long-term impact

“If you think you’re toosmall to be effective, youhave never been in bedwith a mosquito.”

Bette Reese