Does Health Promotion Work?

Puja Myles

Outline of lecture

• Learning Outcomes • Designing a health promotion intervention• Types of evaluation questions and

perspectives• Outcomes• Special evaluation methods• A generic evaluation framework• Practical exercise

Learning Outcomes

At the end of this session you should be able to:• Design a health promotion intervention using

previously covered theoretical frameworks and concepts

• Frame an evaluation question for a given health promotion intervention

• Plan an evaluation for a given health promotion intervention using appropriate methods and outcomes

Designing a health promotion intervention/programme-1

• Identify priorities for action: National priorities/targets Health Needs Assessment Health Equity Audit

• Targeted vs. population approach• Decide Intervention

- Evidence on effectiveness and cost-effectiveness

Designing a health promotion intervention/programme-2

• Decide programme goals

• Monitoring*

-what will you monitor?

-how often?

• Evaluation*

The need for the evaluation

Why do you want to conduct an evaluation:• How well are we doing?• Are we doing things right?• What difference are we making?

Accountability• Has the intervention worked?• Has the money been well spent?• Should we continue to invest in this project?

Efficacy, effectiveness and efficiency*

• Efficacy- does an intervention work in ideal conditions?

• Effectiveness-does an intervention work in real life?

• Efficiency-ratio of useful output to total input; incorporates the notion of minimising waste.

Evaluation perspectives-1

• Developmental/formative

• Process

• Summative

-Outcome evaluation*

-Impact evaluation*

(Ovretveit, J.)

Evaluation Perspectives-2

• Many perspectives • The number of perspectives directly

proportional to number of stakeholders• Depending on perspective adopted,

evaluation question, design and outcomes will change

Health Outcomes

Influence of Health Perspectives 1

Biomedical Model of Health• Health = Absence of disease(pathology)• ‘Functionalist’ approach• Objective, numerical measurements

based on ability to function• Preconceived assumptions about

individual perceptions of health

Influence of Health Perspectives 2

Social Model of Health• ‘Disease’ and ‘Illness’ different concepts• WHO definition of Health• ‘Hermeneutic’ approach to measure

aspects of health unique to individuals• Open-ended questions and qualitative

Qualitative versus Quantitative

• Quantitative- quantifiable- reliable,- generalisable (? possibly)- Decontextualises processes

• Qualitative-provides context -understanding of the ‘whys’-generalisability is a major issue

Health Outcomes- examples

• Health Behaviours• Adherence to treatment• Specific Diseases• Depression• Emotional balance or well-being• Global health status using SIP or SF-36• Self-reported health status

Health Outcomes?

• Self-confidence• Self-efficacy• Social involvement• Sense of achievement/direction• Stress relief • Participation in social networks

Special evaluation methods

• Audits

• Health Equity Audits

• Economic Evaluations

Audit

• Where are we in relation to the Gold standard?

• Audit loop or cycle

Health Equity Audit

• Health Equity profiling compares how the relationship between health need and service provision/use varies across the different dimensions of equity

• Equity profiling is just one step in the health equity audit cycle: re-audit to close the loop!!

Economic evaluation

What is the question?

• Is this activity worthwhile?

• Is this an efficient way to achieve a particular outcome?

• Are the health benefits of this activity justified?

• What is the least cost way to achieve a given outcome?

Allocative efficiency vs. technical efficiency

• Allocative efficiency cost-benefit analysis cost-utility analysis cost-consequences analysis

• Technical efficiency cost-effectiveness cost-minimisation

A good economic evaluation:

• Were both costs and effects of the services/programmes considered?

• Comparison of alternatives

• Statement of perspective from which the evaluation is conducted

A dummies guide to…

…bringing it all together

Applying the PT-DES approach

The Healthy learning Project

• Multi-agency partnership: local authority, education, health, voluntary sector

• Learning advisors in GP surgeries• Referrals by: health professionals, self• Information/advice on learning, careers,

leisure, skills (including life skills), lifestyle

PT-DES:Step 1

‘Involving stakeholders and developing programme theory’

• Identify Relevant Stakeholders• Develop common understanding of

project context and objectives

Step 1 cont.

Stakeholders in the Healthy Learning Project:

• Learning Advisors• Clients/beneficiaries• General Practices• Referrers• Project Steering Group (education,

regional economic partnership, local government, health)

PT-DES: Step 2

‘Formulating and Prioritising evaluation questions’

• Demographic characteristics of clients• Source of referral• Reason for referral (client and referrer)• Responders vs. non-responders• Physical and/or mental health outcomes

PT-DES: Step 3

‘Designing and conducting the evaluation’

Criteria for assessing suggested health outcome measures:

• Relevance to project context• Suitability within time and resource

constraints

Evaluation assignment: Suggested framework 1

• Problem and intervention (what, who, where)

• Why is the evaluation needed (purpose)

• Evaluation question

• Define stakeholders and evaluation perspective

• Define measurable outcomes

Evaluation assignment: Suggested framework 2

• Study design (include discussion of why this is appropriate, strengths, limitations)

• Data collection

• Analysis and dissemination

• Alternative evaluation approaches that could have been used (brief discussion)