Health and Well-Being in Schools Steering Group18 May 2009

Neil CraigPolicy Evaluation & Appraisal Team

NHS Health Scotland

Logic Modelling for Outcomes-Based

Approaches

Outline

• What is an “outcomes-focused approach”?

• What are logic models, how do you create them how can they help?

• Discussion

What is an outcome-based approach?

• “encourages us all to focus on the difference we make and not just the inputs or processes over which we have control.”

• “should clearly demonstrate how the activities of public bodies are aligned with the Govt’s overarching purpose through the National Performance Framework” ... [and are] “supporting the achievement of outcomes”

(Outcome based approach: Working guidance for Scottish public bodies, Sept 2008)

Why now?Part of wider performance management, outcome based

planning system:

“We are moving the whole of government to an outcomes-focused approach to performance” (Scottish Budget Spending Review, 2007)

National PerformanceFramework, HEAT & SOAs

NHS PerformanceManagement:

30 HEAT Targets

SOA: Menu of 67 local outcome

indicators

PURPOSE

PURPOSE TARGETS

STRATEGIC OBJECTIVES

NATIONAL OUTCOMES

NATIONAL INDICATORS AND TARGETS

Greener

Wealthier & Fairer

HealthierSmarter

Safer & Stronger

e.g.CHD mortality Life expectancy Child healthy weightMental health-related outcomesCompletion of child

healthy weight programmes

e.g.CHD mortality Life expectancy Child healthy weightMental health-related outcomes

We live longer, healthier lives

We have tackled the significant inequalities in

Scottish society

We have strong, resilient and supportive

communities…

We give children the

best start in life

We live in well-designed, sustainable places

We value and protect the natural

environment and cut our environmental

impact

SOAs:Guidance for 2009/10

• SOAs are the “means by which CPPs agree their strategic priorities for their local area and express those as outcomes to be delivered by partners, … while showing how they contribute to the Scottish Gov’t’s relevant National Outcomes”

• The SOA “must be a strategic document … underpinned by robust perf. management arrangements… [and] a very clear line of sight from the SOA document.”

SOAs: AccountabilityLetter from SG to Chief Execs

Signing up to an SOA means that partners• “are corporately committed to the agreed outcomes …

and … will take every opportunity to promote and support the achievement of outcomes.”

• “doing this is likely to include each partner looking at how they individually can contribute to outcomes” and

• “each partner will need to show that a ‘golden thread’ runs from the SOA through their planning, resourcing and performance management processes.”

Tools for the job? Logic models

• A convincing picture of what you are trying to achieve that shows the links between your intended inputs, activities, outputs and outcomes.

• Not an attempt to describe reality but your theory about the key things that need to happen to get to your intended outcomes

• A process and a way of thinking as well as an output

Basic model

Resources / Inputs Activities Outputs

Short-term

Outcomes

Long term

Impact

Medium term

Outcomes

ASSUMPTIONS

That affect delivery

EXTERNAL FACTORS

That affect results/impacts

Define your terms• Inputs

– What you invest– E.g. Resources, staff,

funding, equipment, facilities supplies

• Activities– Key things you will have to

do – E.g. marketing,

recruitment, training, running programmes

• Outputs– Countable products– E.g. Number of people trained,

information packs, courses run, number recruited

• Outcomes (short, interim and long-term)

– The changes achieved in your target groups

– E.g. Increased knowledge, increased confidence, improved risk factors, better health

An example: child healthy weight

LONG-TERM HIGH LEVEL

OUTCOMES

INTERMEDIATEOUTCOMES

SHORT-TERM OUTCOMES

Outputs?

Inputs?

Activities/ Processes?

Reduce the rate of increase in the proportion of

children with BMI outwith healthy range by 2018

National Indicators

Menu of Local OIs

(Proportion and no. of obese children in P1)

Increased healthy life expectancy

Reductions in morbidity and mortality from obesity-related causes

Achieve agreed completion rates for child healthy weight intervention programme by 2010/11

HEAT: H3

?

Healthy weight – Cross-sector Contributions?

Inputs

Activities

Outputs

Reach

Short-term outcomes

Intermediate outcomes

High level outcomes

NHS

What NHS does to reach target

population

Child healthy weight intervention

Overweight children & families

Completed healthy weight programmes

Planners

?

?

?

?

Schools

?

?

?

?

Reduced increase in children with BMI outwith healthy range

Improved mental wellbeingReduced inequalities in healthy life expectancy

Reduced prevalence of and inequalities in obesity-related ill-health

EnvironmentsPhysical: More opportunities to lead active lives?

Economic: Less availability/affordability of energy-rich food? Social: Changing attitudes to food and exercise?

SG, UK govts, EU

?

?

?

?

Scottish Govt

?

?

?

?

Vol orgs

?

?

?

?

?

INTERMEDIATE OUTCOMES

Strategic Healthy Weight Outcomes Logic Model (Simplified)

LONG-TERM OUTCOMES

NESTED LOGIC MODELS

Reduced Type 2

Diabetes

Reduced CHD and

stroke

Reduced cancer

NATIONALOUTCOMES

Longer, healthier

lives

Tackled significant inequalities

Other non-health

outcomes?

Reduced prevalence of overweight and

obesity

Improved energy balance

Model a

Model b

Model c

Model d

Model e

Increased food industry corporate

and social responsibility

Changes in availability and affordability of

energy-dense/low energy dense

food

Improved early years nutrition

Increased individual

empowerment and food literacy

Improved media and social

environment

Increased physical activity

Nested model for child healthy weight programmes (simplified)

ActivitiesShort-term Outcomes

(by end of programme)

Intermediate OutcomeReach

Reduced increase in

children with BMI outwith healthy range

??Other NHS?

?

??Other non-

NHS?

?

Outputs

% Target group

completed programme

Improved psychosocial outcomes e.g. self-esteem,

body image

Increased physical activity

Improved nutrition

Improved knowledge and skills

Healthy weight programme: identification, contact, assessment, interventions

5–15 year olds, >91 centile

No. sessionsType of

sessions and range ofcontent

How do you develop a model?

Look for logic in plans and draft a model

Agree intended use(s) of model(s)

Engage stakeholders and adapt and refine

Share model more widely

Use, revisit and refine it regularly

Workshops, discussions, interviews, focus groups

Process• Engage people• Set boundaries• Draw on evidence, logic and ethics• Interrogate and revisit the model

Interrogate and revisit the model

• Is it plausible? • Is it doable?• Is it testable?• Have external constraints/enablers and unintended

consequences (+/-) been considered?

Learning and reflections

• No standardised approach – to the process or the product

• The value is in the process not just the product• Models should be fit for purpose, not ‘perfect’• Logic models should be embedded in part of wider

performance management system and learning culture

• Logic modelling is limited by, but may also contribute to, the existing evidence base

Conclusions: How can LMs help?

• Help you know what and when resources are needed• Provides a focus for evaluation• Helps identify what to measure + when• In complex interventions, provides some credibility

when attribution impossible• Links with bigger picture - e.g. how programme

activities support achievement of national objectivesLogic models provide a framework for integrating

planning, delivery, evaluation and performance management

Questions and Discussion