Social Marketing Conference: Changing Behaviour Through

Communications

30 November 2011

www.charitycomms.org.uk

www.twitter.com/CharityComms

www.facebook.com/CharityComms

www.thensmc.com

Evaluating social marketing

Patrick Ladbury

Evaluation - Simples!

• What behaviour do you want people to do?

• Which people do you want to do that behaviour?

• By when?

• How can we measure it?

An evaluation plan

Questions to ask

• What are we measuring?– Changes in knowledge, attitudes (intention, priority)

and behaviour• What are we measuring against?

– What are our baselines or controls? • Who is doing the measuring?

– Do we need any independent input?• When do we do the measuring?• Can we turn the outcome into an economic ROI?

An evaluation plan

Different types of evaluation

TYPE Definition

FORMATIVE Done throughout the development of a project to pre-test (or ‘sound out’) aspects with the audience or stakeholders. Keeps you on track in terms of what will work with the target segment.

PROCESS A review of how the project was managed – usually involving key internal and external stakeholders.

ECONOMIC Evaluating cost effectiveness , return on investment and value for money

IMPACT (or Output)

Measures the changes that have happened directly as a result of our intervention. These are likely to be changes in knowledge, awareness and, most importantly of all, behaviour

OUTCOME Seeks to link the short term actions (impacts) with the longer term benefits (outcomes)The changes in behaviour will have been encouraged in order to meet a wider, or longer term benefit.

An evaluation plan

Impact and Outcome

An evaluation plan

What are we measuring? How are we going to measure it (method)?

What needs to be done?

Who by?Resource needed

When by?

IMPACT - Knowledge

IMPACT - Attitude

IMPACT – Behaviour

OUTCOME –

Template

A step by step approach

Planning a behaviour change intervention using Social marketing is a step by step process

Using the process helps to ensure more efficient and effective interventions and better use of resources.

A number of young people under 16 were regularly drinking alcohol on the streets of North Tyneside at the weekend.

The number of ASBOs handed out by police was increasing as were the number of complaints received by residents who were feeling less safe on the streets.

.

Getting started: example

Research led to other areas that impacted on the issue and that needed evaluating.

• Alcohol sales to young people

• Attendance and activities at youth clubs

• Young people’s drinking levels

Scoping: example

All

smokers

Smokers

who want

to quit

Smokers

who try

to quit

Smokers

who quit

using

support

Smokers

who

succeed

Smoking cessation services

100% 45%70%* 23% 2.7%

Triggering action

Making quitting more effective

Driving motivation

*NB figures relate to all smokers.

Intervention Mix – Tobacco Control

SUPPORTGive people the means to change

Better NHS Stop Smoking Services

“Quit Kit” for cold turkey - quit plan, information about medication, details for their local stop smoking services.

DESIGNChange physical context

“Pop Up” cessation services in targeted areas e.g. 360 Asda supermarkets

Over 200,000 people signed up to online communities

INFORM/EDUCATEProvide information

Normalising NHS Stop Smoking - designed “for people like me”

Focus on fact that their children 3 times more likely to start

CONTROL/COMPELIncentivise/Disincentivise

Legislation to end tobacco displays in shops

Using tax to maintain high price

Plain packaging of tobacco products

The Results

• 480,000 Quit Kits ordered

• 95% new contacts for the CRM database

• 3 million smokers attempted to quit –08/09

• Nearly 220,000 stayed quit 1 year later

• Budget of £21,115,194 (down 22%)

• 3 year payback of £73.5 million

Past evaluation work: example #1

Measurement Method Timing

Formative Qualitative Development

Outcome Questionnaire – pre and post

Knowledge, attitudes, behaviour (intended

and actual)

Development &

evaluation

Process Benchmark criteria

Horizon scanning

Stakeholder interviews

Emerging key insights, opportunities and

threats

Development,

implementation &

evaluation

Event monitoring Number of attendees

Impact on knowledge/awareness &

behaviour

Implementation &

evaluation

Media analysis Content analysis – qualitative

Impact – n

Web analysis

Implementation &

evaluation

Past evaluation work: example #2

Measurement Method Timing

Awareness and

experience of quitting

Quantitative survey:

Awareness and experiences

Development &

evaluation

Process Benchmark criteria

Horizon scanning

Stakeholder interviews

Emerging key insights,

opportunities and threats

Development,

implementation &

evaluation

Service delivery

evaluation

Qualitative Development &

evaluation

Customer journey

mapping

Observation, including

assessment of sign-posting

Development &

evaluation

Telephone helpline

monitoring

Quantitative survey Development &

evaluation

17

Take Charge – Take the Test

18

New HIV diagnoses in

2006

African Americans make up 13% of the total population

19

Get an HIV test within 3 months of having unprotected sex

What behaviour?

Key Insights

• Fear of results & stigma are main barriers

• Strong family and church ties

• Everyday burdens outweigh concerns about

HIV (e.g., children, work, bills, etc.)

• Concerns about partner’s sexual behaviour are

significant

20

Key Message

• Look out for yourself – Take Charge. Take the Test

• Emphasized:

– Danger from their partners risky behaviour

– Empowerment that comes with getting tested

– Availability and convenience of testing

– The “norm” that others were getting tested

• Targeted advertising – radio, direct mail, outdoor

• Drive people to free HIV testing locations

• Toll free line & website

• Community outreach targeted key influencers (beauty salon owners etc)

• Community “rapid HIV testing” events

• Focus on “normalizing” testing

The Marketing Mix

22

23

The Results

• Significant increase in awareness and intention

• Increased testing rates by nearly 70%

• Calls to the HIV hot line rose 290%

• Set new records for the number of HIV tests conducted in one day

• No significant impact on use of existing HIV testing services

Key Lessons

• Rapid testing events in the community were the

key to success

• They helped to reduce the “Intention – Action”

barrier

• Greater support for cities to expand the number,

reach and impact of these events

24

Value for Money

• VfM is about getting the best health possible for the resources available

• Short term vs. long term impact

Measuring Health Gain, Costs and Savings

• Health Gain is measured as QALYs

– 1 QALY = £25,000

• Costs to all stakeholders: Local Authorities, Government, NHS, Employer

• Savings from long term reductions in costs to

the stakeholders

• Health Gain of behaviour change is taken from WHO Global Health Risk report– NICE used this data, in conjunction with their evidence

base

Using the Tool: Data Needed

• Baseline data for behavioural goal

• Cost of intervention

• Number of people in target segment/audience

• Post intervention data

– E.g. number of quitters

– Age

• % of target audience with high levels of

disadvantage

Conclusion

• Continuous evaluation

• Measuring behaviour which counts

• Who needs to know

• How to sing!

Contact

Patrick Ladburyp.ladbury@thensmc.com

www.thensmc.com/resources/vfm