domestic violence perpetrator programme: social return on …/file/... · 2017-11-06 · economic...
TRANSCRIPT
i
Domestic Violence Perpetrator
Programme: Social Return on Investment
Analysis
Steven Ariss, Parveen Ali, Hazel Squires and Elizabeth Goyder
Economic Evaluation
ii
DECLARATION OF INTERESTS This analysis was conducted by a research team at the University of Sheffield. The report is
primarily for commissioners and delivering organisations of domestic violence and abuse
(DVA) perpetrator programmes with an aim to share good practice, measure outcomes
effectively and demonstrate Social Return on Investment (SROI). The report is available to key
partners such as the local authority, public health commissioners, perpetrator programme
delivering organisations and academics.
The research was funded by the NIHR School for Public Health Research (SPHR) Public Health
Practice Evaluation Scheme (PHPES)1. The views expressed are those of the author(s) and not
necessarily those of the NHS, the NIHR or the Department of Health.
Suggested Citation:
Ariss, S., Ali, P.A. Squires, H. (2017). Domestic Violence Perpetrator Programme: Social Return
on Investment Analysis. University of Sheffield
An electronic copy of this report and other Project publications are available at:
https://www.sheffield.ac.uk/snm/research/doncaster
For any information or questions, please contact:
Dr. Steve Ariss ([email protected])
Dr. Parveen Ali ([email protected])
1 The NIHR School for Public Health Research is a partnership between the Universities of Sheffield, Bristol, Cambridge, Imperial and University College London; The London School for Hygiene and Tropical Medicine (LSHTM); the LiLaC collaboration between the Universities of Liverpool and Lancaster and Fuse; The Centre for Translational Research in Public Health, a collaboration between Newcastle, Durham, Northumbria, Sunderland and Teesside Universities
Economic Evaluation
iii
ACKNOWLEDGEMENTS
The researchers would like to thank all those who contributed to this evaluation, particularly
the clients of the perpetrator programme, the Foundation for Change staff who facilitated
the study and a large number of other professionals in Doncaster including staff from
Doncaster Domestic Abuse Victim Services, public health practitioners, Doncaster
Metropolitan Borough Council who participated and or facilitated the research. We are also
grateful to the members of the project advisory group and members of the client reference
group who have supported this research over a period of two years.
We are also grateful to research team members from University of Sheffield: Katie Powell,
Mary Crowder, Richard Cooper, Sarah Salway, and Jennifer Burr for their contribution to the
research project. We are also grateful to the public health practitioner participants from
Doncaster council: Debbie Leyden, Susan Hampshaw, Sandra Norburn, Bill Hotchkiss, Cal
Lacey and Lucie Waugh for their help and support throughout the project. Special thanks to
the members of Foundation for Change Programme: Julie Sinclair-Day, Nikeisha Bragger,
Sarah Thompson, Fiona Fischbach and Eleanor Webb.
Economic Evaluation
iv
EXECUTIVE SUMMARY
This report documents the Social Return on Investment analysis (SROI) of a voluntary
domestic violence and abuse (DVA) perpetrator programme offered in Doncaster, England.
The programme was commissioned by Doncaster Council for three years from April 2014 from
an organisation called Foundation (www.foundationdomesticabuse.org). The programme
aimed to deliver specialist support to both men and women aged 16 years and over who
recognise a need to address their abusive behaviour and, therefore, access the programme
voluntarily. Clients were offered personalised and ongoing support through a menu of options
including: an awareness workshop; individual sessions or meetings with support worker; an
eight-week programme of structured group sessions; and flexible drop in hub on completion
of the programme.
Methods:
The SROIA covered an observed period of two and a half years, with benefits forecast for a
further two and a half years (i.e. five-year time horizon). The cost for the Local Authority to
commission the services over this time period was £382,500. The SROIA consisted of six
stages: Identify stakeholders; Map the range of inputs, outputs and outcomes; Identify and
measure key outcomes; Assess attribution, sustainability and displacement; Value outcomes;
Calculate the ratio of attributable benefits to costs and sensitivity analysis.
Findings:
A large number of outcomes could not be measured. However, five outcomes contributed
to calculating the impact of the programme. The following three outcomes provided notable
evidence of impact for the programme.
- Reduced substance abuse reported in a selection of case notes and estimated for all
clients:
o impact on society and
o impact on the individual quality-adjusted life-year
- Improved parenting evidenced by behaviour questionnaire, interviews and reported in
case notes; estimated for all clients.
Economic Evaluation
v
- Reduced time of social workers reported in case notes as information collection and
sharing with social workers.
The following two outcomes, demonstrated small values of impact for the programme:
- Reduced need for accident and emergency health services, assuming that the intervention
resulted in 50% of clients reducing their Accident and Emergency attendances down to
the national average.
- Reduced need for separate housing (fewer incidents of providing refuge for partners),
evidenced from case notes describing improved relationships with co-habiting partners
estimated for all clients.
Benefits were forecast over a five-year period and the SROI ratio was calculated using the
actual costs for the programme: assuming that funding is ceased beyond the 30 months of
the intervention. The SROI ratio was calculated at £2.05, with a net present value of
£401,617.93, and a break-even point during the fourth quarter in year three.
Conclusions:
Future economic evaluations of voluntary domestic violence and abuse perpetrator
programmes should consider the difficulties encountered and reported in this study, and seek
to identify measures and data collection mechanisms for the outcomes that were identified
as important, but could not be measured. Two pervasive problems for evaluations of this type
of programme are the loss of clients to follow-up and difficulties in obtaining outcome
measures for victims and families. Despite the disadvantages of relying on clients’ self-reports
it would be useful to further explore the potential for using proxy measures during the
window of engagement with the service-user. While some measures have been developed
for this study, areas of possible advances are described in detail in this report and in the report
for the full evaluation. The analysis allows clear recommendations, including (a) useful
approaches for those for collecting and measuring outcomes and (b) where further research
may be most valuable.
Economic Evaluation
vi
TABLE OF CONTENT DECLARATION OF INTERESTS ..................................................................................................... ii
ACKNOWLEDGEMENTS ............................................................................................................. iii
EXECUTIVE SUMMARY .............................................................................................................. iv
TABLE OF CONTENT .................................................................................................................. vi
LIST OF TABLES ........................................................................................................................ viii
LIST OF FIGURES ...................................................................................................................... viii
LIST OF ABBREVIATIONS ........................................................................................................... ix
INTRODUCTION .......................................................................................................................... 1
1.1 Local Context ............................................................................................................... 3
1.2 Description of the perpetrator programme ................................................................ 4
1.2.1 Awareness workshop ........................................................................................... 5
1.2.2 Individual sessions ............................................................................................... 5
1.2.3 Eight week structured group sessions ................................................................. 6
1.2.4 Flexible, drop-in support ...................................................................................... 6
METHODS ................................................................................................................................... 8
2.1 Key challenges ............................................................................................................. 8
2.2 Principles and Stages of SROI ...................................................................................... 8
2.2.1 Stage 1: Identifying stakeholders....................................................................... 10
2.2.2 Stage 2: Mapping the full range of inputs, outputs and outcomes ................... 12
Mapping outcomes ...................................................................................................... 12
Identifying and Valuing Input....................................................................................... 12
Details of Financial Input ............................................................................................. 12
Clarifying outputs and outcomes ................................................................................. 14
2.2.3 Stage 3: identification and measurement of key outcome indicators .............. 14
2.2.4 Stage Four: Establishing impact ......................................................................... 17
Economic Evaluation
vii
Deadweight and displacement .................................................................................... 17
Attribution .................................................................................................................... 18
Drop-off ........................................................................................................................ 18
Calculating impact ........................................................................................................ 18
2.2.5 Stage Five: Calculating the SROIA ...................................................................... 19
Projecting into the future ............................................................................................ 19
Calculating the net present value ................................................................................ 19
Calculating the SROIA ratio .......................................................................................... 19
Sensitivity analysis ....................................................................................................... 19
Payback period ............................................................................................................. 20
2.3 Strengths and limitations of the methodology ......................................................... 20
FINDINGS .................................................................................................................................. 22
3.1 Difficulties in measuring outcomes ........................................................................... 29
3.2 SROI Calculations ....................................................................................................... 31
3.2.1 Sensitivity Analysis ............................................................................................. 32
DISCUSSION .............................................................................................................................. 38
RECOMMENDATIONS FOR FUTURE RESEARCH ....................................................................... 40
REFERENCES ............................................................................................................................. 41
Economic Evaluation
viii
LIST OF TABLES
Table 1. 1: Summary of the group sessions ............................................................................... 7
Table 2.1: Principles of SROI ...................................................................................................... 9
Table 2. 2: Details of SROIA related data collection activity ................................................... 10
Table 2.3: Key stakeholders and rationale for inclusion in the SROIA .................................... 11
Table 2.4: Outcomes for stakeholders ..................................................................................... 16
Table 3.1: SROI assumptions and values ................................................................................. 23
Table 3.2: Counted SROI Values ............................................................................................... 33
Table 3.3: 0 to 2.5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)
.................................................................................................................................................. 34
Table 3. 4: 2.5 to 5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)
.................................................................................................................................................. 35
Table 3.5: Low sensitivity analysis ........................................................................................... 36
Table 3.6: Sensitivity analysis comparisons ............................................................................. 37
LIST OF FIGURES Figure 2.1: The process of SROIA ............................................................................................... 9
Figure 2. 2: Impact map ........................................................................................................... 13
Economic Evaluation
ix
LIST OF ABBREVIATIONS A & E Accident and Emergency
CAFCAS Child and family court advisory and support service
CRG Client reference group
CSEW Crime survey of England and Wales
DAC Domestic abuse caseworker
DHR Domestic Homicide Reviews
DMBC Doncaster Metropolitan Borough Council
DVA Domestic violence and abuse
F4C Foundation for Change
FGD Focus group discussion
FGM Female genital mutilation
GP General Practitioner
IDVAs Independent domestic abuse advisor
IPV Intimate partner violence
MARAC Multi-agency risk assessment conference
MORM Multifactor offender readiness model
MRC Medical research council
NICE National Institute of Health and Clinical Excellence
NIHR National Institute of Health Research
NSPCC National Society for the Prevention of Cruelty to Children
PAG Project advisory group
PHPES Public Health Practitioners Evaluation Scheme
PHQ-9 Patient Health Questionnaire-9
PSSRU Personal Social Service Research Unit
QALY Quality-adjusted life-year
ScHARR School of Health and Related Research
SD Standard deviation
SMART Sustainable, measurable, achievable, realistic, time-bounded
SPSS Statistical package for social sciences
SROI Social return on investment
SROIA Social Return on Investment Analysis
UCLAN University of Central Lancashire
UK United Kingdom
WEMWBS Warwick Edinburgh Mental Well Being Score
1
INTRODUCTION
This report documents the economic evaluation using a Social Return on Investment Analysis
(SROIA) of a voluntary domestic violence and abuse (DVA) perpetrator programme offered in
Doncaster, England. The evaluation was conducted as part of a wider evaluation programme
funded by the National Institute of Health Research (NIHR) Public Health Practitioners
Evaluation Scheme (PHPES), through the School for Public Health Research (SPHR). The work
was carried out between April 2015 and March 2017, and was led by a team of researchers
from the School of Health and Related Research (ScHARR) and School of Nursing and
Midwifery, University of Sheffield with input from staff from Foundation for Change (F4C),
domestic violence perpetrator programme and Doncaster Metropolitan Borough Council
(DMBC).
Domestic violence and abuse (DVA) remains a major social and public health problem that
intersects social, economic, cultural, geographical and religious boundaries. The World Health
Organization (2014) defines it as a pattern of abusive behaviour perpetrated by an ‘intimate
partner’ or ex-partner resulting in physical, sexual or psychological harm. The acts may include
physical aggression, sexual coercion, psychological abuse and controlling behaviours.
Controlling behaviour refers to ‘a range of acts designed to make a person subordinate and/or
dependent by isolating them from sources of support, exploiting their resources and
capacities for personal gain, depriving them of the means needed for independence,
resistance and escape, and regulating their everyday behaviour’ (UK Home Office, 2013). DVA
has been found to be associated with severe physical and psychological consequences in
victims ranging from substance abuse, depression, posttraumatic stress disorders, suicidal
ideation, injury, and death (Alhabib, Nur, & Jones, 2010; Hamilton, Koehler, & Lösel, 2012).
While development and provision of victims support programmes is necessary, appropriate
programmes aiming at DVA perpetrators to help change their attitudes and behaviours
through understanding the consequences of their abuse are important (Akoensi, Koehler,
Lösel, & Humphreys, 2013; Hamilton et al., 2012).
Economic Evaluation
2
In this report, the term DVA, is used to describe the range of violent or abusive behaviour
between intimate partners or other close family members. Other terms such as ‘Intimate
partner violence’ (IPV), and ‘domestic abuse’ are also used when referring to previous work
using different definitions. The term ‘perpetrator’ is used to refer to an individual committing
DVA and the term ‘victim’ is used to refer to the victim or survivor of DVA.
While some research has been conducted to explore the effectiveness of perpetrator
programmes (Akoensi et al., 2013), methodological issues affecting the quality of studies
make it difficult to derive firm conclusions. Research has also been conducted to determine
the individual and societal cost of DVA (Bonomi, Anderson, Rivara, & Thompson, 2009; Farmer
& Tiefenthaler, 1997; Yodanis, Godenzi, & Stanko, 2000). However, not much information is
available about the cost, cost effectiveness or the value of DVA perpetrator programmes. This
could be due to the difficulties in determining the economic benefits and especially the social
value of such programmes. The value produced by such programmes in the form of changes
in DVA related attitudes and behaviours can be subtle, difficult to measure and realised over
a long time period.
Social Return on investment (SROI) analysis is a framework that enables measurement and
quantification for a broad range of values. The process aims to draw attention to issues of
inequality, environmental degradation and wellbeing by incorporating social, environmental
and economic costs and benefits associated with a particular programme or organisation
(Nicholls, Lawlor, Neitzert, & Goodspeed, 2009; 2012; Wood & Leighton, 2010). In fact, the
Public Services Act that came into force on 31 January 2013, mandates people responsible for
commissioning public services to consider wider social, economic and environmental benefits
of any service prior to procurement processes (Cabinet Office, 2016). The SROI process has
been used to evaluate the impact of a wide range of initiatives including social enterprise
(Rotheroe & Richards, 2007), a transport to work programme (Wright, Nelson, Cooper, &
Murphy, 2009), a community befriending programme (Arvidson, Battye, & Salisbury, 2014)
and peer support for people with dementia (Willis, Semple, & de Waal, 2016). However, very
few attempts have been made to determine the social value of DVA perpetrator programmes
(Nef consultancy, 2013; Perfect Moment, 2010). This report aims to present the findings of a
SROI analysis to quantify the social value of a voluntary DVA perpetrator programme offered
Economic Evaluation
3
in Doncaster, England: to identify and report social outcomes that are not always
measureable with regards to their monetary value.
1.1 Local Context
Doncaster has a relatively high rate of reported DVA and available routine data suggest a
steady increase over the past five years (Doncaster Domestic Abuse Service, 2016). The
reported rate of DVA is higher than other parts of South Yorkshire and is one of the highest
in the country. Over the last year there has been a steady rise in the number of domestic
abuse related crimes recorded by South Yorkshire Police. Forty-seven percent of DVA
offenders were an ex-partner, partner or spouse of the victim. Offenders were
disproportionately male, aged between 20 and 34, of Black ethnicity and unemployed at the
time of the offence, compared with the general population.
Six out of the 11 wards across South Yorkshire with the highest number of DVA offenders are
in Doncaster. The areas in Doncaster with the highest number of reported domestic abuse
offences have high levels of deprivation, and are characterised by: low value housing, low
income, high levels of unemployment, above average levels of mental health issues, drug and
alcohol abuse and high levels of accident and emergency (A &E) admissions, low levels of
educational attainment and high levels of crime and anti-social behaviour (Doncaster
Domestic Abuse Service, 2016). DVA is associated with significant negative health effects as
well as economic costs on victims, their families, and on public services and wider society
(Bellis, Hughes, Perkins, & Bennett, 2012). The overall wider public cost of DVA in all cases for
Doncaster is estimated to be over £110 million (Doncaster Domestic Abuse Service, 2016).
In 2012, Doncaster’s DVA services were subjected to a comprehensive review carried out by
the National Society for the Prevention of Cruelty to Children (NSPCC) and Kafka Brigade. The
resulting strategy set out objectives and key priorities for local agencies and partnerships to
work together for an integrated response taking a whole family approach. The Foundation for
Change (F4C) programme was commissioned as part of the range of related interventions that
included: Domestic Abuse Hub (first response, centre of expertise), prevention and education
(raising public awareness, work with schools/colleges), a Multi-Agency Workforce
Economic Evaluation
4
Development Strategy and Capability Framework, support for victims including housing and
refuge provision and services to support children and young people.
1.2 Description of the perpetrator programme
The perpetrator programme was commissioned by Doncaster Council for two years from April
2014 from an organisation called Foundation (www.foundationdomesticabuse.org) with a
budget of approximately £300,000 and was extended to a further one year till March 2017.
The programme aimed to deliver specialist support to DVA perpetrators, including both men
and women aged 16 years and over. The programme was designed to cater for perpetrators
who recognise a need to address their abusive behaviour and, therefore, access the
programme voluntarily. In common with a number of other DVA interventions for
perpetrators and victims, the perpetrator programme embeds motivational interviewing as a
core element, a directive client-centred counselling style that aims to elicit behaviour change
by helping clients to explore and resolve ambivalence. Every client is supported by a named
support worker who employs motivational interviewing principles (Miller & Rollnick, 2012;
Rollnick & Miller, 1995) and techniques to enhance the client’s sense of self efficacy and
willingness to constructively engage with the programme from the first assessment
appointment onwards.
The programme also uses the trans-theoretical Stages of Change framework (DiClemente &
Prochaska, 1982; Prochaska, 2013; Prochaska & DiClemente, 1986) to sequence
interventions. In addition, the programme recognises the potential for wider contextual
factors to inhibit behaviour change, or act as triggers, and includes one-to-one sessions where
the client and support worker co-produce a support plan, designed to maximise the potential
for behaviour change by addressing barriers to the process for example: financial difficulties,
homelessness, and alcohol abuse. Self-observation and self-monitoring against SMART
(Specific, Measurable, Achievable, Realistic, Time-bounded) targets is encouraged in regular
review sessions.
The programme is delivered jointly by two support workers with additional support from a
programme manager and an administrative officer. Two support workers who were initially
Economic Evaluation
5
involved in the programme have now moved to another programme offered by the
organisation and other two support workers (started work in September 2016) have taken
over the role. Since April 2014 four other support workers and two assistant support workers
joined the programme on short term contracts. All support workers were educated up to
degree level (degree in psychology and criminology). Except two, all support workers (n-10)
who have worked since inception of the programme were females and their age ranged
between 26-41 years (mean 30.4; Mode 28).
Clients are referred by professional organisations using a standard referral form. Clients can
also self-refer in to the programme. Such clients may also be signposted to the perpetrator
programme by various other organisations. Following referral, all clients are assessed for their
eligibility and willingness to complete the programme. Following acceptance onto the
scheme, a client is offered personalised and ongoing support through a menu of options
including: an awareness workshop; individual sessions or meetings with support worker; an
eight-week programme of structured group sessions; and flexible drop in hub on completion
of the programme. In the following section, brief details of these activities are presented.
1.2.1 Awareness workshop
The awareness workshop is the first session clients attend with F4C. These were introduced
to maximise client engagement and reduce the number of dropouts following initial risk
assessment. It is intended to provide appropriate programme information to help prospective
clients understand what will be expected of them and make an informed decision about
involvement in the programme.
1.2.2 Individual sessions
One-to-one sessions or meetings are offered to clients throughout the programme. The aim
of these sessions could be different depending on the individual client needs. The one-to-one
sessions take place at different stages of the perpetrator programme and the examples
include:
- Pre-group work: clients attend two or more one-to-one sessions with their designated
support worker prior to their engagement in group work.
Economic Evaluation
6
- Alongside group sessions; one-to-ones could be requested by clients who require
additional support outside of group sessions or by support workers to address concerns
about client attitudes or commitment to the programme.
- Programme end: final ‘exit’ one-to-one session following completion of the group
sessions.
- As an alternative for clients unable to attend group sessions; for those identified as
unsuitable for group sessions, such as mental health issues which could not be managed
effectively or triggered within the group (i.e. Autism, personality disorder, PTSD) or child
care issues or work commitments making it difficult for people to attend group session.
- One to one sessions were also offered to female perpetrators. Group work sessions for
female perpetrators were also set up at victim services to facilitate attendance.
1.2.3 Eight week structured group sessions
An eight week (three hours per week) series of structured group sessions are offered as part
of the perpetrator programme. In rare cases, the sessions are also offered as one-to-one
sessions as mentioned previously. Table 1.1 provides an overview of the sessions.
1.2.4 Flexible, drop-in support
Clients can continue to access flexible monthly drop-in hubs, beyond the end of the
programme, to ‘touch base’ and talk about any concerns that they may have. This support
arrangement is co-developed with clients who, following completion of the programme, are
indicated as in need of additional support.
Economic Evaluation
7
Table 1. 1: Summary of the group sessions Session Title Session Objectives/Learning Outcomes
What is a healthy relationship?
• To further develop awareness of abusive behaviour in intimate and family relationships
• To practice skills to improve healthier relationships with their intimate partners and families
Managing emotions • To develop an awareness of emotional states that are difficult to cope with • To recognise triggers to our own emotions and develop awareness of
physical signals that accompany emotional states • To recognise unhealthy and healthy ways of coping with difficult mind
states and the affect these have on relationships • Practice skills to improve coping strategies and plan for healthier
relationships
Communication Skills
• To develop an awareness of good communication skills • To practice saying how we feel without being abusive • To practice skills to improve communication with partners and plan for
healthier relationships
Negotiating a compromise
• To practice active listening and assertive statements to resolve conflict without resorting to abuse
• To use a problem solving approach to resolve conflict with partners in a non-abusive manner
• To identify the arguments that most often occur in intimate relationships and plan a healthy way to reach an agreement
The impact of domestic abuse on children
• To understand the effects of DVA on children • To identify what children, need from parents • To plan healthier relationships with children without partners and families
Positive parenting • To improve understanding of the impact of physical/violent discipline methods on children
• To identify healthy ways of encouraging good behaviour and setting up appropriate boundaries
• To weigh up costs and benefits of smacking • To plan ways to be positive parent/role model
Substance misuse • To improve understanding why and how we use alcohol • To understand how alcohol effects relationships • To identify safe limits for alcohol use • To plan for healthier relationships with our partners and families
Relapse prevention Plan
• To develop a detailed plan for improving relationships with partners and families
Note: The objectives are taken from each individual session
Economic Evaluation
8
METHODS
2.1 Key challenges
The approach to economic analysis of the programme was Social Return on Investment
(SROI). “SROI can retrospectively measure outcomes that have already occurred (evaluative-
type) or can prospectively predict how much value will be generated if the intervention meets
its intended outcomes (forecast-type)” (Banke-Thomas, Madaj, Charles, & van den Broek,
2015, p. 1471). However, there are considerable barriers to empirically assessing the impact
of DVA perpetrator programmes in terms of valuing outcomes. Some of the main difficulties
include:
Low levels of voluntary participation in evaluation
Loss of participants to follow-up
Difficulty accessing data, which could quantify the extent of the benefits experienced
by the victims, and any children
Difficulties accessing potential beneficiaries
The breadth of benefits being uncertain; does it affect subsequent partners, the
wider community (e.g. police, health, employer, welfare, school etc.)
Potentially very long time-periods for full realisation of benefits, especially effects on
children into adulthood (e.g. PTSD, depression, replication of learned behaviour)
Therefore, some potentially important outcomes were not included in the final calculations
as it was not possible to collect data regarding all impacts of the programme (see Section 3.1
which sets out the difficulties in measuring outcomes).
2.2 Principles and Stages of SROI
For the purposes of this study an evaluative type social return on investment analysis (SROIA)
was carried out. The SROIA, in line with Department of Health’s best practice guidance (DH,
2011), is based on seven principles as suggested by the SROI network and summarised in table
2.1. The SROIA involves six stages as illustrated in Figure 2.1. The following section of the
report describes each of the five stages, prior to reporting of the SROI analysis that were
undertaken for this work.
Economic Evaluation
9
Table 2.1: Principles of SROI
Principal Definition
Involve stakeholders To inform what gets measured and how this is measured and valued by
involving stakeholders
Understand what
changes
Articulate how change is created and evaluate this through evidence
gathered, recognising positive and negative changes as well as those that are
intended or unintended
Value the things that
matter
Use financial proxies in order that the value of the outcomes can be
recognised. Many outcomes are not traded in markets and as a result their
value is not recognised
Only include what is
material
Determine what information and evidence must be included in the accounts
to give a true and fair picture, such that stakeholders can draw reasonable
conclusions about impact
Do not over-claim Only claim the value that organisations are responsible for creating
Be transparent Demonstrate the basis on which the analysis may be considered accurate and
honest, and show that it will be reported to and discussed with stakeholders
Verify the result Ensure independent appropriate assessment
Figure 2.1: The process of SROIA
Stage One
•The identification of key stakeholders
Stage Two
•Mapping of the full range of inputs, outputs & outcomes
Stage Three
•Identification and measurement of key outcome indicators
Stage Four
•Establishing Impact: Assessment of attribution, sustainability and displacement of other services
Stage Five
•Calculating the ratio of attributable benefits to costs plus sensitivity analysis
Stage Six
•Reporting using and embedding
Economic Evaluation
10
2.2.1 Stage 1: Identifying stakeholders
After establishing the preliminary scope of the project, the first stage of SROIA involved
identifying stakeholders to be included in the exercise. An initial group of stakeholders2
including service providers, commissioners and academics were consulted to identify further
stakeholder groups and potential individuals to contact. Following this, we conducted three
individual interviews and three SROIA workshops (each covering all three stages of the SROIA)
involving nineteen individuals representing various stakeholder groups (Table 2.2). The
participants included staff members of the DVA perpetrator programme, representatives of
associated services including domestic violence victim services, commissioners, public health
practitioners, social workers, clients of the perpetrator programme and partners of the clients
(victims/ survivor of violence). Further information about data collection is presented in the
next section.
Table 2. 2: Details of SROIA related data collection activity
Workshop 1 Workshop 2 Workshop 3
Individual Interviews
Number of participants
4 5 10 3
Age (range)
25-44 25-44 32-54 25-40
Gender Male Female
2 2
0 4
1 9
0 3
Description of participants
Two domestic abuse case workers (DACs), and two victims whose partners were undertaking perpetrator programme
Service users of perpetrator programme
DVA practitioners from perpetrator programme and victim services; public health professionals; commissioners; researcher
2 Stakeholders refer to any group of individuals or organisation that is affected by the
service/intervention and experience.
Economic Evaluation
11
Through various interactive exercises, the participants identified other key stakeholders, their
contribution and impact on the programme and vice versa. Table 2.3 outlines the key
stakeholders included and the rationale for inclusion and exclusion in the SROIA.
Table 2.3: Key stakeholders and rationale for inclusion in the SROIA
Stakeholder/Role Reason for inclusion in SROIA
Commissioners Local Council, public health, South Yorkshire Police
Provide finances for the programme. Commission various services in the town with an aim to reduce incidence and prevalence of DVA
DVA Perpetrator Programme Staff
Deliver DVA perpetrator programme and ensure that it meet the needs of service users and commissioners
Service users (perpetrators) Access DVA perpetrator programme in an effort to change their behaviour. Involvement in the programme can help them improve their behaviour and consequently improve their relationships with their partner and others
Partners of service users Are affected by the programme activity as improvement in the abusive behaviour of their partner can improve their intimate relationship
Children of service users and their partners
May be affected by the DVA in their family. Improvement in the intimate relationship of their parents/ guardians can have positive impact on their life
Immediate family and friends May include extended family of perpetrator/ service user and their partner. They may act as support system for the victim, children and/or the perpetrator.
Children services Indirect beneficiary: Aim to keep children safe and may have to intervene in situations where children are living in DVA affected families.
Social services Indirect beneficiaries: Aim to help children, adults and families to take control of and to improve their lives in conditions where their security, safety or ability to participate in civic life are restricted.
DVA victim support services Indirect beneficiary: Aim to support victims of DVA and keep them safe
Police and criminal justice system
Indirect beneficiary: Aims to reduce the number of DVA incidents in the community and react in the event of an incident
Health services Indirect beneficiary: Benefit from short-term reduction in treatment for physical harm and longer-term mental health and alcohol and drug abuse
Stakeholders Reason for Exclusion
School and Education system May not be aware of involvement of perpetrator in the perpetrator programme and therefore may not be able to comment about the programme.
Neighbours Proportion of affected neighbours will be low and difficult to identify and measure impact.
Economic Evaluation
12
Neighbourhood teams May not be aware of the perpetrator involvement in the programme; Difficult to identify and measure impact.
2.2.2 Stage 2: Mapping the full range of inputs, outputs and outcomes
Mapping outcomes
An impact map sets out the programme theory of the intervention, and communicates how
an intervention is expected to make a difference, and how inputs are expected to result in
outputs which lead to outcomes. An SROIA impact map is similar to a ‘theory of change’ or
‘logic model’; it describes the overall rationale for the programme and sets out linked
assumptions of the relationships between activities and outcomes. Activities mentioned
above played a key role in mapping inputs and outputs and identification of key outcome
indicators and appropriate data sources to measure those outcomes. Figure 2.2 provides a
visual overview of the Impact Map. Please note that the stages outlined in the Impact Map
correspond to stages 1-3 outlined in Figure 2.1.
Identifying and Valuing Input
Inputs are things that stakeholders contribute in order to make activities possible. The inputs
were relatively straightforward to identify with the help of the perpetrator programme staff.
For example, the financial input was primarily from the funding received from the
commissioning body. This funding was then used to pay for the staff salaries, running cost of
the office premises etc. Contributions from clients and other stakeholders were mainly in the
form of their time, which is considered as an input, however, according to standard SROIA
approach, “…the time spent by the beneficiaries on a programme is not given a financial
value” (Nicholls et al, 2009, p.32). Consultations with stakeholders established that inputs
from other stakeholders did not determine whether the programme activities went ahead.
The only exception to this was the time spent for referral and signposting from other
organisations, though, this was considered to be part of the everyday business of these
stakeholders and not an additional activity. Therefore, the time of practitioners in other
organisations (e.g. social worker, probation, and children’s services) has not been costed.
Details of Financial Input The original contract for the service was paid for by the Local Authority at a rate of £306,000
for two years, which was then extended for a further six months. Costs were scaled up to
Economic Evaluation
13
Figure 2. 2: Impact map
Valu
e Ou
tcom
e
Better family relationship
Reduction in DVA perpetration
Improved physical & mental health
Improved parenting: outcomes for children
Reduced housing needs
Reduced demand on other agencies
Improved employability
Reduced substance abuse
Measu
re Ou
tpu
t
Increased awareness of personal behavior and its impact leading to changes in DVA attitudes and behavior
Increased awareness of available resources
Numbers of clients completing aspects of the course
Assistance with wider contextual issues (e.g. housing)
Communication and coordination with other agencies
Activity
One-to-one support session
Eight group sessions
Drop in Hub
Personalized support
Increased interaction opportunities
Opportunities to reflect & role modeling
Qu
antify In
pu
t
Staff
Facilities
Resources
Stakeho
lders
Commissioners
Staff
Service users & partners
Children
Family and friends
Children services
Social services
DVA support service
Police and criminal justice system
Health services
Economic Evaluation
14
two and a half years, resulting in a total cost of £382,500. The service was provided in
quarterly cycles (four per year) as an eight-week course, with four to five weeks between
courses. Costs and values of returns were calculated for each cohort on a quarterly basis; the
study period therefore covered 10 quarters/cycles, each with a cost of £38,250.
Clarifying outputs and outcomes
Outputs are quantitative summaries of activities. Sometimes outputs are repeated for several
stakeholders at this stage because they form part of the theory of change. However, repeated
outputs are not counted in the final calculation, so double counting cannot occur (Nicholls et
al., 2012). As mentioned earlier, the data collection activities mentioned above not only
helped identify key stakeholders but also helped in mapping inputs and outputs and
identification of key outcome indicators and appropriate data sources to measure those
outcomes.
2.2.3 Stage 3: identification and measurement of key outcome indicators
The perspectives of the different stakeholders are at the centre of the valuation process,
particularly for guiding the identification and measurement of key outcome indicators. We
used a mixed-method, theory-driven evaluation approach involving a wide range of
stakeholders, including service commissioners, service providers, clients, social workers and
other professionals (Nicholls et al., 2012). In addition to the SROIA specific data collection
activities mentioned above, we used various other strategies to collect empirical evidence
(for the larger evaluation of the programme3) and these are summarised below (Ali, Ariss,
Powell, Crowder, & Cooper, 2017).
20 in-depth interviews including nine open-ended individual interviews and eleven
repeat narrative interviews involving a total of 14 different clients.
Five iterative, ‘theory-focused’ interviews involving six members of staff of the
perpetrator programme.
3 An electronic copy of the full evaluation report is available at: https://www.sheffield.ac.uk/snm/research/doncaster
Economic Evaluation
15
11 in-depth interviews involving 12 stakeholders including commissioners, social
workers, professionals from other services such as the police, probation, women’s
centre, domestic abuse victim services, and children's services.
Three focus group discussions (including one with perpetrator programme staff and
two with other stakeholders) involving 21 participants.
Ethnographic observations of six one-to-one sessions between clients and their
keyworkers. We also reviewed recordings of each of the eight group sessions.
Case note review of 33 clients
Review of routinely collected data of 281 clients.
Pre and post-programme attitude survey completed by 42 clients.
Self-administered questionnaire administered at the start or the programme (T0;
N=42), at the end of the programme (T1; N=11)) and three months’ post completion
of the programme (T2; N=4).
Final calculations were based on empirical evidence. Outcomes for which we could not
identify appropriate evidence were included in the SROIA, where necessary, but given a value
of zero to allow values to be inserted in the model, if they become available. Outcomes for
which no evidence was available to support them were not included in the sensitivity analysis;
in line with the SROIA principles of only including ‘what is material’ and being careful not to
over-claim.
All assumptions in the SROIA model are based on the most appropriate evidence, together
with expert judgement. Following the ethos of SROIA, these are conservative and transparent,
such that these are open to challenge so that they are either improved or displaced in time
by empirical evidence. A key assumption necessary to conduct this SROIA is that; where data
availability is limited, and evidence exists for a proportion of a representative sample of
service-users, then this is assumed to pertain to all clients at the same ratio across the
programme.
Data collected from various sources (individual interviews, focus group discussions, SROIA
stakeholder activities, and questionnaires) helped identify a wide range of changes for various
Economic Evaluation
16
direct and indirect beneficiaries which might include; clients / service users, partners of
service users (victims), children, family members and friends, other services (local services,
social services, children services, police/ criminal justice system, health services, homeless
charities/ refuges and Department of work and pensions and employers) as shown in Table
2.4.
Table 2.4: Outcomes for stakeholders
Stakeholders Outcomes (based on how stakeholders describe the change)
Perpetrators - Improved parenting skills
- Better mental health/well-being
- Lower use of substances/alcohol
- Make more informed decisions
- Increased motivation to change
Victims - Reduced incidents of DVA and repeat incidents
- Reducing time that victims spend in abusive relationship
- Improved family relationship
- Improved health and well being
Immediate family and friends
- Fewer incidents of providing refuge for partners
- Lower levels of worry/ Improved family relationships
Children - Fewer children involved in incidents of DVA
- Improved school attendance
- Improved well-being/mental health
Victim services (DACs, Counsellors, refuge staff) and other agencies
- Increased number of referrals to victim services
- Changes in type of support requested (less crisis)
- Preventing escalation to MARAC
Social Services/ Local Authority Children's services
- Reduced costs of supporting/caring for children and family members
o Reduced time of Social Worker (e.g. Children's Services)
o Reduced number of children requiring adoption
Police/ vulnerability units/ criminal justice system
- Reduction in incidents of abusive behaviour reported to police or other agency (Reduced numbers of police call outs/policing cost reduced rate of domestic abuse)
Employers, Department of Work and Pensions
- Attendance of employee/perpetrator/ victims at work; performance of perpetrator at work
Health services - Reduced health care needs (DVA related)
Homeless charities, women’s refuges Housing services
- Reduced homelessness/ need for separate housing
Economic Evaluation
17
For the later stages of the SROIA, we counted values that had been attributed to outcomes,
to arrive at the impact of the programme. Where evidence of outcomes was not available,
these have been given zero occurrences so that they do not contribute to the final calculation.
2.2.4 Stage Four: Establishing impact
Establishing impact refers to determining how much of the outcome would have happened
anyway in the absence of the programme/ intervention. It also explores what proportion of
the outcome can be isolated as being added by the programme activities (Cabinet Office,
2016; Nicholls et al., 2012). Establishing impact is important as it reduces the risk of over-
claiming. Measures and values attributed to outcomes are detailed in the findings section
(Table 3.1)
Deadweight and displacement
Deadweight is a measure of the amount of the outcome that would have happened anyway
without the programme. This was both assessed by stakeholders through recourse to their
expertise and experiences and established through available evidence. Cabinet office
guidance indicates that this is rarely a perfect estimate: “measuring deadweight will always
be an estimate since a perfect comparison is not possible. Instead, you need to seek out
information that is as close to your population as possible. The more similar the comparison
group, the better the estimate will be” (Nicholls et al, 2009, p.56). It is measured as a
percentage, which is deducted from the total quantity of the outcome. To understand this
better, it may be useful to consider an example. For instance, if 30% more people on a
programme gain employment than would have been expected without the programme the
deadweight is calculated as 100% (business as usual) expressed as a percentage of 130%
(programme outcome) = 77% deadweight.
Displacement is an assessment of the extent to which the outcome measures have
demonstrated a change, which has been absorbed elsewhere. For instance, reductions in DVA
could change into other forms of violence to others. This would require an adjustment by
either including the stakeholder now being affected or estimating the effect of double
Economic Evaluation
18
counting as a percentage to be deducted from the total. No displacement was calculated for
the outcomes included in the model.
Attribution
Attribution is an assessment of the extent to which outcomes were caused by other external
influences (e.g. other programmes, organisations or people). The extent to which outcomes
are assessed at being attributable to other sources is deducted from the programme outputs.
“It will never be possible to get a completely accurate assessment of attribution. This stage is
more about being aware that your activity may not be the only one contributing to the change
observed than getting an exact calculation. It is about checking that you have included all the
relevant stakeholders.” (Nicholls et al, 2009, p.56).
Within Doncaster, there are no similar voluntary interventions aimed at addressing the
abusive behaviour of low-risk DVA perpetrators. There are other services that clients can
access to potentially address problematic behaviour (such as substance abuse), which is also
addressed by elements of the programme. However, evidence was not available for clients
accessing other services. Indeed, if clients were considered to need external interventions,
they were not admitted onto the programme.
Drop-off
Drop-off is concerned with how long the outcomes last. The amount that the value of an
outcome will be expected to reduce over time, or be influenced by other factors, so that
attribution becomes lower. It is calculated for outcomes that last longer than one year, as a
fixed percentage that is deducted year on year. This is calculated on a case-by-case basis for
each outcome. Whilst the full benefits of some outcomes are only realised over time, others
are short-term and rapidly decrease over time. Details of these calculations are included in
the findings section (Table 3.1). The time horizon for the analysis is a five-year period.
Calculating impact
Impact is calculated by multiplying the financial proxy by the quantity of an outcome to give
a total. From this total, percentages are then deducted for deadweight, displacement and
Economic Evaluation
19
attribution. This is repeated for each outcome to give the impact for each. These individual
impacts are then combined to give a figure for the total impact.
2.2.5 Stage Five: Calculating the SROIA
Projecting into the future
Projecting the value of outcomes into the future consists of setting out the value for each
outcome for a determined time-period (i.e. five years), and copying across the number of
time periods it will last. Any drop off calculated above is then deducted. Details of these
calculations are included in the findings section (Tables 3.3 & 3.4)
Calculating the net present value
The Net Present Value (NPV) is calculated by comparing costs and benefits over a time period.
Discounting of 3.5% per annum over the projected time periods is used to account for money
today having greater value than money in the future (For the public sector, the basic rate
recommended in HM Treasury’s Green Book (2016) is 3.5%). This gives the present value of
benefits. The value of investments is then deducted to calculate the NPV.
Calculating the SROIA ratio
The SROI ratio is calculated by dividing the present value by the value of inputs. The SROI ratio
was calculated using two approaches to valuing the inputs; one of which reflects the actual,
short-term funding for the programme, and one approach which demonstrates the return for
a hypothetical longer-term funding model. The first of these used the actual costs for the
programme, taking account of the fact that the funding for the programme ceased. The
second approach assumed that the funding for the programme would continue over the same
five-year period as the benefit calculations.
Sensitivity analysis
After calculating the ratio, it is important to assess the extent to which this would be changed
by altering some of the assumptions from previous stages. The sensitivity analysis assesses
the importance of various elements of the model. It is therefore possible to explore which
assumptions have the greatest effect on the model. The standard requirement is to check
Economic Evaluation
20
changes to: estimates of deadweight, attribution and drop-off; financial proxies; the quantity
of the outcome; and the value of inputs, (where non-financial inputs have been valued).
To estimate the sensitivity, calculations were made to explore the effect that would result
from changing each estimate to lower-level assumptions, both individually and for the whole
model. By calculating this, the sensitivity of the analysis to changes in estimates can is shown.
This reporting of the model including lower-level assumptions demonstrates the amount of
change to the model necessary to make the ratio change from positive to negative or vice
versa (i.e. lower than £1 value for £1 investment)..
Payback period
The ‘payback period’ describes how long it would take for the investment to be paid off in
terms of the accrual of social value. Specifically, it describes at what point in time the value
of the social returns starts to exceed the investment. This can be used to determine risk in a
project. While a short payback period may seem less risky, a longer payback period is often a
feature of activities that are able to demonstrate significant long-term outcomes. For these
types of interventions longer-term core funding is required.
2.3 Strengths and limitations of the methodology
As with any approach, an SROIA approach has a number of strengths and limitations and it is
important to appreciate these early on in the process. In terms of strengths, the SROIA
enables identification and presentation of a single monetised ratio of positive and negative
outcomes that can be easily understood and conveyed to a variety of stakeholders. The SROIA
values the engagement of stakeholders and therefore provides a platform where
stakeholders are involved in not only informing the outcomes but the values and
meaningfulness of those outcomes from stakeholders’ perspectives. This facilitates
transparency and accountability. An additional strength is the ability to value more than
simple organisational or individual fiscal transactions; placing value on longer-term social
benefits, which are often the most important intended outcomes of this type of social
intervention.
Economic Evaluation
21
Like any quantitative approach, one of the biggest limitations of the SROIA, however, relates
to difficulties associated with assigning financial values to soft outcomes. Another limitation
includes difficulties in determining the counterfactual (what would have happened without
the investment), although ‘deadweight’ calculations are designed to account for this. It is also
important to remember that monetary values presented as a result of SROI analysis (for every
£1 of investment, a value of £X is returned) is not an actual value. It simply represents a proxy
of an organisation's activity and assumptions regarding holistic value for money rather than
simply changes in financial flows or the generation or cash saving, which can be
communicated in a clear and concise way. This approach is also sensitive to assumptions
made about the validity of outcomes, value of these outcomes and the number of specific
outcomes to be counted. Notwithstanding this, SROIA is a recognised and valued impact
reporting method, which puts stakeholders at the heart of the evaluation.
Economic Evaluation
22
FINDINGS
As discussed earlier, a wide range of stakeholders were consulted to identify appropriate
outcomes to be included in the SROI model. In order to adhere to the rule of SROI to ‘value
the things that matter’, all of these were considered for inclusion. All identified outcomes are
listed below in Table 3.1. Outcomes for which no evidence was identified (i.e. no recorded
instances) are all included in the model, but have no effect on the calculations as they are
indicated by a ‘Quantity of Outcome’ as £0.00. Below table 3.1 there is a list of these outcomes
for which there was no evidence and reasons why.
It is important to note that the large number of potential outcomes with no attributed values
is mostly due to not having adequate measurement processes to capture instances of these
outcomes, rather than the evaluation demonstrating no impact on these outcomes.
Quantification of these outcomes could be included in the model in future, if improvements
are made in outcome capture.
The key outcome themes for which there was evidence identified during the mixed-method
evaluation of the programme are:
- Reduced substance abuse:
o impact on society and
o impact on the individual quality-adjusted life-year (QALYs)
- Improved parenting
- Reduced time of social workers
Minor outcomes, for which there was some evidence of small benefits are:
- Reduced need for accident and emergency health services
- Reduced need for separate housing: Fewer incidents of providing refuge for partners
Table 3.1 details all of the assumptions, values and recorded instances of outcomes. Following
this, the text describes details of all outcomes; including reasons why they were not included
in the final calculations.
Economic Evaluation
23
Table 3.1: SROI assumptions and values
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
Reduction in DVA and repeat incidents: physical health
1 x A&E attendance 2014/15= £132 per incident No evidence available on whether the programme prevents A&E attendance of partners or not. Deadweight has not been calculated, but should take into account comparable A&E attendance rates.
£132.00 0.00 £0.00
Reduced need for victim support: The average cost of giving support to each victim-survivor by an independent advocacy domestic violence scheme = £501 per incident
No evidence available on whether the programme reduces the need for victim support or not. Number of victims receiving support is not known. Deadweight should consider victims no longer needing support due to termination of relationship/imprisonment
£501.00 0.00 £0.00
Reduction in DVA and repeat incidents: mental health
Women suffering DVA used four times as many mental health resources as average. If per capita cost of mental health services are multiplied by 4, this generates a cost of £332 per abused woman.
No evidence available on whether the programme results in reduction of mental health services used by partners
£332.00 0.00 £0.00
Reduced accident and injury
Reduced A&E attendance of client 1 x A&E attendance 2014/15= £132.
A&E attendance of cohort (55 cases=20%) compared to national average 29-65 year olds (13%). There was little evidence to suggest that there is a reduction in A&E attendance as a result of the intervention. However, a reduction in violent behaviour and alcohol consumption was reported, and the clients had a slightly (7%) higher incidence of attendance than the national average. It could be claimed as a reasonable assumption that as a result of the intervention 50% of clients reduce A&E attendances to the national average: 50% of difference would=2 cases, or 0.2 per quarter. As there is low confidence around this assumption, reference should be made to the effect of not including this assumption of impact as shown in the sensitivity analysis.
£132.00 0.20 £26.40
Economic Evaluation
24
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
Improved mental health
Completing a screening questionnaire (£33), followed by care management for those found to be suffering from, or at risk of developing, depression and/or anxiety disorders (£254 for 6x face to face CBT). Not included in calculations as this is an output not outcome. Evidence of improved mental health is required
Screening only is conducted with health questionnaire. 10% deadweight assumes that 1 in 10 clients receive alternative mental health screening within health services. 5% per quarter drop off estimates that repeat screening might be required.
£33.00 0.00 £0.00
Reduced need to provide refuge for victims
Family and friends who help with accommodation estimated at the same rate as Housing Benefit for 28 weeks = £1,509 for each woman and her family. Per quarter =£647. The 2001 BCS IPV found that 39% of women moved out, at least temporarily, after the incident of domestic violence that they defined as the worst. Of these 90% went to stay with family and friends, while 4% went to a refuge. The accuracy of assumptions for this benefit would be improved with evidence to indicate how long a victim might spend in alternative accommodation; this would allow a more informed drop-off value to be applied
Not many clients were co-habiting on access to the programme. No evidence is available on whether there was an increase in co-habitation. However, 5 clients specifically described [in routine data] improved relationships with partners. The assumption is that, for the partners of these clients, this has prevented escalation of DVA to the point that they feel the need to move out. Drop off at 50% per quarter allows for the temporary nature of living in alternative accommodation. It is assumed that on average half of all partners would return home every 13 weeks anyway, without the intervention (<1% benefit after 2 years).
£647.00 0.50 £323.50
Average non-London Refuge rent/week = £416.29: 1 month x4= £1,665.16
Few clients were co-habiting on access to the programme. No evidence of increase in co-habitation. Very small percentage of women leave home to stay in a refuge.
£1,665.16 0.00 £0.00
Reduced substance abuse
The average harmful drinker costs society £2,500 annually; the average Class A drug taker costs society £13,200 annually in the UK (2009 prices). Alcohol-related problems, crime and anti-social behaviour currently cost society over £25 billion a year. This financial proxy was used in a New Philanthropy Capital case study of an O2 Think Big grant given to a youth club in Carlisle.
Case notes report recognising problem and subsequent stopping or moderating drinking. Drop off at 10% to account for relapse rates (Finney & Moos, 1991) Search of 31 case notes for high number of references for 'Alcohol' (x 4 cases=13%: 41, 57, 35 & 14) revealed a number of ‘FIT Kit’ one-one sessions for alcohol abuse (6, 8, 4 & 3) average number = 5.25 sessions.
£738.27 3.65 £2,281.25
Economic Evaluation
25
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
Costs have been inflated to 2009 prices (from the baseline of 2003 figures). These costs apply across central and local government, not just the NHS Valuation Source: Cabinet Office and PM's Strategy Unit (2003) Alcohol misuse: how much does it cost? Report Source: NPC O2 Think Big case study: Danny Boyle's Youth Centre Reference: Value £2500 per harmful drinker per year (£625 per quarter, adjusted for inflation to 2017= £738.27). 100% attribution and 10% drop off. (http://www.globalvalueexchange.org/ valuations/8279e41d9e5e0bd8499f2bed)
Assume this is representative of cohort (13% of 281)= 36.5 cases total over programme = 3.65/quarter.
Reduced substance abuse
Universal screening by GPs of all patients, followed by a 5-minute advice session for those who screen positive for alcohol misuse = £18.40 [28.1 clients avg per quarter]; This valuation of a similar service is included for information only as this is an output rather than an outcome.
The programme included screening followed by intensive group session on substance abuse rather than a 5-minute advice session as defined in the valued intervention. So costings are very conservative in terms of the intervention. Zero Deadweight: evidence shows that screening in primary care is at a very low level. 5% per quarter drop off estimates that screening will be required at least every 5 years on average
£18.40 0.00 £0.00
Reduced substance abuse
Prevention of harmful drinking: quality- adjusted life years. FIT Kit 1:1 alcohol sessions. The state of the nation facts and figures on England and alcohol (alcoholconcern.org.uk/assets/files/PressAndMedia/state.of.the.nation.pdf.) 12.1 million harmful drinkers £21.9 billion in human costs (reduced quality- adjusted life years). Value £1809.91 per person per year (/4)= £452/quarter
Text search of 31 case notes for high number of references for 'Alcohol' (x 4 cases=13%: 41, 57, 35 & 14) revealed a number of FIT Kit one-one sessions for alcohol abuse (6, 8, 4 & 3) average number = 5.25 sessions. Assume this is representative of cohort (13% of 281)= 36.5 cases total over programme = 3.65/quarter. Case notes report recognising problem and subsequent stopping or moderating drinking. Drop off at 10% to account for relapse rates (Finney & Moos, 1991)
£452.00 3.65 £1,649.80
Reduced demand on
F4C staff members assist social workers: Time per hour of Children's Services Social Worker @ £54 per hour of client-related work (5x£79=£395 including
Case note reviews demonstrated close working with social workers and sharing of detailed information, by email and phone calls @ 5 Hours contact per quarter
£270.00 10.30 £2,781.00
Economic Evaluation
26
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
other agencies: Children
cost of qualifications). 5x£54=£270. 103 total referred by children's services or self-referral via Children's services
per client (4x1 hour one-to-one plus conservative estimate of information gathering from the group sessions at 1 hour per client from a total of 24 hours). Total 5 hours. Deadweight is zero, as intensive access to voluntary perpetrators is not managed by other agencies. Attribution is 100%. Drop off is zero as this is an ongoing and renewed process for each client
Outcomes for children: Improved parenting
Fiscal savings to public sector agencies derived from the delivery of parenting programmes for parents of five-year old children with conduct disorder -; the research assumes that, for parents completing programmes, 33% of children improve to 'no problems' and 5% to 'moderate conduct problems', but behaviour changes are not sustained beyond one year for 50% of children who initially improve. Attribution is 100%, as all outcomes are attributable to the programme. Deadweight is zero as it is assumed that clients do not access similar services, drop off is calculated at 0, as it is built in to the model. http://www.globalvalueexchange.org/valuations/584580a71c584a1f40e4384
Programme evidence: Behaviour questionnaire ‘The children have not been affected by the abuse’ T0=(17) 40.5, T1=0 (0), T2=04 (100). These results support the programme theory that increased awareness of the effect of their behaviour (T0-T1 children reported as being affected by DVA 59.5%-100%), results in changed behaviour (T2=0%). Limitations of this evidence are that small numbers completed T2 questionnaires and the analysis of responses was not linked to individuals. However, this is also supported by the qualitative findings regarding the reported impact of the session on the effects of DVA on children and corresponding reports of changes in behaviour to protect children from the harms of DVA. Age 6 (savings over the first year following delivery of the parenting programme), £324; age 7-16 (savings for years 2-11 after programme delivery), £2,493 (averaging out at £249 per year over the ten year period); age 17-30 (savings for years 12-25 after programme delivery), £551 (£39 per year over the 14 year period). The outcome is therefore valued at £324 per year for 4 quarters + £249 per year for 6 quarters (£373.50) (697.50/10=£69.75 per quarter). Only applicable to completed clients with children n= 66 (6.6 per quarter).
£69.75 6.60 £460.35
Economic Evaluation
27
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
Outcomes for children: Mental health
Mental Health Foundation (2005) report one pilot study, of children aged 4-8 referred with conduct disorder, found that the mean extra cost was £15,282 a year (range £5,411-£40,896).
No current evidence available on whether or not the programme prevents children developing MH problems. Possible external evidence of incidences of CAMH problems in abusive households and effectiveness of similar interventions. Zero deadweight and 100% attribution should be applied if only prevented incidences resulting from the programme are counted
£3,820.50 0.00 £0.00
Outcomes for children: stress management
Improved relationships with children and reduced exposure to DVA, assumes that stress management services will not be required. Stress management course for young person £630 (UK council for psychotherapy). This value is not included in calculations due to the possibility of double counting with benefits for improved parenting
Clients specifically reported better relationships with children and better understanding of the effects of DVA on children [3 cases from routine data]. However, as this is not specific routinely collected data, more cases are possible. Deadweight is zero, as there is no evidence that the clients would seek or be able to access similar behavioural change and educational interventions. Self-reported attribution is 100%. Drop off at 10% per quarter is an estimation of the likelihood that clients will use the knowledge they gain less as time goes by and they forget the learning.
£630.00 0.00 £0.00
Improved life decisions
Make more informed decisions and more motivated to change. Life coach (minimum for 10 weekly sessions) = £600. completers per quarter=12.1. This valuation is included for information only. It is not included in calculations as provision of life-coaching is an output not outcome. Despite this being considered an important outcome by stakeholders, it was not possible within the scope of this study to adequately define and measure ‘improved life decisions’.
Motivational interviewing and other behaviour change techniques embedded in sessions over 12 weeks: completers only. Deadweight is zero, as there is no evidence that the clients would seek or be able to access similar behavioural change and educational interventions. Drop off at 10% per quarter is an estimation of the likelihood that clients will use the knowledge they gain less as time goes by and they forget the learning. Attribution = 60% (i.e. 6 sessions) to prevent double counting of parenting sessions, valued above.
£600.00 0.00 £0.00
Improved employability
Assume savings for the state are equivalent to the amount a family can earn on benefits in a year £500/
No evidence available on changes in employability. £122,200.00 0.00 £0.00
Economic Evaluation
28
Outcome Relationship/Assumptions Additional assumptions Value of outcomes per quarter
Quantity of Outcome
Total Value
(employment and attendance at work)
couple and single parent and £350/single per week for one year: value £122,200
F4C clients had a lower rate of employment than the general Doncaster population (44.1% compared to 72.5%); more clients were looking for work (34.5% compared to 26.4%). However, there were much lower numbers (1.8%) than the general population receiving disability allowance (1.2%) or long-term sick (6.5%). This is probably due to the relatively low age of the F4C clients.
Social Work & Adoption Foster services
Adoption: cost for placement and ongoing support = 1 child=£27,000; 2 children = 150%; 3 or more children= 200% Plus £750/month ongoing.
We were not able to access data regarding any reduced need for adoption or statutory child care. Deadweight and attribution (estimated at 50%) will depend on quality of the evidence and causal links of outcomes with the programme.
£27,000.00 0.00 £0.00
Fostering £603 per child per week No evidence available to indicate whether the programme reduced the need for adoption or statutory child care
£603.00 0.00 £0.00
Local authority care £3,181.60 per resident week No evidence available to indicate whether the programme reduced the need for adoption or statutory child care
£3,181.60 0.00 £0.00
Criminal justice system: Police call outs
Assuming reduced call outs @ common assault £215; Other £1,000; serious wounding £9,127; sexual offences £3,837
No evidence available to indicate whether the programme resulted in reduced call outs. However 55/275 previous arrests or convictions for assault (and assault with other crime) in previous 6 months
£215.00 0.00 £0.00
Educational services
SROI study of troubled families attributed costs for Education Welfare Officer of £1188
Some evidence from case note review of reduction in school absenteeism from between 60%-100%. However, this evidence was not included in calculations as it was not reliably or systematically collected. And deadweight will be high because of involvement of other agencies e.g. The child has as a keyworker/counsellor who she can see every morning
£1,188.00 0.00 £0.00
Total £8,605.10
Economic Evaluation
29
3.1 Difficulties in measuring outcomes
A brief summary of difficulties in measuring instances of outcomes is given below. Ways to
address these should be considered as recommendations for evaluating SROIA for
perpetrator programmes in future.
Better mental health/well-being: Whilst validated mental health assessment tools are used
(WEMWBS, PHQ-9 & GADS), these are only completed at one point during the intervention.
If these were completed pre and post completion of the course, a value could be placed on
any identified change.
Make more informed decisions: This is a rather vague concept, which was not consistently
articulated by stakeholders during the evaluation, and could be applied to a very wide range
of decisions. It would be beneficial to explore areas where clients are making poor decisions,
and aspects of the course that are aimed at targeting these decision making processes. To a
large extent this process has been applied in the case of decision making around the use of
alcohol and drugs. However, there is scope to apply this in other areas, and then use measures
that target specific outcomes.
Increased motivation to change: a particular problem with measuring this outcome is that
motivation to change is an important part of the initial assessment process. Clients are
therefore encouraged to display high levels of motivation initially, which might affect
measurement of this outcome. The clients also have complex motivations that are usually
high at the time of contacting the service, such as gaining access to children, which can be
significantly affected by external factors.
Reduced incidents of DVA and repeat incidents: In order to establish changes in rates of DVA,
clients would need to report behaviour following the course. The large number of clients that
completed the behaviour questionnaire, but were then lost to follow up, demonstrate the
difficulties of longitudinal data collection in this field.
Reducing time that victims spend in abusive relationship: This is a particularly difficult
concept to define. Measurement depends on a number of factors including whether the
Economic Evaluation
30
relationship has ended, or abusive behaviour has ceased. In addition, the impact of the course
on these factors is very difficult to attribute.
Lower levels of worry/ Improved family relationships: A number of instances of improved
family relationships were reported by clients. However, this concept could be viewed as a
proximal outcome, or a necessary precursor of outcomes that are more straightforward to
measure and valued (such as reduced need for friends and family to provide accommodation.
Children: improved wellbeing/mental health and fewer children involved in incidents of
DVA: evidence was not collected on incidents of DVA perpetrated on children. Clients
reported positive effects of improved understanding about the impact on children of
witnessing DVA, but as with reduced incidents of DVA, this is very difficult to measure.
However, the perceived impact of DVA on children was collected using the behaviour
questionnaire, which supported the hypothesis that the course had an effect upon the impact
of DVA on children. The large numbers lost to follow up, is a concern in collecting these data.
Improved school attendance: There is a self-reported item on the behaviour questionnaire
to measure school attendance. However, low numbers of completions (especially for follow
up) means that no meaningful data could be collected.
Increased number of referrals to victim services: Mechanisms for linking F4C and victim
services were found to be lacking. Attempts were made to improve these linkages throughout
the duration of the study.
Changes in type of support requested from other agencies (including less crisis support): It
is unlikely that the programme will have a significant effect on the types of work carried out
by other agencies as these services have thresholds and criteria for triggering an intervention,
which are not necessarily dependent on characteristics of the F4C clients.
Preventing escalation to a multi-agency risk assessment conference (MARAC): this outcome
is very challenging to measure in terms of the counterfactual evidence, as these clients
represent a subset of potential MARAC cases, which cannot be identified without them taking
Economic Evaluation
31
part in the course. It is therefore particularly difficult to estimate the effect of the programme
on this outcome.
Reduced number of children requiring adoption: It is unclear as to whether the programme
has an effect on this outcome. Case notes revealed that once actions have begun, attendance
on the course is unlikely to change the course of events leading to children being taken into
care or adopted. Further evidence could be sought through follow-up with Children’s Services
to seek expert opinions and examples of cases where attendance on the programme might
have had an effect. In cases where the authorities are not proceeding to remove children, it
is very difficult to establish the counterfactual (i.e what would have happened without
attendance on the programme).
Reduction in incidents of abusive behaviour reported to police or other agency (Reduced
numbers of police call outs/policing cost reduced rate of domestic abuse): Detailed evidence
from police regarding individual clients was not available during the study. This could be
difficult to acquire for large numbers of service users.
Attendance of employee/perpetrator/ victims at work; performance of perpetrator at
work: This outcome was not systematically collected. Gaining information about victims is
particularly challenging for a voluntary perpetrator programme. Information about clients
would also benefit from longer-term follow up, which is a particular challenge for this client
group.
3.2 SROI Calculations
Tables 3.3 & 3.2 show the SROI ratio of the programme from the start of the programme to
the end of a five-year period of impact calculation. The reported SROI model assumes the only
investments into the programme are those actually input during the study period of two and
a half years. By quarter three of year three, the total value of the investment is therefore fixed
at £382,500, After the first two and a half years, there is zero cost of the programme and the
quarterly benefits begin to decline due to drop off and discounting.
Economic Evaluation
32
The programme does not quite achieve pay back during the study period of two and a half
years. After achieving the benefits of the final quarter, the SROI ratio is £0.96-£1.00, or 96
pence of social value for every pound spent. However, many of the social benefits of the
programme (particularly regarding benefits for children and health and social costs of
substance abuse) are long term and even allowing for drop off and discounting they continue
to accrue. By quarter four in the third year, the SROI ratio becomes positive (£1.10) and the
programme achieves pay back on investment. By the end of year five the SROI ratio is showing
greater than double the return on investment (£2.05).
3.2.1 Sensitivity Analysis
Table 3.2 below shows the counted elements of the SROI model. These values and
assumptions were tested to assess what difference adjustments would make to the final
calculation. Table 3.5 presents low sensitivity analysis. This low estimation is required for the
programme to not demonstrate a positive return over a five-year period. Table 3.6 shows the
returns on investment over two timescales for a programme funded for two and a half years
(i.e. the actual funding period for the evaluation). The Y3Q3 column demonstrates the SROI
ratio immediately after the funding ceased for a two-and-a-half-year long programme. The
Y5Q4 column demonstrates the cumulative effects of benefits that continue to accrue for a
further two and a half years following the two and a half year funding window (total return
over five years).
A particularly sensitive value included in the model, is provision of a route for engagement
with a difficult to reach cohort in order to assess and intervene to address issues such as;
mental and physical health, housing, finance, employment, substance abuse and criminal
behaviour. The English average (2016) inflation adjusted cost per service user contact for drug
outreach services is £104.39 (Table 3.1). This has been included as the value for a single
contact. However, the minimum number of contacts for completers is between 10 and 12,
and even non-completers would be expected to have more than a single contact with a
support worker. The addition of instances of this value makes very large adjustments to the
assessment of benefits. For instance, if two contacts rather than one are applied, the
Economic Evaluation
33
Table 3.2: Counted SROI Values
Outcome Relationship Assumption Value/Qtr Quantity/Qtr Value
Reduced substance
abuse
Prevention of harmful drinker costs
to society
Case notes report recognising problem and
subsequent stopping or moderating
drinking, 3.65 per quarter.
£738.27 3.65 £2,694.69
Reduced substance
abuse
Prevention of harmful drinking:
quality adjusted life years
(13% of 281)= 36.5 cases total over
programme = 3.65 per quarter
£578.81 3.65 £2,112.66
Improved parenting Fiscal savings to public sector
agencies derived from the delivery
of parenting programmes
All completed clients with children n= 66
(6.6 per quarter).
£82.39 6.60 £543.77
Reduced accident and
injury
Reduced A&E attendance of client 2 cases, or 0.2 per quarter £132.00 0.20 £26.40
Reduced need to provide
refuge for victims
Family and friends who help with
accommodation
Assumption that prevented escalation of
DVA (reported by 5 clients) to the point
that the partner feels the need to move
out
£893.17 0.50 £446.59
Reduced demand on
other agencies:
Especially Children’s
Services
F4C staff members assist social
workers
4x1 hour one-to-one plus estimate of
information gathering from the group
sessions at 1 hour per client
£270.00 10.30 £2,781.00
Total £8,605.10
Economic Evaluation
34
Table 3.3: 0 to 2.5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)
Y1_Q1 Y1_Q2 Y1_Q3 Y1_Q4 Y2_Q1 Y2_Q2 Y2_Q3 Y2_Q4 Y3_Q1 Y3_Q2
Discounted
benefits per
quarter
£0.00 £8,457.10 £16,083.97 £23,055.98 £29,484.54 £35,446.72 £41,000.23 £46,191.09 £51,057.67 £55,632.90
Present Value
(cumulative value
of all benefits)
£0.00 £8,457.10 £24,541.07 £47,597.06 £77,081.60 £112,528.32 £153,528.55 £199,719.64 £250,777.31 £306,410.21
Total value of
Investment/Inputs
£38,250.00 £76,500.00 £114,750.00 £153,000.00 £191,250.00 £229,500.00 £267,750.00 £306,000.00 £344,250.00 £382,500.00
Net Present Value
(value of all
benefits minus
programme costs)
-£38,250.00 -£68,042.90 -£90,208.93 -£105,402.94 -£114,168.40 -£116,971.68 -£114,221.45 -£106,280.36 -£93,472.69 -£76,089.79
SROI £0.00 £0.11 £0.21 £0.31 £0.40 £0.49 £0.57 £0.65 £0.73 £0.80
Economic Evaluation
35
Table 3. 4: 2.5 to 5 year SROI for a 2.5 year funded programme (281 clients over 10 quarters)
Y3_Q3 Y3_Q4 Y4_Q1 Y4_Q2 Y4_Q3 Y4_Q4 Y5_Q1 Y5_Q2 Y5_Q3 Y5_Q4
Discounted
benefits per
quarter
£59,945.53 £56,233.50 £53,061.22 £50,288.80 £47,832.77 £45,639.13 £43,669.87 £41,896.14 £40,294.70 £38,846.05
Present Value
(cumulative value
of all benefits)
£366,355.7
5
£422,589.2
5
£475,650.4
6
£525,939.2
7
£573,772.0
4
£619,411.1
7
£663,081.0
4
£704,977.1
9
£745,271.8
9
£784,117.9
3
Total value of
Investment/Input
s
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
£382,500.0
0
Net Present Value
(value of all
benefits minus
programme costs)
-£16,144.25 £40,089.25 £93,150.46 £143,439.2
7
£191,272.0
4
£236,911.1
7
£280,581.0
4
£322,477.1
9
£362,771.8
9
£401,617.9
3
SROI £0.96 £1.10 £1.24 £1.38 £1.50 £1.62 £1.73 £1.84 £1.95 £2.05
Economic Evaluation
36
Table 3.5: Low sensitivity analysis
Outcome Relationship Assumption
Reduced substance
abuse
harmful drinker costs to society Increased drop off from 10% to 20% to assume higher relapse rate and reduced
attribution from 100% to 50% to allow for other service interventions having an impact
Reduced substance
abuse
Prevention of harmful drinking:
quality of life adjusted years
Increase drop off from 10% to 20% to assume higher relapse rate and reduced attribution
from 100% to 50% to allow for other service interventions having an impact
Improved parenting Fiscal savings to public sector
agencies derived from the
delivery of parenting programmes
No change to assumptions; there is good evidence that clients learn about and are
strongly affected by the potential psychological harm to children caused by their abusive
behaviour, and respond well to learning about improved parenting. The research evidence
for valuing benefits is good and is conservative in claims about benefits.
Reduced accident and
injury
Reduced A&E attendance of
client
Assume no cases of reduced attendance
Reduced need to
provide refuge for
victims
Family and friends who help with
accommodation
Assume that the course only contributed 50% rather than 100% attribution to this
outcome
Reduced demand on
other agencies:
Especially Children’s
Services
F4C staff members assist social
workers
Assume deadweight increase from 0% to 50% as information could come from alternative
sources
Economic Evaluation
37
Table 3.6: Sensitivity analysis comparisons
Y3-
Q3
Y5-
Q4
Breakeven
point
Original SROI analysis 0.96 2.05 Y3_Q4
Assume no value for housing or A&E 0.93 2.02 Y3-Q4
Increase to 20% drop off for alcohol interventions 0.85 1.72 Y4-Q1
Add deadweight for reduced demand on other services to 50% 0.75 1.55 Y4-Q2
Increase to 20% drop off + reduce attribution to 50% for alcohol
interventions
0.66 1.44 Y4_Q3
Assume no value of reduced demand on other services 0.58 1.10 Y5-Q2
All low SROI assumptions (as shown in table 3.6) 0.46 0.95 -
programme demonstrates a marginal cost saving of £2,743 per year. However, to prevent
double-counting of activities, we have applied the value to the effort required for the first
contact only.
The sensitivity analysis demonstrates that the model returns a positive SROI ratio over five
years except when all assumptions are considerably reduced (table 3.5). Key assumptions that
have important sensitivity implications and therefore would benefit from further
investigation are the effect of the alcohol abuse intervention within the programme, and the
relapse rates of participants. The other assumption that has important implications for the
model, and would benefit from further investigation, is the value of information gathering,
communication and collaboration with other services, specifically children’s services. The
evidence available suggests that the F4C programme delivers a social return on investment of
between £2.05 and £0.95 over five years, for a two and a half year funding period. However,
this estimate could be much higher as there are additional potential benefits, included in the
model, that were not valued as there was no evidence of outcomes available. If ongoing
funding is included in the model, after five years the net present value is forecast at £302,208,
with a SROI ratio of £1.40 and a break-even point of the 1st quarter in year 4.
Economic Evaluation
38
DISCUSSION The SROIA attempts to place values on the benefits of outcomes from the programme; from
the perspective of various stakeholders and society in general. This is a useful approach for
considering the value of interventions from a broader perspective than simply organisational
or individual fiscal effects. The approach is particularly suited to social programmes that have
a wide range of long-term effects. It can be used to make judgements about whether it is
likely that a programme demonstrates a net return to society compared to the societal costs
of delivery.
Voluntary DVA perpetrator programmes are notoriously problematic to evaluate, particularly
regarding obtaining accurate and long term outcome measures, whilst maintaining
confidentiality, gaining consent and working with highly sensitive data from a range of
organisations. For these and other reasons, described in detail above, difficulties were
experienced in measuring and valuing assumptions about expected outcomes. Therefore,
there are many potential outcomes that might have provided additional value to the SROI
analysis, but which were not included in the final calculations. The model should therefore be
considered a conservative estimation of returns on investment; although it should be
recognised that some possible negative influences were also not able to be collected. In
particular, the study did not systematically collect evidence of clients accessing other services
so no attribution to external influences was calculated for the SROI ratio. However, attribution
was deducted in the sensitivity analysis to estimate the effect on the final calculation of
possible external influences on outcomes.
In addition, no displacement was calculated. This is potentially a significant limitation as there
is also the possibility that the programme could result in negative impacts, such as those
resulting from disguised compliance, clients learning new forms of abusive behaviour, and
false feelings of security for victims. In providing details of all outcomes, whether calculated
or not (in Table 3.3), it is possible for users of these evaluation findings to use these key
outcomes in future SROIA, and make adjustments to the calculations to potentially arrive at
more accurate findings for other services. It is also worth noting that little accurate evidence
was available for the rate of drop-off for some of the effects of the programme.
Economic Evaluation
39
Whilst the calculations shown above are for returns calculated for the actual two-and-a-half-
year period of the study, it is also worthwhile to demonstrate what the return might be if the
programme were funded for a longer period of time. The findings of the SROI analysis indicate
that if this programme were funded for five years, the SROI ratio would be a £1.40 return for
each £1.00 invested in the programme, and it would break even in the first quarter of year
four. However, a large number of outcomes that were identified by stakeholders as being
important to take into consideration were not able to be counted.
There are only five outcomes that demonstrate large benefits (harmful drinking costs to
society and the individual, improved parenting, reduced need to provide refuge for victims
and reduced demand on other agencies). There are fourteen outcomes that are not counted,
and therefore the results are highly uncertain. Despite this, a sensitivity analysis showed that
if the programme were only funded for two and a half years and very low estimates of benefits
were applied, then the return after five years would be approaching break-even point at
£0.95. In terms of benefits to society, organisations and individuals, it can therefore be stated
with some certainty that, over the medium to long-term, the programme returns higher value
than the resources required to provide it.
The voluntary perpetrator programme has a number of outcome values that are realised over
the long term. For instance, improved parenting can result in social value even once children
have become adults. There is strong evidence that the legacy of poor parenting can contribute
to significant social costs later in life, such as the costs related to victims of crime.
Interventions when children are five years old are often calculated on a twenty-five-year
cycle, until the child is thirty years old. However, for the purpose of this study only a five-year
period is considered, which means that some benefits may be underestimated.
One of the key benefits of SROI methodology is that it does not apply normative principles,
but rather is contextually determined. Users of these findings would be encouraged to use
their judgement and established SROI methods to understand the validity of the benefits
counted in this model for their own situation.
Economic Evaluation
40
RECOMMENDATIONS FOR FUTURE RESEARCH
Future economic evaluations of voluntary DVA perpetrator programmes should consider the
difficulties encountered with this study, and seek to identify measures and data collection
mechanisms for a wide range of outcomes. Many have successfully been developed for this
study. However, there are considerable advances that could still be made; some of which are
described in detail in this report and in the companion report for the full evaluation.
The current difficulties of obtaining good quality outcomes data for these types of services
puts in question the desire from policy-makers and commissioners to have evidence of
returns on investment (ROI). It should be recognised that there are significant difficulties
involved in demonstrating ROI and improvements need to be made regarding putting
measures in place. It should also be recognised that interpretation of findings requires a deep
understanding of the methodology, including the strengths and weaknesses.
Two pervasive problems for evaluations of this type of programme are the loss of clients to
follow-up and difficulties in obtaining outcome measures for victims and families. To an
extent, these could be gathered using proxy measures during the window of engagement with
the service-user. For instance, asking questions about disruption to children’s schooling and
the input of other services. For the purposes of affordable evaluation and monitoring of
services, this could be a more cost-effective approach to understanding benefits, although
the disadvantages of relying on clients’ self-reports are clear. Another important area of
further research is the resilience of benefits, which would allow greater confidence in the
values used to calculate drop-off. This evidence could also help in the design of interventions
to more appropriately support perpetrators following the initial intervention.
Future studies should also be aware of the potential for negative impacts, such as those
resulting from disguised compliance, clients learning new forms of abusive behaviour, and
false feelings of security for victims. However, it is not currently apparent how these negative
impacts could be measured. The analysis that has been carried out allows clear
recommendations, including (a) useful approaches for those for collecting and measuring
outcomes and (b) areas that would benefit from future research.
Economic Evaluation
41
REFERENCES Akoensi, T. D., Koehler, J. A., Lösel, F., & Humphreys, D. K. (2013). Domestic Violence
Perpetrator Programs in Europe, Part II A Systematic Review of the State of Evidence. International journal of offender therapy and comparative criminology, 57(10), 1206-1225.
Alhabib, S., Nur, U., & Jones, R. (2010). Domestic violence against women: Systematic review of prevalence studies. Journal of family violence, 25(4), 369-382.
Ali, P., Ariss, S., Powell, K., Crowder, M., & Cooper, R. (2017). Doncaster Domestic Abuse Perpetrator Programme: Evaluation Report, 2017. Retrieved from https://www.sheffield.ac.uk/snm/research/doncaster
Arvidson, M., Battye, F., & Salisbury, D. (2014). The social return on investment in community befriending. Int J Public Sector Manag., 27. doi:10.1108/ijpsm-03-2013-0045
Banke-Thomas, A. O., Madaj, B., Charles, A., & van den Broek, N. (2015). Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health, 15(1), 582. doi:10.1186/s12889-015-1935-7
Bellis, M. A., Hughes, K., Perkins, C., & Bennett, A. (2012). Protecting people promoting health: a public health approach to violence prevention for England. Liverpool: Department of Health.
Bonomi, A. E., Anderson, M. L., Rivara, F. P., & Thompson, R. S. (2009). Health Care Utilization and Costs Associated with Physical and Nonphysical‐Only Intimate Partner Violence. Health services research, 44(3), 1052-1067.
Cabinet Office. (2016). Social Value Act: information and resources. London: Cabinet Office, UK Retrieved from https://www.gov.uk/government/publications/social-value-act-information-and-resources.
DiClemente, C. C., & Prochaska, J. O. (1982). Self-change and therapy change of smoking behavior: A comparison of processes of change in cessation and maintenance. Addictive behaviors, 7(2), 133-142.
Doncaster Domestic Abuse Service. (2016). Doncaster Domestic Abuse Strategy 2016-2020. Retrieved from http://doncasterdomesticabuse.co.uk/information-professionals
Farmer, A., & Tiefenthaler, J. (1997). An Economic Analysis of Domestic Violence. Review of Social Economy, 55(3), 337-358. doi:10.1080/00346769700000004
Finney, J. W., & Moos, R. H. (1991). The long-term course of treated alcoholism: I. Mortality, relapse and remission rates and comparisons with community controls. Journal of studies on alcohol, 52(1), 44-54.
Hamilton, L., Koehler, J. A., & Lösel, F. A. (2012). Domestic violence perpetrator programs in Europe, part I: A survey of current practice. International journal of offender therapy and comparative criminology, 0306624X12469506.
Miller, W. R., & Rollnick, S. (2012). Motivational interviewing: Helping people change: Guilford press.
Nef consultancy. (2013). Social valuation of Refuge services for survivors of domestic violence. nef consultancy report for Refuge Retrieved from London: http://www.refuge.org.uk/files/Refuge-SROI-report-25-09-13-NCV2.pdf
Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2009). A guide to social return on investment. London: Office of the Third Sector, The Cabinet Office.
Economic Evaluation
42
Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A Guide to Social Return on Investment. London: The Cabinet Office.
Perfect Moment. (2010). Strength to Change Return on Investment Study. Retrieved from Cornwall: http://www.ccrm.org.uk/images/docs/8.3apm%20stc%20roi%20exec%20summary%2030nov10.pdf
Prochaska, J. O. (2013). Transtheoretical model of behavior change Encyclopedia of behavioral medicine (pp. 1997-2000): Springer.
Prochaska, J. O., & DiClemente, C. C. (1986). Toward a comprehensive model of change Treating addictive behaviors (pp. 3-27): Springer.
Rollnick, S., & Miller, W. R. (1995). What is motivational interviewing? Behavioural and cognitive Psychotherapy, 23(04), 325-334.
Rotheroe, N., & Richards, A. (2007). Social return on investment and social enterprise: transparent accountability for sustainable development. Social Enterprise Journal., 3. doi:10.1108/17508610780000720
UK Home Office. (2013). Domestic violence and abuse. Retrieved from https://www.gov.uk/guidance/domestic-violence-and-abuse#history
Willis, E., Semple, A. C., & de Waal, H. (2016). Quantifying the benefits of peer support for people with dementia: A Social Return on Investment (SROI) study. Dementia, 1471301216640184.
Wood, C., & Leighton, D. (2010). Measuring social value: the gap between policy and practice. Undercurrent(2), 7.
World Health Organization. (2014). Violence against women. Retrieved from http://www.who.int/mediacentre/factsheets/fs239/en/
Wright, S., Nelson, J. D., Cooper, J. M., & Murphy, S. (2009). An evaluation of the transport to employment (T2E) scheme in Highland Scotland using social return on investment (SROI). Journal of Transport Geography, 17(6), 457-467.
Yodanis, C. L., Godenzi, A., & Stanko, E. A. (2000). The benefits of studying costs: A review and agenda for studies on the economic costs of violence against women. Policy Studies, 21(3), 263-276.
The research was funded by the NIHR School for Public Health Research
(SPHR) Public Health Practice Evaluation Scheme (PHPES) The views
expressed are those of the author(s) and not necessarily those of the
NHS, the NIHR or the Department of Health.
The NIHR School for Public Health Research is a partnership between the Universities of Sheffield,
Bristol, Cambridge, Imperial and University College London; The London School for Hygiene and
Tropical Medicine (LSHTM); the LiLaC collaboration between the Universities of Liverpool and
Lancaster and Fuse; The Centre for Translational Research in Public Health, a collaboration between
Newcastle, Durham, Northumbria, Sunderland and Teesside Universities