porchlight yps canterbury forecast sroi - can invest · porchlight young persons’ service...

Porchlight Young Persons’ Service Canterbury

Forecast SROI

April 2013 – March 2014

16th August 2013

This report has been submitted to an independent assurance assessment carried out by The

SROI Network. The report shows a good understanding of the SROI process and complies

with SROI principles. Assurance here does not include verification of stakeholder

engagement, data and calculations. It is a principles-based assessment of the final report.

Rohan Martyres

Porchlight YPS Canterbury Forecast SROI 2013

2 P a g e

Contents

1. Executive Summary ......................................................................................................................... 4

1.1. Social Return on Investment (SROI) ........................................................................................ 5

1.2. Methodology ........................................................................................................................... 5

1.3. Outcomes and Theories of Change ......................................................................................... 6

1.4. SROI Ratio.............................................................................................................................. 12

1.5. Caveats .................................................................................................................................. 16

1.6. Insights and Recommendations ............................................................................................ 18

2. SROI Assurance – Summary of Evaluation for Assurance Purposes ............................................. 20

3. Introduction to Porchlight ............................................................................................................ 30

4. Introduction to Social Return On Investment ............................................................................... 32

The Principles of SROI ....................................................................................................................... 32

5. SROI Stage 1: Establishing Scope and identifying key stakeholders ............................................. 33

5.1. Establishing scope ................................................................................................................. 33

5.2. Identifying stakeholders........................................................................................................ 34

5.3. Deciding how to involve stakeholders .................................................................................. 40

5.4. Summary of Stakeholder Engagement ................................................................................. 40

5.5. Details on data collection & sampling .................................................................................. 42

6. Consideration of Materiality ......................................................................................................... 46

6.1. Audit Trail .............................................................................................................................. 47

7. SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies ........... 48

7.1. General Comments – Stage 3: Outcomes and Theories of Change ...................................... 49

7.2. YPS Users ............................................................................................................................... 51

7.3. YPS Users’ families ................................................................................................................ 64

7.4. Partner Organisations ........................................................................................................... 66

7.5. Funders ................................................................................................................................. 77

8. SROI Stage 4: Establishing impact ................................................................................................. 78

8.1. YPS User Impact Factors ....................................................................................................... 78

8.2. Users’ families ....................................................................................................................... 79

8.3. Referral and partner organisations ....................................................................................... 79

9. SROI Stage 5: Calculating the SROI ............................................................................................... 80

9.1. The SROI ratio ....................................................................................................................... 80

10. SROI Stage 6: Reporting ............................................................................................................ 85

10.1. Engaging with Stakeholders .................................................................................................. 85

10.2. Insights and recommendations ............................................................................................. 85

10.3. Verifying the Report .............................................................................................................. 87


3 P a g e

11. Appendices ................................................................................................................................ 88

11.1. Appendix 1. List of Figures and Tables .................................................................................. 88

11.2. Appendix 2. The Stages of an SROI ....................................................................................... 89

11.3. Appendix 3. Stage 2 Interview questions .............................................................................. 90



11.6. Appendix 6. Audit Trail – Staff .............................................................................................. 99

11.7. Appendix 7. Excel Workbook Calculations of YPS User Outcomes & Financial Proxies...... 104

11.8. Appendix 8. Calculations for time inputs & short-term time-savings (Referral organisations). ................................................................................................................................ 105

11.9. Appendix 9. Discussion of Stated Preference methodology ............................................... 107

11.10. Appendix 10. Social Impact Map ......................................................................................... 111

11.11. Appendix 11. Glossary ......................................................................................................... 113

11.12. Appendix 12. References .................................................................................................... 115


4 P a g e

1. Executive Summary

Porchlight is a Kent-based charity that works to help the most isolated and vulnerable in their

communities to access housing and related support services, to achieve their vision of a society

without homelessness or poverty.

Porchlight have supported accommodation projects based in Ashford, Canterbury, Dover, Ramsgate

and Tonbridge, and outreach support for street homeless people across the county. They provide a

range of services supporting people with complex needs such as mental ill-health and have specific

services for young vulnerable and homeless people. One of Porchlight’s primary services for

homeless youth is their Young Persons Service (YPS), which provides supported accommodation one-

to-one personal support to help youth aged 16-21 to achieve education, employment, future

independent living and other outcomes.

Porchlight’s Canterbury-based YPS is run from three accommodation sites:

- One ‘Direct Access’ hostel for 3-6 months emergency residency staffed 24/7. Maximum

capacity: 7 people with high-level support needs.

- One ‘Medium Stay’ hostel for 12 months residence staffed 9-5 Monday-Friday and

occasional weekends. Maximum capacity: 8 people with medium-level support needs.

- ‘Move-on’ Accommodation, not-staffed.

o One house for 6-12 months residency (maximum capacity: 3 people with low-level

support needs)

o One apartment for 6-12 months residency (maximum capacity: 3 people with low-

level support needs )

- Total Capacity: 21 people

Porchlight’s YPS in Canterbury is staffed by 9 full-time personnel and 1 manager, with oversight by

Porchlight’s Director of Operations.

Porchlight commissioned Rohan Martyres from CAN Impact, a social impact consultancy, to evaluate

Porchlight’s YPS in Canterbury, using a Social Return on Investment methodology. The author would

like to thank Gill Bryant, Emily Upfield and other staff at Porchlight for their contributions to the

SROI analysis, to representatives of partner organisations and public agencies involved in this report,

and Naheed Rahman for her comments on an earlier draft of this report. Finally, a special thanks to

all the YPS users who participated in this study, and generously shared their experiences and

opinions about Porchlight.


5 P a g e

1.1. Social Return on Investment (SROI)

Social Return on Investment (SROI) is an approach to understanding and managing the impacts of a

project, organisation or policy. It is based on stakeholders and puts financial value on the important

impacts identified by stakeholders that do not have market values.

SROI measures change in ways that are relevant to the people or organisations that experience or

contribute to it. It tells the story of how change is being created by measuring social, environmental

and economic outcomes and uses monetary values to represent them. SROI is about value, rather

than money. Money is simply a common unit and as such is a useful and widely accepted way of

conveying value. In the same way that a business plan contains much more information than the

financial projections, SROI is much more than just a number. It is a story about change, on which to

base decisions, that includes case studies and qualitative, quantitative and financial information.

SROI, then, can help organisations develop a holistic understanding of the social value it creates, and

importantly, of each aspect in the ‘story of change’ it brings to its stakeholders.

1.2. Methodology

Completing an SROI study typically involves a multi-method approach involving qualitative

interviews, quantitative surveys and desk-research. This Forecast SROI study of Porchlight’s YPS

involved extensive engagement with major stakeholder groups including:

- Porchlight Users:

o Focus groups and 1-1 interviews with 12 people

o A paper survey completed by 22 YPS users. Given total estimated population of 50

YPS users p.a., responses are subject to a margin of error of approximately 16%.1

o Follow-up and feedback with 5 people.

- Staff:

o Focus groups with 9 people

o A paper survey completed by 9 staff

o Follow-up interviews with 3 people

- Referral and partner organisations

o 1-1 interviews with 5 representatives of organisations including charitable partners,

social services teams and YPS’ commissioner at Kent County Council Supporting

People Programme.

o Review of full report by YPS commissioner

1 For details, see Section 5.5 Details on data collection & sampling.


6 P a g e

A summary of evaluation activity and processes according to SROI independent assurance principles

is provided in Section 2 of this report from page 20.

1.3. Outcomes and Theories of Change

1.3.1. YPS Users

The outcomes that YPS Users are likely to experience as a result of their stay at Porchlight were

determined through focus groups, interviews and previous research. These outcomes are:

Shorter-term (immediate) outcomes:

- Having a ‘home base’ / safe accommodation to live in

- Improved ability to live independently, including self-care and living skills such as cooking,

shopping, budgeting and managing debt

Short- to medium-term outcomes:

- Improved physical health

- Reduced alcohol and drug misuse

- Improved mental and emotional health

- Stronger, more supportive social support networks, including friendships, relationships and

relationships with family

- Spending time more meaningfully, including through study, training/education and gaining

formal qualifications, employment, and volunteering.

Medium-term outcomes:

- Reducing offending and interaction with police and justice system.

- Being more motivated in life and feeling able to take responsibility

- Being better able to manage a private tenancy and your accommodation

Longer-term outcomes are:

- Increased long-term likelihood of employment

- Reduced likelihood of being long-term homeless (in adulthood)


7 P a g e

Negative Outcomes

- Expulsion from Porchlight for repeatedly breaking house rules

- Disruption to work (if YPS users previously lived outside Canterbury)

- Disruption to relationships (if YPS users previously lived outside Canterbury)

It is hypothesised that these outcomes are connected together in a ‘theory of change’ as follows:

Through staying in Porchlight accommodation and receiving support from dedicated key-workers,

YPS Users first (1a) gain safe accommodation and a home ‘base’ from which to live their lives, (1b)

make new friends and gradually experience some improvements in the relationships with extended

family, and (1c) learn practical skills to mange money and live independently. Second, as a result of

these changes, some YPS users (2a) experience improvements in their mental and physical health and

(2b) use their time more productively by engaging in education, vocational and other forms of

training and employment. Third, at the same time some YPS users also (3a) gain in confidence and

maturity in their interactions with others and setting life goals for themselves, (3b) engage less in

anti-social and criminal behaviour that bring them into contact with the police and the justice

system, and (3c) more generally gain the strength and ‘resilience’ to respond to negative events and

‘life’s knocks’ without losing confidence or reverting to damaging behaviours that would result in

being excluded from independent housing or the family home. Fourth, some YPS users experience

negative outcomes: (4a) disruption to existing friendships and work when they move from outside

Canterbury into YPS Canterbury, and more significantly (4b) potentially becoming homeless again

following expulsion from Porchlight due to repeatedly breaking house rules. Fifth and finally, as a

result of the foregoing changes, a number of YPS users will (5a) have significantly increased likelihood

of achieving long-term employment in adulthood and (5b) avoid adult homelessness.

This ‘theory of change,’ is reproduced in visual form below, with estimated numbers and

percentages of YPS Users experiencing each outcome. The estimated numbers are based on survey

results and Porchlight data as outlined in the main body of the report.


8 P a g e

Figure 1. Visual Theory of Change of forecast outcomes for YPS Users and their families

All of these outcomes are valued as part of this SROI analysis, with financial carefully proxies

selected to ensure there is no ‘double counting’ of outcomes when calculating the final SROI ratio

(see Section 7.2.5 on page 61 for further details).

1.3.2. YPS Users’ families

YPS Users’ improved relationships with their families is also an outcome for their families. It is

estimated that families of 23 users (46% of total) will experience this outcome.

A further ‘outcome’ for users’ families is the avoided costs of their child’s upkeep. This includes food

and board, clothing, leisure and recreation and so forth. Caveats regarding family outcomes are

outlined below.


9 P a g e

1.3.3. Partner organisations

The improving situations of YPS Users, and the support they receive from Porchlight key workers,

result in a range of further outcomes for partner organisations, including:

- Local Authority social services, including children’s services and adult services - Other charity providers, particularly a youth drug & alcohol support agency - Canterbury College - NHS: mental health services, GPs and hospital A&E departments - Police and the justice system

It is hypothesised that through the work of Porchlight’s YPS, homeless youth:

(1) No longer fall under the local authority duty of care, which discharges Local Authorities from

their housing obligations.

(2) Engage more constructively with partner organisations, which results in YPS users:

a. Making quicker short term progress with partner organisations and thereby a short-

term time saving for partner organisations, and

b. Staying in college for longer or attending college, which results in increased

(government sourced) revenue for the college.

c. Making greater long-term improvements as described in the previous theory of

change for YPS users, which results in:

i. Increased likelihood of employment as adults, and increased tax revenue for

HMRC

ii. Reduced burden on a range of statutory and other public services including

social services, social housing, police and the justice system, the NHS and

DWP. This reduced burden results in reallocation of public resources away

from homeless youth and adults to other groups in need.

Furthermore:

(3) Partner organisations receive relatively more appropriate referrals from Porchlight than

elsewhere, resulting in a time saving and consequent resource reallocation to other clients.

(4) Porchlight and partner organisations share knowledge which results in partner organisations

more effectively being able to deliver on their own work and meet their own contractual

obligations.

The above hypothesised ‘theory of change’ is outlined in visual format in Error! Reference source

not found. below.

Figure 2 below summarises this theory of change in visual format.


10 P a g e

Figure 2. Theory of Change of forecast outcomes for YPS Canterbury’s partner organisations.

1.3.4. YPS Staff

YPS Canterbury is staffed by 9 people, who each work full-time, including 4 night shifts per month, to

support the young people living in YPS accommodation.

It is important to note that Porchlight has a positive discrimination policy of employing some

individuals that may be unlikely to be employed elsewhere. This includes former users of Porchlight

services. Outcomes for Porchlight staff identified through a focus group and a survey include:

Immediate Outcomes:

- Access to further training, both provided by Porchlight and as time-off to attend courses

from external providers.

- Higher income: Above-sector average pay, albeit reduced by unpaid travel expenses &

untaken holiday/TOIL

- Work with young people (with chaotic lives)

- Act as a key worker to assist young homeless people


11 P a g e


- Gain new skills & work experience, through first post-university position or within a new

sector

- Greater understanding of desired career direction, either through first position post-

university position or within a new sector

- Being better informed about social support systems, by engaging with public services on

behalf of clients.

o This also allows staff to pre-empt obstacles in client work, with outcomes for

clients/users

- Occasional ‘small wins’ with YPS users, by helping them clients materially changing their

personal situation and attitudes in various ways.

- A sense of satisfaction / feeling they are ‘making a difference,’ by helping clients improve

their situation and avoid becoming long-term homeless

- A sense of working within a supportive / close-knit team through the positive work

environment.

Longer-term outcomes:

- Self-affirmation and sense of self-worth, through pursuing work that allows staff to express

who they are and what they believe in.

- Becoming less naive about people and society in general, by seeing what can and cannot be

easily changed regarding homelessness and related issues.

Negative outcomes for staff were that they:

- Become ‘hardened’ and callous/less sensitive to people’s suffering

- Experience increased stress & the ‘occasional downer’ due to the pressures of working with

their clients (YPS users) whose lives cannot be cannot be easily improved

- Needed to undertake shift work and extra work hours for out-of-house activities with clients

- Experience disturbed sleep patterns, due to variable work hours and occasional work-related

stress

- Disrupted family/social lives

- Domestic partners needing to take on a greater emotional and practical burden due to

irregular hours

o Partner take on greater emotional/practical burden (family cost)

o Less time spent with friends

- Less patience and energy with family, due to occasional work-related stress


12 P a g e

However, as the SROI analysis developed, it became clear that outcomes for staff were not ‘material’

from an SROI-perspective. Please see the ‘materiality audit trail’ in the main sections for the report

and the impact map in the appendices to this report for more details.

1.4. SROI Ratio

It is forecasted that in the 2013/14 financial year, every £1 invested in Porchlight’s YPS will likely

produce £5.95 of social value, within a confidence range of £4.30p – £7.60p. This is the equivalent of

approximately £86,000 of social value per YPS user in 2013/14 on total inputs of approximately

£15,000 per YPS user.

It is forecast that YPS Service will require funds and social inputs valued at £726,000 £726,500. This

includes (approximately):

- £499,900 in cash funding from DCLG / Kent County Council (Supporting People Programme)

- £207,200 in cash funding from other sources, notably DWP / Housing Benefit.

- £11,600 in costs for YPS Users, including nominal rent payments

- £7,800 worth of inputs from other stakeholders

Based on these inputs, it is forecast that the Canterbury YPS will create a total of approximately

£4,322,000 worth of social value. Of this amount:

- £1,900,000 of social value will accrue to YPS Users (£12,300 per user), who experience a

range of outcomes as outlined in section 1.3.1 on page 6 above.

- £2,248,000 of social value will accrue to Local Authority Housing and Social Services (£45,000

per YPS User) who avoid costs of housing homeless individuals and can allocate resources

away from supporting homeless people to other groups in need.

o £1,232,000 of this will be accrued in the short term, due to cash savings from local

authority legal obligations to house homeless youth in alternate forms of

accommodation, such as bed and breakfasts or foster care, during the time they are

at Porchlight.

o £754,000 of this will be accrued in the medium term, due to cash savings from local


accommodation, such as bed and breakfasts or foster care, between the time they

leave Porchlight and the age of 18.

o £262,000 of this will be accrued in the long-term, due to resource reallocation away

from YPS preventing youth becoming adult homeless.

- £101,000 of social value will accrue to YPS Users’ families


13 P a g e

- £73,000 of social value will accrued to the NHS.

A proportional breakdown of this social value is provided in Figure 3 below.

Figure 3. Breakdown of YPS Canterbury forecast social value by major stakeholder group

1.4.1. Specific outcomes

From an SROI perspective, any given outcome would be considered important to the extent that it

contributes a large proportion of the total social value created or destroyed by the service. As a

result, the final SROI ratio is relatively sensitive to changes in outcomes of ‘large’ value. Table 1

below lists stakeholders & outcomes in order of their contribution to the total amount of social

value created.


14 P a g e

Rank Stakeholder Outcome £ Social Value

(PV)

% of

Total PV

1 YPS Users Reduction in long-term homelessness £1,597,416 37.0%

2 Local Authority Housing

Services (short-term)

Increased cash resources to dedicate to other youth, resulting from avoided

legal obligation to house homeless youth (as they are staying with Porchlight)

£1,231,616 28.5%

3 Local Authority Housing

Services (medium-term)

Medium-term resource reallocation due to avoided legal obligation to house

homeless youth (due to reduced need to housing for youth immediately post-

Porchlight):

£ 753,835 17.4%

4 Local Authority Social &

Housing Services (long-term)

Long-term resource reallocation due to reduced need to manage adult

homelessness

£ 261,915 6.1%

5 YPS Users Increased long-term likelihood of employment £ 184,244 4.3%

6 YPS Users Having a place to stay (safe accommodation) £ 60,007 1.4%

7 Users' families Avoided cost of accommodation/food £ 85,976 2.0%

8 NHS Long-term resource reallocation due to substantially improved physical &

mental health outcomes of homeless adults

£ 73,465 1.7%

9 YPS Users Improved mental and emotional health £ 43,073 1.0%

10 YPS Users Better friendships, relationships and social networks £ 17,706 0.4%

11 Users' families Improved relationship with YPS Users £ 15,493 0.4%


15 P a g e

12 YPS Users (Negative) Expulsion from Porchlight for repeatedly breaking house rules -£ 3,551 0.1%

13 YPS Users Improved physical health £ 1,293 0.0%

14 YPS Users (Negative) Reduced contact with friends due to excessive travel cost/time after

moving to Porchlight

-£ 1,153 0.0%

15 YPS Users Managing money and personal administration, including budgeting and reducing

debt

£ 1,122 0.0%

16 0 0 £ - 0.0%

Table 1. Outcomes in descending order of absolute value.


16 P a g e

An analysis of variations in assumptions behind the most important outcomes informed the

development of the SROI confidence interval (£4.30p – £7.60p of social value for each £1 invested).

The outcomes of most uncertainty (as opposed to outcomes of most variation in value), are

outcomes 1 & 4 above:

- YPS Users: Reduction in long-term (adult) homelessness

- Local Authority Social & Housing Services (long-term): Long-term resource reallocation due

to reduced adult homelessness

Care has been taken to ensure calculations with respect to these outcome are as conservative as

possible, and recommendations (see below) are provided to reduce uncertainties with these

outcomes in future.

1.5. Caveats

Several caveats must be made to the foregoing, with the primary issues affecting this SROI outlined

below.

Users – preventing adult homelessness

Access to users once they have left Porchlight is difficult. As a result, and in the absence of a

sufficient sample of interviews with ‘ex’ YPS users, estimates of long-term outcomes achieved by

users remains hypothetical. This is particularly the case in assessing the impact of YPS on preventing

adult homelessness. The section below provides recommendations on ways for Porchlight to remain

in contact with YPS Users after leaving their YPS.

For the current analysis, Outcomes Star has been used to estimate long-term outcomes (see caveat

below and subsequent recommendations). Specifically, existing Porchlight data on improvements by

YPS users against Outcomes Star measures during the course of their stay, and importantly,

responses in a one-off survey with 22 out of 50 YPS users currently residing in Porchlight

accommodation on Outcomes Star dimensions was used to determine likely reductions in long-term

adult homeless. While judged reasonable by the SROI Working Group, this use of data is a

projection only for forecast SROI purposes, and must be verified in a subsequent, evaluative SROI.

Outcomes Star and methodology for long-term impact measurement

Porchlight currently uses Outcomes Star (Homelessness) as the primary means of measuring its

impact on YPS Users. For further information on Outcomes star see (Triangle Consulting, 2013).

Crucially, Porchlight only measures YPS Users while they stay at Porchlight and have had little

success in remaining in contact with users once they leave Porchlight, let alone measuring outcomes

they experience over the long term.

There is a (slowly) growing body of evidence that improvement on outcomes as measured by the

‘Outcomes Star – Homelessness’ may predict long-term outcomes (Triangle Consulting, 2013).


17 P a g e

However, as outlined in the main body of this report, this research is ambivalent and contradictory.

Accordingly, the use of Outcomes Star outcomes within the period that YPS Users remain at

Porchlight only would only provide rough estimates for long-term social impact. It is therefore only

appropriate for use with a Forecast SROI only.

Users – assumed no negative long-term outcomes beyond fact of expulsion.

An assumption was made that users generally do not experience negative long-term outcomes as a

result of being accommodated at Porchlight, except when users are expelled from Porchlight for

repeated and serious failure to follow Porchlight house-rules. It is assumed that the only adverse

effects this has is (1) it requires users to find housing elsewhere, and (2) the neutral effect of denying

users the possibility of gaining positive outcomes through Porchlight. These assumptions must be

explored further in subsequent, evaluative SROI analyses, particularly for YPS Users following their

departure from Porchlight after expulsion.

Outcomes for users’ families

Positive and negative outcomes for users’ families is currently determined only via users, who may

give very biased accounts. Direct engagement with families is necessary to determine whether

outcomes for families should be considered material.

More generally, greater data concerning preventing adult homelessness, outcomes of expulsion, and

outcomes for families would result in more realistic, accurate estimates of the social value of

Porchlight’s YPS.

Impact Factors

Levels of deadweight, attribution, displacement, duration and drop-off were estimated solely

through a survey of users combined with judgement of staff without recourse to further evidence.

Although final impact factors selected are conservative, this remains a weakness in the current

analysis.

Valuations for user outcomes

The estimate of social value for users’ most significant outcome, avoiding long-term adult

homelessness was derived using estimated quality-adjusted life years (QALY) between homeless and

non-homeless people, and an academic study of the average stated preference (willingness to pay)

of individuals for an increased number of QALYs. Although conservative assumptions were used,

based on recent academic and sector research, this valuation must be treated with caution as an

estimate only.


18 P a g e

1.6. Insights and Recommendations

It is of interest that in this analysis (1) excluding the primary outcome of ‘avoiding adult

homelessness,’ 7% of the total value of the Young Persons’ Service accrues to the young homeless

clients themselves, whereas (2) 52% accrues to Local Authority housing and social services

departments in the short- and medium- term cost-savings through avoiding legal obligations to

house homeless youth. Furthermore, (3) in this analysis the proportion of value accruing to the NHS

and justice system was very small (2%).

The reason connecting these disparities is that local authorities will make cost-savings for all 50 YPS

users per year, whereas the SROI Working Group conservatively forecast that, for example, only 14

homeless youth per year will avoid adult homelessness, and that figures is subject to deadweight of

25% and attribution of 50%.

As a result, a follow-up evaluative SROI should reassess valuations of outcomes for YPS users and,

importantly, more accurately track the number of YPS users that avoid adult homelessness as a

result of Porchlight’s intervention.

1.6.1. Recommendations

Porchlight already has a well-embedded system to measure progress of YPS users against the

Outcomes Star Homelessness framework. This involves regular check-ins and estimates of changes

in outcomes on the Outcomes Star 11-point scale. However, several changes should be made to

correct the weaknesses outlined above, and allow for effective progress to an evaluative SROI.

Incorporate objective indicators into outcome measurement

Specific objective indicators, particularly for the outcomes that are most sensitive for SROI analysis

and not correctly ‘objectively’ measured, should be embedded within YPS Staff reporting adjacent to

their reporting on regular Star Outcome measurements for YPS Users. These are outlined in the

main body of the report.

Increase engagement with graduating and former YPS users for longitudinal assessment

Engaging with YPS Users after they leave Porchlight is crucial to understanding the long-term impact

of Porchlight for its users, and for government agencies. A variety of means could be adopted to

boost contact and impact measurement with former YPS Users. These include:

- Regular social events and engaging ‘news’ emails for former YPS users

- Small token gestures/prizes for former YPS users that complete follow-up impact surveys.

Increased engagement with former YPS users would allow for longitudinal assessment of outcomes.

This would allow for:

- More accurate ‘before’ and ‘after’ measurements of the YPS’ impact on users, with less

scope for recall/response bias, and also provide a basis for robust statistical analysis.


19 P a g e

- For valuations, use of a life-satisfaction approach in addition to stated preference to provide

for triangulation of data for subjective valuation of users (and family) outcomes. A

significantly larger sample will also increase robustness, particularly to ameliorate issues of

‘scope sensitivity.’ This will better ensure that the exact extent/size of each outcome

experienced by users is taken into account when producing valuations.

- Use of statistical techniques to estimate the likely level of impact factors: deadweight,

attribution through use of regression analyses, and duration/drop-off by measuring

outcomes in the years after young people leave Porchlight.

Integrate data systems

If possible, Porchlight should link datasets between its YPS and adult homelessness services, to allow

for long-term tracking of Porchlight end-users that return to Porchlight as adults.


20 P a g e

2. SROI Assurance – Summary of Evaluation for Assurance Purposes

SROI Report Type: Forecast

Principle 1. Involving stakeholders

“The important thing here is to test for a comprehensive identification of stakeholders and a

convincing rationale for those that have been included and excluded from the engagement,

feedback and reporting processes. Stakeholders are those people or organisations that experience

change as a result of the activity and they will be best placed to describe the change. This principle

means that stakeholders need to be identified and then involved in consultation throughout the

analysis.”

Criteria Activity undertaken / References

1. Is the process for deciding which stakeholders are relevant for inclusion in the analysis clear and sound

- Stakeholders were initially identified in consultation Porchlight staff, augmented through discussion with users, and other stakeholders, and re-checked with Porchlight staff.

See Section ‘ - Summary of Stakeholders and

Hypothesised Outcomes’ on page 39

2. Have all stakeholders considered to experience material changes - positive & negative/intended or unintended - been consulted about what changes for them.

- All Stage 2 & Stage 3 interview scripts asked for positive, unintended & negative outcomes. See Appendices for interview scripts.

3. For an evaluative report, are total and sample numbers of stakeholders clear and is there any reason to think that an insufficient number of stakeholders have been consulted?

4. For a forecast report, where fewer stakeholders have been engaged, is there clear justification for sample size used or are there clear plans and recommendations cited in the report to address this during any future planned cycle of analysis

- Only YPS Users were engaged at lower numbers than full population.

- See Section 5.5.2 User sampling on page 43 for analysis of sample taken and conclusions regarding representativeness of the wider YPS population.

- YPS Users’ family were not engaged directly due to lack of permissions. See Section on recommendations.

5. Is there clarity around how initial engagement data has been gathered and recorded.

- See Section 5.4 Summary of Stakeholder Engagement and Section 5.5 Details on data collection & sampling on page 40


21 P a g e


below.

6. Is there evidence of open ended stakeholder inquiry about what changes have been experienced, including unintended or negative change

- All Stage 2 interviews used non-leading, open ended questions.

- See Appendices for interview scripts.

7. Does initial stakeholder engagement have a clearly explained link to outcomes claimed in the report.

- Outcomes and visual impact maps were developed through use of data gathered in SROI Stage 2.

- See Section 7.2 from page 51 for description of outcomes derived from Stage 2 data collection.

8. Is there evidence in the report citing how stakeholders have been further involved at other stages throughout the SROI analysis.

- Stakeholders engaged in Stage 3, verifying results and in review of the report.

- See Section 5.4 Summary of Stakeholder Engagement on page 40.

Additional Criteria

9. Have significant stakeholders been fully involved in determining indicators

10. Have significant stakeholders been fully involved in determining financial proxies

11. Is there evidence of significant stakeholders being consulted at all stages where appropriate and useful to the analysis

Inclusion of Elements required by SROI Network (January 2013) ‘Supplementary Guidance on

Stakeholder Involvement’

Section Name Stakeholder should be involved Activity undertaken / References

1. Identifying stakeholders

- In consultation with the initial list, stakeholders should be asked if they think others are experiencing change as a result of the activity

Stage 2 interviews included canvassing for further stakeholders. See Appendix for interview scripts

2. Developing outcomes and Outcome indicators

- In determining the outcomes they experienced (taking care that outcomes are recorded against the stakeholder that experiences the outcome).

- In agreeing the relevant outcome in a chain of events and intermediate outcomes, where necessary to reflect progress toward an outcome. (Information identified in this stage may change decisions on which stakeholders should be involved)

Theories of Change for each stakeholder were confirmed with each stakeholder group. See Section 5.5 (Details on data collection & sampling) from page 42 for details


22 P a g e

Section Name Stakeholder should be involved Activity undertaken / References

3. Collecting outcomes data

- Stakeholders will be a source of data for quantities of outcomes measured using the above indicators. (However remember that this Guidance does not cover this step. The step is included for completeness in relation to the overall process)

Quantities of outcomes were gathered using extensive input from all stakeholders. See Section 5.5 (Details on data collection & sampling) from page 42 for details

4. Deadweight and displacement

- In the assessment of the availability and usefulness of comparable services and on deadweight issues.

- (Information identified in this stage may change decisions on which stakeholders should be involved)

‘Impact Factors’ determined with substantial contribution from stakeholders through Phase 2 data collection. See Section 8 (SROI Stage 4: Establishing impact) from page 78.

5. Attribution - In assessing attribution of outcomes to other contributors. (Information identified in this stage may change decisions on which stakeholders should be involved)

6. Drop-off - In assessing drop-off of outcomes over time. in assessing approaches to reporting drop-off data in the longer term.

7. Verification - In reviewing o The theory of change o The value and range of

outcomes - In assessing the appropriateness and

usefulness of the form of reporting back to them.

- Within the determined scope and audience, stakeholders should receive appropriate reports and other SROI-related communication materials

Theory of change was reviewed in paper diagram format for YPS Users (appropriate format), and electronically for all other stakeholders Value and range of outcomes were provided in ppt presentation and report formats as appropriate for other stakeholders. Section 5.5 (Details on data collection & sampling) from page 42 for details

8. Using the results

- Within the determined scope and audience, stakeholders should be informed of changes in performance over time

N/A – Forecast SROI. However, results of ongoing data collection/monitoring for an evaluative SROI will be shared with stakeholders.


23 P a g e

Principle 2. Understand what changes

“The report should be tested for a clear explanation of the theory of change for included

stakeholders. First of all this will require a test that the scope of the analysis has a clear purpose and

timescale. After this the assessor should check that report demonstrates understanding of the chain

of events as well as reporting on consideration of changes that the stakeholders intend and do not

intend, and changes that are positive and negative. This principle requires the theory of how these

changes are created to be stated and supported by evidence. Assessors should come to a judgement

about the quality/ reasonableness of any data presented.”


1. Has the author/s made clear that the analysis is either a forecast or evaluative study.

- See Title page

2. Has a clear scope and timescale that distinguishes between the investment period and projected period for outcomes been stated for the SROI analysis.

- The investment period and the projected period for outcomes are aligned. See Section 5.1 Establishing scope on page 33.

3. Is the rationale for choices made around activities included and excluded clear and convincing.

- All activities related to the operation/running of a discrete Porchlight project and geography.

- Section 5.1 Establishing scope on page 33.

4. Is the theory of change explicit and for stakeholders considered significant to the change analysis, are the relationships between input, output and outcome clearly demonstrated in the report and adjudged to be reasonable.

- Stakeholders ratified theories of change as covering all major outcomes/changes (positive and negative for them).

- For links between inputs, outputs, outcomes and Theories of Change for different stakeholders, see Section 7 from page 48.

5. Have unintended and negative outcomes been considered and included

- Unintended and negative outcomes considered for most stakeholders. See Impact Map in appendices and Section 7.1.1 Negative and Unintended Change on page 49 .

6. In an evaluative report are there enough stakeholders included at the data gathering stage to support the quantity of change relating to each outcomes.

7. Is the theory of change corroborated in the report through reference to other supporting data where necessary.

- Theory of Change is based on the Outcomes Star methodology used by a range of charities. References and discussion of the limitations are discussed in Section 7.1.2 Recommendations and Caveats – Further Changes to be assessed on page 49.


24 P a g e


8. Are the indicators reasonable and do they provide adequate information to show that the change is measurable.

- For descriptions of indicators used, see o Impact Map in Appendices o Further descriptions of

indicators for each stakeholder

group in Section 7 SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48.

9. Where appropriate have objective as well as subjective indicators been used without double counting

- Both objective and subjective indicators have been used where possible. See:

o Impact Map in Appendices o Further descriptions of

indicators for each stakeholder

group in Section 7 (SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies) from page 48

10. Do all outcomes relate properly to the stakeholder for which they are claimed

- See description of outcomes linked to each stakeholder see:

o Impact Map in Appendices

11. Is there a clear chain of events applied and reported in determining different outcomes and quantities of outcomes for stakeholders, including what happens to those in the cohort who do not experience a given outcome

- In Stage 3 engagement, the extent to which stakeholders (especially YPS Users) experienced each outcome was measured using outcome indicators.

- A clear theory of change and summary of outcomes (including forecast non-progression along the chain) for YPS Users is provided in Section 7.2.2 on page 52.

- Assumptions about YPS users not experiencing outcomes – and recommendations for further measurement – is provided in Section 7.1.1 and 7.1.2 on page 49.

12. Are the claimed outcomes clearly explained in the report, including unintended and negative change and have outcomes alone been taken forward to valuation.

- For explanation/description of outcomes see relevant sub-sections (namely, ‘Outcomes and Theory of Change’ and ‘Financial Proxies’) for each stakeholder group in Section 7 (SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies) from page 48.

- For description of items taken forward to valuation, see Impact Map in appendices


25 P a g e


13. Does the [Impact Map] include figures for the duration of outcomes with explanations

- Please refer to Impact Map.

14. Is the Impact map clear and transparent and is the reporting of change completely consistent with Impact map contents.

- To identify whether the report is congruent with the Impact map (in Appendices) in terms of inputs, outcomes, indicators, proxies and ‘impact factors’ (duration, deadweight, attribution, etc), see relevant sections (Inputs, Outcomes, Indicators, etc) for each stakeholder in Section 7 SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48.

Additional Criteria

15. Has the analysis dealt with distance travelled? Is there an analysis of the chain of cause and effect between short, medium and longer term outcomes

- See YPS User Theory of Change – with declining numbers toward long-term outcomes, on page 57.

Principle 3. Value things that matter

“This principle is concerned with how outcomes are valued in the SROI process. The important thing

here is that financial proxies are used in order that the value of the outcomes can be recognised.

Assessors will look to ensure that financial proxies are clear, appropriate to the outcome and also

that indicators are clear and appropriate. Any data for indicators and financial proxies will require

source of information. Assessors should come to a judgement about the quality/ reasonableness of

any data presented.”


1. Have all relevant inputs by significant stakeholders been included and valued and if not valued have reasons been given

- See Impact Map vs. relevant ‘inputs’ sub-sections in Section 7 SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48.

2. For an evaluative study, have all the material outcomes been given a value

3. For a forecast study, where some outcomes have not been valued, does the report cite specific requirements and recommendations for valuation over any ensuing cycle.

- See Section 7.1.2 Recommendations and Caveats – Further Changes to be assessed on page 49.

4. Are the financial proxies evidenced in the report adjudged to be reasonable and appropriate to the outcome.

- See Impact Map. For further details, see relevant sub-sections of Section 7 SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48.

5. Are Financial Proxies appropriate to the - See relevant sub-sections of Section 7


26 P a g e

stakeholder for whom the value is claimed SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48

6. In a forecast study, where an identified outcome is not recorded with at least one indicator and financial proxy, is there a reasonable explanation for the exclusion.

- See Sections 7.1.2 Recommendations and Caveats – Further Changes to be assessed

7. In a forecast study, in cases where outcomes have not been measured and valued, does the report include recommendations for capturing the value in any ensuing cycle of analysis.

- See Sections 7.1.2 Recommendations and Caveats – Further Changes to be assessed.

Additional Criteria

8. Has a range of financial proxies been varied in sensitivity analysis

- See Section 9.1.3 on page 83

9. Is there a statement on issues arising from the use of financial proxies in this analysis

- See Section 7.2.5 Financial Proxies on page 61.

Principle 4. Only include what is material

“This principle requires an assessment of whether a person would make a different decision about

the activity if a particular piece of information were excluded. This covers decisions about which

stakeholders experience significant change, as well as the information about the outcomes. Deciding

what is material requires reference to a number of aspects, for example the organisations own

policies, its peers, societal norms, and short-term financial impacts.”


1. Is there explanation in the report to justify what materiality test has been brought to bear on decisions to include and exclude outcomes (or stakeholders who do not experience material outcomes).

- See Section 6 Consideration of Materiality.

2. Are reasons given for situation in which no material changes were expected to occur to excluded stakeholders


3. Is there evidence that materiality tests have been applied to stakeholders and outcomes during the whole process as part of decisions around significance for deadweight, value, quantity of change and around relevance relating to initial inclusion of outcomes rather than just at the beginning

- Materiality was considered at beginning, and post consideration of deadweight and other ‘impact factors,’ value, quantity of change, etc.


4. Is there anything that would lead the assessor to conclude that there have been exclusions that would lead to different stakeholder decisions and conclusions about the activity

For list of initially included/excluded stakeholders, see Section

- Summary of Stakeholders and Hypothesised Outcomes on page 39.

- For list of stakeholders/outcomes subsequently excluded, see Section 6


27 P a g e

Consideration of Materiality.

5. In a forecast report where materiality tests cannot yet be applied to stakeholders or outcomes, has this been explored in sensitivity analysis and recommendations for any ensuing cycle of the analysis

- Sensitivity: See Section 9.1.3 on page 83

- Recommendations: See Section 7.1.2 Recommendations and Caveats – Further Changes to be assessed on page 49.

Additional Criteria

10. Have the scope of the study, the analysis of change, the included outcomes and the claimed impact been based on a comprehensive analysis of materiality issues that have been explained in the report

Principle 5. Do not over claim

“This principle requires reference to trends and benchmarks to help assess the change caused by the

activity, as opposed to other factors, and to take account of what would have happened anyway. It

also requires consideration of the contribution of other people or organisations to the reported

outcomes in order to match the contributions to the outcomes.”


1. Have all inputs that would lead to the included outcomes been given a value for calculation of the SROI ratio

- See Impact map.

2. Has double counting been avoided, for example when choosing more than one indicator per outcome

- See additional columns in impact map for multiple indicators, with final quantities based on averages of measurements on different indicators rather than double counting.

3. Has double counting been avoided through clarity of reporting on the chain of events that might lead to different outcomes for the stakeholder group.

- Double counting not relevant for YPS User outcomes (given simultaneous nature of outcomes, using Outcomes Star methodology. See Section 7.2 from page 51.

- Number of users providing outcomes for partner organisations do not involve double-counting, as outcome valuations are independent of each other. See partner organisation theory of in Section 7.4 from page 66.

4. Are the numbers of outcomes claimed per stakeholder group out of the total membership of that group credible and reasonable

- See Impact Map.

5. For an evaluative analysis are the figures used for deadweight and attribution based on trends and benchmarks or a systematic and clearly explained


28 P a g e

estimation process using information from stakeholders or other external information.

6. For a forecast analysis are the figures estimated for deadweight and attribution subject to sensitivity analysis and plans for better data capture over the ensuing period

- Sensitivity analysis involved variations to deadweight and attribution.

7. Does the analysis discuss decisions on displacement and include a figure if appropriate with reasonable and convincing explanation

- For discussion of decisions made on displacement, see Section 8 SROI Stage 4: Establishing impact on page 78.

8. Does the analysis consider how outcomes drop-off over time

- Drop-off is explicitly incorporated in impact factors. Section 8 SROI Stage 4: Establishing impact on page 78.

9. For an evaluative analysis, are durations used based on research evidence.

10. For a forecast study where durations used have not been based on research evidence – is there a reasonable explanation and is it clear that any assumptions made have been subjected to sensitivity analysis and are to be monitored in the future

- Sensitivity analysis involved variations to duration.

- See ‘Sensitivity Analysis’ worksheet of Impact Map excel workbook

Additional Criteria

11. Has a full counterfactual been included for deadweight

12. Has primary research been conducted in assessing attribution

13. Has drop off varied over time, or to different stakeholder groups

Principle 6. Be transparent

“Assessors will look for evidence that reports demonstrate the basis on which the analysis may be

considered accurate and honest, and show that it will be reported to and discussed with

stakeholders. This principle requires that each decision relating to stakeholders, outcomes,

indicators and benchmarks; the sources and methods of information collection; the difference

scenarios considered and the communication of the results to stakeholders should be explained and

documented.”


1. Is there an audit trail both of what is and what is not included relating to stakeholders, outcomes and financial proxies

- For audit trail of stakeholders and outcomes judged immaterial see:

o Impact Map o Section 6.1 Audit Trail on page

47.

2. Is the sensitivity analysis adjudged to include appropriate elements relevant to

- See Section 9.1.3 on page 83.


29 P a g e

the study with clear information on which aspects and which assumptions have been assessed for sensitivity

3. Are all data sources referenced - References to sources are provided in-text throughout the report. For full sources (bibliography) see Appendix 11.10.

4. Is there enough information on the data set and are all calculations set out in a way that makes it possible for the calculation to be replicated and to arrive at the same result of social return

- Data set based on interview scripts (see Appendices).

- Excel spreadsheet of calculations of YPS User outcomes and financial proxies is provided in Appendix 11.4.

5. Where appropriate is there information on the source of financial proxies, detailed enough that would enable the reader to refer.

- All financial proxies are referenced. See:

o Impact Map o Relevant sub-sections in

Section 7 SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial Proxies from page 48.

- For references (bibliography) see Appendix 11.10.

Additional Criteria

6. Is there enough information on data and sources included that would allow a full audit of the report

Principle 7. Verify the result

“Although an SROI analysis provides the opportunity for a more complete understanding of the

value being created by an activity, it inevitably involves subjectivity. Appropriate independent

assurance is required to help stakeholders assess whether or not the decisions made by those

responsible for the analysis were reasonable.”


1. Has the report been reviewed by at least one stakeholder

- Results reviewed by Porchlight staff.

2. If there has not been a formal process for review by stakeholders – is there an explanation that is reasonable and does the report contain recommendations that would address such a process in the near future.

- Review of primary results (e.g. theory of change, valuations, value) conducted with stakeholders as outlined in Section 5.4 Summary of Stakeholder Engagement on page 40.

- For formal review process, see Section 10 SROI Stage 6: Reporting on page 85.

Additional Criteria

3. Has the report been subject to independent peer review

- Report in its entirety reviewed by key stakeholder (commissioner). See Section 5.5.5 on page 44 for further information.

4. Has there been a formal documented process for review by stakeholders


30 P a g e

3. Introduction to Porchlight

Originally founded in 1974, Porchlight is a charity headquartered in Canterbury and working

throughout Kent to help the most isolated and vulnerable in their communities to access housing

and related support services. Their vision is of a society without homelessness or poverty, and they

have a mission of changing people’s lives for the better. To achieve this, Porchlight helps vulnerable

and isolated people in their communities to access housing and related support services, works with

stakeholders to prevent community breakdown and reduce poverty, and seeks to change negative

attitudes and beliefs around exclusion and homelessness.

Porchlight have supported accommodation projects based in Ashford, Canterbury, Dover, Ramsgate

and Tonbridge, and outreach support for street homeless people across the county. They provide a

range of services supporting people with complex needs such as mental ill-health and have specific

services for young vulnerable and homeless people. Porchlight run a range of services, including:

- Kent Outreach Service – supporting rough sleepers and people in temporary

accommodation, with early morning outreach sessions, local drop-in sessions., and a 24-

hour outreach telephone helpline.

- Supported accommodation service – providing accommodation in hostels with medium to

high support, with specialist services for young people and those with enduring mental

health issues.

- Floating Support service – providing support and advocacy for people already in housing to

address issues affecting housing security, particularly ex-offenders and people with enduring

mental health issues.

- Youth Homelessness Education Programme – running workshops and talks for young people

about homelessness.

- Young Persons Service – providing supported accommodation, which includes support to

achieve education, employment, future independent living and other outcomes, for youth

aged 16-21.


31 P a g e

Young Persons Service in Canterbury

Porchlight’s Young Persons Service (YPS) in Canterbury provides supported accommodation

specifically for young people. This includes housing, as will as a range of recreational activities, and

specific support to: learn life skills, promote healthy behaviours, improve literacy/numeracy, attend

college, access training, and seek employment.

The Canterbury YPS is a service for youth (aged 16 – 21) run from three accommodation sites:

- One ‘Direct Access’ hostel for 3-6 months emergency residency staffed 24/7. Maximum

capacity: 7 people with high-level support needs.

- One ‘Medium Stay’ hostel for 12 months residence staffed 9-5 Monday-Friday and

occasional weekends. Maximum capacity: 8 people with medium-level support needs.

- ‘Move-on’ Accommodation, not-staffed.

o One house for 6-12 months residency (maximum capacity: 3 people with low-level

support needs)

o One apartment for 6-12 months residency (maximum capacity: 3 people with low-

level support needs )

Porchlight also run a YPS in Tonbridge with a short-term hostel for youth with medium-level support

needs. For reference, the YPS in Tonbridge support approximately 10-20young people per year.

It is estimated that for the forecast Period, Porchlight’s YPS Canterbury will support 50 young

people. Porchlight’s YPS in Canterbury is staffed by 9 full-time personnel and 1 manager, with

oversight by Porchlight’s Director of Operations.


32 P a g e

4. Introduction to Social Return On Investment

Social Return on Investment (SROI) is an approach to understanding and managing the impacts of a

project, organisation or policy. It is based on stakeholders and puts financial value on the important

impacts identified by stakeholders that do not have market values.

SROI measures change in ways that are relevant to the people or organisations that experience or

contribute to it. It tells the story of how change is being created by measuring social, environmental

and economic outcomes and uses monetary values to represent them.

SROI is about value, rather than money. Money is simply a common unit and as such is a useful and

widely accepted way of conveying value. In the same way that a business plan contains much more

information than the financial projections, SROI is much more than just a number. It is a story about

change, on which to base decisions, that includes case studies and qualitative, quantitative and

financial information. SROI is based on seven principles as follows.

The Principles of SROI

1. Involve stakeholders. Understand the way in which the organisation creates change

through a dialogue with stakeholders.

2. Understand what changes. Acknowledge and articulate all the values, objectives and

stakeholders of the organisation before agreeing which aspects of the organisation are to be

included in the scope; and determine what must be included in the account in order that

stakeholders can make reasonable decisions.

3. Value the things that matter. Articulate clearly how activities create change and evaluate

this through the evidence gathered

4. Only include what is material. Make comparisons of performance and impact using

appropriate benchmarks, targets and external standards.

5. Do not over-claim. Demonstrate the basis on which the findings may be considered

accurate and honest; and showing that they will be reported to and discussed with

stakeholders.

6. Be transparent. Ensure appropriate independent verification of the account.

7. Verify the result. Use financial proxies for indicators in order to include the values of those

excluded from markets in same terms as used in markets.

A full SROI involves six distinct stages. These can be seen in Appendix 11.2 on page 89. An

explanation of terms can be seen in Appendix 11.11 on page 113.


33 P a g e

5. SROI Stage 1: Establishing Scope and identifying key stakeholders

The first stage of an SROI involves three steps:

1. Establishing scope

2. Identifying stakeholders

3. Deciding how to involve the stakeholders

5.1. Establishing scope

The scope of an SROI analysis is an explicit statement about the boundary of what is being

considered.

This stage should help to ensure that what is being proposed is feasible. The issues which need to

be considered include:

Purpose

To complete a forecast SROI which will identify outcomes for all material stakeholders

relating to Porchlight’s Young Persons’ Service in Canterbury.

This report is intended to be used for three purposes, in order of importance:

o To illustrate to commissioners and other stakeholders the value of the service in

terms of both added value and value for money.

o To identify social impact value drivers to help Porchlight replicate the successful and

more effective aspects of the delivery model.

o To assist Porchlight to undertake future SROI analysis on other Porchlight services.

Audience

Internal Stakeholders (e.g. Trustees and Senior Management)

Potential Funders and Influencers

Resources

The SROI Working Group will include:

Rohan Martyres (CAN Impact)

Gill Bryant (Head of Operations, Porchlight)

Lisa Oeder (Head of Development and Communications, Porchlight);

Emily Upfield (Acting Manager, YPS, Porchlight)


34 P a g e

Activities to be included

The activities to be included are the Young Persons Service in Canterbury (2 hostels and 1 shared

house). For the year 2012/13 these properties accommodated 49 young people, and are forecast to

accommodate 50 young people in 2013/14.

Period of time

This will be a forecast SROI for the period April 2013 through March 2014.

Summary Scope

A forecast SROI which will identify outcomes for all material stakeholders relating to the

Porchlight Young Persons’ Service in Canterbury between 1 April 2013 and 31 March 2014,

to be used to demonstrate the value of the service to commissioners and other external

stakeholders.

5.2. Identifying stakeholders

YPS Users – Background and Context

The most salient stakeholder group of the YPS is its end users, which in 2012/13 comprised 49 young

people aged 17 – 22 (27 female, 22 male). Previous research identifies age and gender as the

primary demographic variables (as opposed to other variables relating to previous family and

medical history) affecting outcomes for young people in accommodation (St Mungo's, 2012). Basic

demographic information on these users in 2012/13 is provided in Figure 4 through Figure 6 below.

Figure 4. Basic Demographic Profile of the 49 YPS Users (Canterbury) in 2012/13.

3

9

11

3

1

3

7

8

3

1

0

2

4

6

8

10

12

17 18 19 20 21 22

Nu

mb

er

of

YP

S U

sers

Age

Female

Male


35 P a g e

Figure 5. Length of stay profile of the 49 YPS Users (Canterbury) in 2012/13.

Figure 6. Inward referral profile of the 49 YPS Users (Canterbury) in 2012/13.

YPS Users ‘Journey’

In terms of housing outcomes, YPS users access all three levels of YPS hostels, often in chronological

order, with exits as outlined in Figure 7 below. 43 of the 49 YPS users left the YPS within 2012/13.

17

6

4

18

2 2

0

2

4

6

8

10

12

14

16

18

20

1. 0 - 6 months 2. 6 - 12 months 3. > 1 year

Nu

mb

er

of

YP

S U

sers

Length of Stay at YPS

Female

Male

1 1

3

9 9

3

1 1

8

5

7

1

0 1 2 3 4 5 6 7 8 9

10

Nu

mb

er

of

YP

S U

sers

Referral Pathway to YPS

Female

Male


36 P a g e

Figure 7. Housing Outcomes for 49 YPS users

YPS User – Existing Outcomes Data.

Porchlight already reports on outcomes for its wider accommodation and floating support services

programmes using two frameworks.

First, Porchlight reports numbers of clients achieving outcomes within the Department of

Community and Local Government (CLG) Supporting People Outcomes framework.2 This framework

comprises 15 outcome indicators within 5 areas:3

- Achieving Economic Well-Being:

o Number of clients who receive correct welfare benefits

o Number of clients who reduce overall debt

o Number of clients who in paid work

- Enjoy and Be Active:

2 See Porchlight (2011). Annual Report and Financial Statements for the year ended 31 March 2011, p.5.

Accessed at http://www.charity-commission.gov.uk/ on 19 January 2012. 3 Department of Community and Local Government (2011). Supporting People Quarterly Client Records and

Outcomes – April 2010 - March 2011 final. Accessed at http://www.communities.gov.uk/publications/corporate/statistics/supportingpeoplefinal2011 on 19 January 2012.

http://www.charity-commission.gov.uk/

http://www.communities.gov.uk/publications/corporate/statistics/supportingpeoplefinal2011%20on%2019%20January%202012

http://www.communities.gov.uk/publications/corporate/statistics/supportingpeoplefinal2011%20on%2019%20January%202012


37 P a g e

o Number of clients who participated in education or training/ Achieve a qualification.

o Number of clients who participated in leisure, faith, cultural or informal learning

o Number of clients who participated in work-like activities

o Number of clients who made contact with external groups, family and friends

- Be Healthy:

o Number of clients who are managing physical health better

o Number of clients who are managing mental health better

o Number of clients who are managing drug or alcohol misuse better

- Stay safe:

o Number of clients maintained accommodation (where there was risk of eviction)

o Number of clients who complied with statutory orders and related processes

o Number of clients who avoided self harm

o Number of clients who avoided causing harm to others

- Make a positive contribution:

o Number of clients who had ore involvement/ control at service or wider community

level

At a user-level, Porchlight also uses Triangle Consulting’s Outcomes Star (Homelessness) to measure

its work with impact for end-users. The Outcomes Star measures progress of individuals on a 10-

step ‘ladder of change’ in ten thematic areas (MacKeith, Burns, & Graham, The Outcomes Star:

Supporting change in homelessness and related services (2nd Edition), 2008):

- Motivation and taking responsibility

- Self-care and living skills

- Managing money and personal administration

- Social networks and relationships

- Drug and alcohol misuse

- Physical health

- Emotional and mental health

- Meaningful use of time

- Managing tenancy and accommodation

- Offending

The Outcomes Star, was originally developed by Triangle Consulting in 2003 for London-based

homelessness charity, St Mungo’s. It has since been adopted by a range of homelessness charities,

and also sparked the creation of Outcome Stars for other sectors. For further description of the

development of the Outcomes Star, please see (MacKeith, 2011).


38 P a g e

YPS Users – Hypothesised Outcomes

Porchlight’s CLG and Star Outcome data suggests that YPS users may self-report a range of

outcomes:

- Accommodation, which reduces reduced vulnerability.

- Better able to manage finances, leading to greater economic well-being.

- Re-establishing contact with family and friends.

- Receiving one-to-one support and referrals, leading to better management of substance

misuse, mental health and physical health issues.

- Education and support, leading to higher self-confidence, and reduced risk of harm from

others.

- Making friends (including via Porchlight’s buddy system).

- Preparing for tenancy (Pre-tenancy training – skills to live independently and be a

responsible tenant), leading to less issues with maintaining private tenancy

Funders

A breakdown of total YPS forecast cost for the 2013/14 year is provided in Table 2 below.

Funding Source Likely £ Amount (2013/14) % of Total

DCLG/Kent County Council (Supporting People

Programme)

£499,856.00 69%

Other Funders (DWP Housing Benefit) £207,191.00 29%

YPS Users £11,575.20 2%

Local Authority Social Services £3,915.58 1%

NHS £3,266.23 0%

Education Providers £421.27 0%

YPS Staff £247.60 0%

Grand Total £726,472.88 100%

Table 2. Sources of YPS Funding


39 P a g e

Summary of Stakeholders and Hypothesised Outcomes

The below table provides a list of stakeholders who may experience material outcomes. This list was

developed through discussion within the SROI Working Group, with reference to prior with

additional suggestions taken from other stakeholders: some YPS end users in SROI Stage 2

stakeholder engagement suggested family members of YPS users be added to the list

The above suggest a range of outcomes can be hypothesised for different stakeholders, as outlined

below. Please note these outcomes are hypothetical at this stage and will be explored with

stakeholders during the course of this study.

Stakeholder Hypothesised Outcomes

YPS end-users

Higher outcomes on Homelessness Star and CLG measures, leading to:

o Greater confidence and well-being o More stable housing and financial situation o More positive family and social situation o Less substance and alcohol misuse o More productive use of time (e.g. study, training, work) o Reduced adult homelessness

Young person’s family members

Greater contact and better relationships with YPS user

YPS staff Salary Sense of satisfaction (-ve) stress and burden of engaging with young people.

Local Authority Social services

Discharged statutory duty of care obligations (via Porchlight)

Local Authority Housing Dep’t

Discharged statutory duty of care obligations (via Porchlight)

Department of Justice (Youth Offending Services)

Information sharing Reduced reliance on services

Referral organisations (inward and outward)

Greater cooperation with Porchlight to achieve organisational objectives

Funders Material outcomes for youth

Health Services (GP, hospitals)

Long-term reduced reliance on services

DWP Marginal increase in costs/staff time due to YPS users maximising benefits derived from DWP.

Schools/College Increased YPS user participation and scoring within education system.

Table 3: Range of hypothesised outcomes of Porchlight for different stakeholders


40 P a g e

5.3. Deciding how to involve stakeholders

The final part of this first stage of the SROI is to consider how to involve the stakeholders in the

analysis and the number of stakeholders involved from each group. Another important

consideration is how often and when to engage with the stakeholders, who are often needed to

check the analysis as it develops.

5.4. Summary of Stakeholder Engagement

A summary of stakeholder engagement is shown in Table 4 below.

Stakeholder Method of Engagement

Stage 2: Identifying outcomes

Stages 3-4: Valuing outcomes & establishing impact

Checking Results

YPS end-users (n=49)

1 Focus group (n=3)

1-1 semi-structured interviews (n=9)

Paper survey (n=22)

Theory of Change: paper survey (n=22)

Overall results: interview (n=5)

YPS Users’ families Estimated via users only, as no family member gave permission to interview

Estimated via users only (paper survey, n=22)

None

Porchlight Hostel Staff (n= 9)

1 Focus group (n=9)

Follow-up interviews (n=2)

Online survey (n=9)

Theory of Change: presentation (n=2)

Overall results: presentation (n=2)

Funders (KCC Supporting People Commissioners, BBC Children in Need) (n=2)

semi-structured interview (n=1) Theory of Change: & overall results: presentation (n=1)

L.A. Social Services (Canterbury and Shepway) (n=4)

1-1 semi-structured interview (n=2) Follow-up email and conversation (n=2)

Local Authority Housing Dep’t (Canterbury) (n=1)


Referral organisations (n=2)


NHS, MoJ, HMRC None – desk research only

Schools/College (n=1) 1-1 semi-structured interview (n=1) Follow-up email and conversation (n=1)

Table 4: Proposed Key Stakeholder Engagement


41 P a g e

Excluded Stakeholders

Stakeholder Possible Outcomes Reason for exclusion

None

Table 5. Excluded Stakeholders

No stakeholders were excluded prior to Stage 2 stakeholder engagement. Exclusions after this are

outlined in an audit trail in Section 6 Consideration of Materiality on page 46 below.


42 P a g e

5.5. Details on data collection & sampling

Further details on stakeholder engagement are provided below.

5.5.1. Users

Phase 1 (Mapping Inputs, Activities & Outcomes – SROI Stage 2)

- One face-to-face focus group with 3 YPS Users, lasting one hour (n=3)

- 1-1 semi-structured interviews, in-person and by telephone, lasting on average 40 minutes

each (n=9).

- The interview script is reproduced in Appendix 11.3.1.

- Interviews were requested from all available users: those currently staying within YPS or in

contact with Porchlight post-departure.

Phase 2 (Evidencing and valuing outcomes – SROI Stage 3-4)

- 22 current/former users completed a paper based survey.

- Given the estimated 50 YPS Users in total p.a., 22 survey responses provides a margin of

error of approximately 16%.4

- The survey collected numerical data on outcomes (numbers achieving outcomes, £ valuation

of outcomes) as well as to confirm the theory of change.

- Questions and data were transcribed into electronic format for coding (in excel)

- The survey script is reproduced in Appendix 11.4.1.

Phase 3 (Checking results – SROI Stage 6)

- Theories of Change were verified by all Phase 2 respondents, and refined in Phase 3.

- Final results, including theory of change and valuations (WTP) of outcomes were ‘sense

checked’ with 5 users in person.

- Changes made are outlined in Section 10.1 on page 85 below.


4 Margin of error at 95% confidence intervals = /

x

, where N = population size (50), and n = sample size

(22). Source: (ComRes, 2013; Wikipedia, 2013). By convention, a margin of error of 5% is generally considered an acceptable standard for robust academic, social science research. However, this margin of error is arguably acceptable for a Forecast SROI for 50 YPS users.


43 P a g e

5.5.2. User sampling

Data was only collected from approximately one third of all YPS users. The figures below compare

sample against the 2012/13 population on basic demographic information.

Figure 8. Comparison of YPS Population and survey sample broken down by gender.

Figure 9. Comparison of YPS Population and survey sample broken down by duration of stay.

Although the proportion of users accessing short-term accommodation is under-represented, this

group do not experience outcomes to a great extent. As a result, the above graphs suggest that

40%

50%

55%

45%

0%

10%

20%

30%

40%

50%

60%

Female Male

Pro

po

rtio

n o

f C

ate

gory

(Sa

mp

le o

r P

op

ula

tio

n)

Gender

Sample

Population

40%

25%

35%

71%

16% 12%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1. 0 - 6 months

2. 6 - 12 months

3. > 1 year

Proportion of Category

(Sample or Population)

Duration of YPS Stay

Sample

Population


44 P a g e

sampling is sufficiently representative to provide a suitable level of confidence for the purposes of

an initial, forecast SROI.

5.5.3. Users’ families

Some users’ claimed their relationships with their families had changed (and in fact improved) as a

resulting of staying at Porchlight. However, no user family members accepted requires for

interviews. Therefore, no data was collected directly from users’ families. Rather data collection

was undertaken indirectly via YPS users at of Phase 1 – 3 (outlined above). Recommendations to

redress this are provided in section 7.1.2 on page 49 below.

5.5.4. Staff

Phase 1 (Mapping Inputs, Activities & Outcomes – SROI Stage 2)

- A face-to-face focus group lasting 1.5 hours was conducted by the SROI Practitioner with all

staff (n=9)

- The interview script is represented in Appendix 11.3.2.

- Follow-up confirmation of theory of change with 2 staff members.

Phase 2 (Evidencing and valuing outcomes – SROI Stage 3-4)

- Online interview, with staff inputting responses directly into the online ‘survey monkey’

questionnaire (n=9).



- Theories of Change were verified by 2 staff members.

- Final results were ‘sense checked’ with 2 staff members.

5.5.5. Funders, referral, and partner organisations

Phase 1 & 2 (SROI Stages 2-4) were covered within a single interview.

- Interviews were administered either in person or by telephone, by the SROI Practitioner

(n=5)

o Charitable partner

o Referral organisation

o Public agency – social services (children’s team)

o Public agency x 2 – child & adult mental health services


45 P a g e

o Funder – Commissioner of youth accommodation and other services in Kent via the

Kent County Council Supporting People Programme.



- Referral and partner organisations – results were ‘sense checked’ with individuals by

telephone / email.

- Funder – theory of change, final results and a full report of all SROI stages, were ‘sense

checked’ and signed-off on by 1 funder/commissioner.

- No changes were made.


46 P a g e

6. Consideration of Materiality

One of the principles of SROI is to only include what is material. The principle requires a:

“Determin[ation of] what information and evidence must be included in the accounts to give a true

and fair picture, such that stakeholders can draw reasonable conclusions about impact.” SROI

Network (2012, p. 97)

This principle requires an assessment of whether a person would make a different decision about

the activity if a particular piece of information were excluded. This covers decisions about which

stakeholders experience significant change, as well as the information about the outcomes. Deciding

what is material requires reference to the organisation’s own policies, its peers, societal norms, and

short-term financial impacts.

For SROI analyses which will be public documents and meet the standard required for assurance,

materiality means that the material outcomes for all stakeholders should be included. The principle

of verification and third party assurance of public documents is important in order to give those

using the account comfort that material issues have been included.

The consideration of materiality within the scope of this SROI involved several successive stages, and

was conducted in line with the AccountAbility AA1000AS standard (AccountAbility, 2008)

- The first filter is Relevance: stakeholders to not believe outcomes are important to them,

regardless of the ‘amount’ of outcome:

o Exclude outcomes that are not relevant

o Exclude stakeholders with no relevant outcomes

- The second filter is Significance: the real or potential scale of the outcome (both positive

and negative) has passed a threshold that means it influences decisions and actions.

o Exclude outcomes with little quantity of change

o Exclude outcomes whose change has little value

o Exclude outcomes whose impact (post duration, deadweight, etc) has little value

Please note that for Porchlight to comply with the AA1000 principle of materiality in its entirety, the

consideration of material outcomes and sustainability more generally will need to be conducted not

only during users’ stay at Porchlight accommodation, but ideally in the years subsequent. See

AccountAbility (2008, p. 15) for further details.


47 P a g e

6.1. Audit Trail

The following outcomes and/or stakeholders were excluded from the final SROI summation as the SROI analysis progressed. All excluded outcomes are

included in the impact map for transparency, with valuations set to £0.

Stakeholder Outcome Stage Excluded Reason for exclusion

Staff

All outcomes except for salary Following Stage 3 Non-material outcomes after consideration of amount of outcome.

See Appendix 11.6 (Appendix 6. Audit Trail – Staff) for further details.

Stakeholder excluded (incl. Salary) Following Stage 4 Salary considered non-material, after considering deadweight.

See Appendix 11.6 (Appendix 6. Audit Trail – Staff) for further details.

Partner organisations

Received more appropriate/eligible referrals (compared to average referrer)

Following Stage 2 Non-material outcome after considering significance (size of outcome)

Increased knowledge sharing b/n partner organisations

Following Stage 2 Non-material outcome after considering significance (size of outcome)

Work more effectively to deliver contracts Following Stage 2 Non-material outcome after considering significance (size of outcome)

Public and statutory services

Education Providers (increased revenue for each student retained)

Following Stage 4 Amount of revenue considered non-material for stakeholder, after

considering significance (size of outcome) due to impact factors

HMRC: Increase tax revenue Following Stage 4 Amount of revenue considered non-material for stakeholder, after

considering significance (size of outcome) due to impact factors

NHS: All outcomes except long-term resource reallocation due to substantially improved physical & mental health outcomes of adults avoiding adult homelessness

Following Stage 4 Outcomes considered non-material for stakeholder, after considering

significance (size of outcome) due to impact factors

Justice System: Resource reallocation due to reduced youth/homeless contact with police and justice system)

Following Stage 4 Outcomes considered non-material for stakeholder, after considering

significance (size of outcome) due to impact factors

Table 6. List of stakeholders and outcomes excluded from SROI analysis.


48 P a g e

7. SROI Stages 2 – 3: Mapping Inputs, Outputs, Outcomes, Indicators & Financial

Proxies

This section combines two SROI Stages.

The second stage of an SROI involves three steps:

1. Identifying all stakeholders’ investments or ‘inputs’

2. Identifying activity outputs and outcomes for all stakeholders

3. Developing theories of change for all stakeholders

The third stage of an SROI involves several steps:

1. Developing indicators to measure the existence/extent of identified outcomes

2. Identifying the quantity and duration of identified outcomes

3. Developing financial proxies to value identified outcomes

4. Ascribing monetary values (£) to identified outcomes

The activities investigated for this SROI are forecast interactions between Porchlight’s Young Persons

Service in Canterbury and its users, staff, funders and others, based on interactions recorded during

the period April 2012 through March 2013.

The data collection processes outlined above allowed the change identified for each stakeholder to

be explored, measured and valued and recorded on an impact map. Accordingly, the remainder of

this report is best understood when read together with a copy of the full impact map (see Appendix

11.8) which should be printed to A3 size at least.

For ease of reading, the remainder of this section provides a discussion of general issues, followed

by description of activity in SROI Stage 2 & 3 (and the major inputs, outputs, outcomes, indicators

and financial proxies) for each stakeholder group in turn.


49 P a g e

7.1. General Comments – Stage 3: Outcomes and Theories of Change

During the first phase of data collection (SROI Stage 2) for each stakeholder group, as outlined in

Section 5.5, an understanding of outcomes and theories of change were developed for (and ratified

by) each stakeholder group. These are discussed below for each stakeholder group in turn.

For the purposes of this Forecast SROI, it is assumed that stakeholders not experiencing outcomes

do not experience adverse affects, unless otherwise stated. However, this assumption must be

explored further in subsequent, evaluative SROI analyses, particularly for YPS Users following their

departure from Porchlight.

7.1.1. Negative and Unintended Change

Negative and unintended changes were explored throughout the process, especially at SROI Stages 2

and 3. These are outlined in the below sub-sections under each stakeholder and in the Impact Map

(see Appendix 11.8).

7.1.2. Recommendations and Caveats – Further Changes to be assessed

Users – preventing adult homelessness.

Engaging users once they have left Porchlight is difficult. In the absence of a sufficient sample of

interviews with ‘ex’ YPS users, direct measurement of long-term outcomes remains hypothetical.

This is particularly the case in assessing the impact of YPS on preventing adult homelessness, and the

extent, duration and drop-off in the prevention of adult homelessness .

It is therefore strongly recommended that Porchlight attempt to find ways to remain in contact with

YPS Users after leaving Porchlight. In the absence of this, Outcomes Star has been used to estimate

long-term outcomes (see recommendation and caveats below).

Outcomes Star and methodology for long-term impact measurement


‘Outcomes Star – Homelessness’ is a predictor of long-term outcomes. For example, a St Mungo’s

briefing outlined that the use of the Outcomes Star system anecdotally corresponds with other

systems to measure qualitative outcomes (Triangle Consulting, 2013). However, further St Mungo’s

research notes that typically “Positive outcomes peak at 6 -12 months, longer stays can be

associated with a decline in progress and an increase in mental health problems. Clients with

multiple needs were most likely to slip backwards substantially.” (St Mungo's, 2012, p. 4).

Furthermore, to date, the only formal, publicly available statistical data comparing Outcomes Star

data with ‘hard’ data connected with Department of Community and Local Government Supporting

People Funding (for homelessness), suggests that “there is currently no evidence that there is a


50 P a g e

statistical linkage between the recorded by the Outcomes Star (the ‘soft’ outcomes) and that

recoded by the Link database (the ‘hard’ outcomes)” (Boswell & Skillicorn, 2009, p. 20).

For these reasons, the use of Outcomes Star outcomes within the period that YPS Users remain at


appropriate for use with a Forecast SROI only.

For an evaluative SROI, Outcomes Star data will need to be collected for YPS users long after they

depart Porchlight, or an alternate impact measurement tool with academic, peer-reviewed studies

demonstrated validity and reliability will need to be selected. This will result in more realistic,

accurate estimates of the social value of Porchlight’s YPS.

Users – assumed no negative long-term outcomes beyond fact of expulsion.

An assumption was made that users generally do not experience negative long-term outcomes as a

result of being accommodated at Porchlight, except when users are expelled for repeated and

serious failure to follow Porchlight house-rules. It is assumed that this has no adverse effects

(beyond the neutral effect of denying users the possibility of gaining positive outcomes). This

assumption must be explored further in subsequent, evaluative SROI analyses, particularly for YPS

Users following their departure from Porchlight.

Outcomes for users’ families

Without direct access, positive and negative outcomes for users’ families can only be verified via

users, who may give very biased accounts. Direct engagement with families is necessary to

determine whether outcomes for them should be considered material.

Synergies between partner organisations

Several partner organisations believed that joint work with Porchlight resulted in synergies and a

‘multiplier effect’ in outcomes for users.

For example, according to Porchlight and KCA (drug & alcohol misuse service provider) staff:

- Porchlight gives users “the desire and means to minimise substance misuse, through safe

accommodation, developing purpose and goals in life, and improved health, resulting in

substance misuse.”

- Meanwhile, KCA’s work with users to reduce drug and alcohol dependency, which reduces

the likelihood of breaking Porchlight house rules or conflict with other Porchlight residents.

The dynamics of these synergies could be analysed and measured in further SROI analyses.


51 P a g e

7.2. YPS Users

7.2.1. Inputs and Outputs

YPS Users typically provide two major inputs:

1. Time to complete application process:

o Prospective YPS Users complete an application form, usually with support from a key

worker at a referral organisation, and attend an interview with YPS.

This is not considered a ‘material’ input from an SROI perspective against

both relevance and significance criteria.

2. Nominal rent payments averaging £10.60p per week for the duration of their stay in

Porchlight accommodation

With a capacity of 21 users per year, total rent from YPS Users is £11,575.20.

Note on Duration:

- The average duration of stay at Porchlight of YPS Users surveyed was 6 months, within a

range of ‘some weeks’ to 1 year. For this SROI, a 1 year duration of input is assumed.

YPS Users typically experience three sets of outputs:

1. Accommodation that users describe as surprisingly clean and comfortable, at one or

more of three different stages:

o High need short-term accommodation (3-6 months)

‘Direct Access’ hostel for emergency residency. This is staffed 24/7 by

Porchlight YPS key workers.

Typically this is the first type of accommodation for YPS Users when they

first come to Porchlight.

o Medium needs, medium-term accommodation (up to 12 months)

‘Medium Stay’ hostel for 12 months residence.

Staffed 9-5 Monday-Friday and occasional weekends.

o Low needs, medium-term accommodation

‘Move-on’ Accommodation for 6-12 months residency.

Not staffed.

2. Regular formal meetings and casual conversation with a dedicated key worker.

o Each YPS User is assigned a dedicated key worker for the duration of their stay at

each type of accommodation.

3. Occasional small, short-term loans for grocery shopping and travel for

training/education purposes.

o Separate from the outcomes secured (shopping, training/education), these

immediate outputs are not considered ‘material’ from an SROI perspective.


52 P a g e

7.2.2. Outcomes and Theory of Change

Method

To expand on Section 5.5 (Details on data collection & sampling) on page 42 above, the outcomes

and summarising visual theory of change outlined in this section were identified and framed using a

three stage process. First, outcomes and the initial theory of change were developed through (a)

open-question interviews and focus groups with YPS users (SROI Stage 2) and (b) comparison with

Porchlight’s existing data collection frameworks comprised of the Homelessness Outcomes Star and

DCLG reporting guidelines. Second, the visual format of this theory of change was confirmed with

YPS Users through the use of a survey (SROI Stage 3). Third and finally, the visual theory of change

was ratified through 1-1 interviews (SROI Stage 6).

Outcomes

Through stakeholder engagement outlined in Section 5.5 (Details on data collection & sampling) on

page 42 above, users reported a range of outcomes and changes in their lives as a result of accessing

Porchlight’s YPS. These include:

Immediate Outcomes:

- Stayed in comfortable, safe accommodation

- Received support to pursue training & education, including courses at college/university,

and training for the army and territorial army.

o Support included emotional support from key workers, and in some cases small

amounts of funding for travel.

- Participated in a occasional in new/different social activities with other YPS users for free,

including fishing, horse-riding, sports, going to the movies, etc.

- Learn domestic and financial management skills: learning to shop, cooking & budget for

yourself.

Longer-term Outcomes

- Changed friendship groups / made new friends

- Increased ability to live independently

- Reduced debts

- Increase confidence & maturity

- Make future plans and goals for themselves

- Reduced substance misuse (dealing and/or consuming)

- New/renewed contact with family members

Negative Outcomes


53 P a g e

- Inability to cope in Porchlight environment resulting in expulsion from Porchlight due to

behaviour viewed as inappropriate.

- Freedom to make some bad decisions (e.g. stop going to school)

- Best case: moves into more supportive accommodation.

- Worst case: made homeless again

- Disruption to work (if YPS users previously lived outside Canterbury)

- Disruption to relationships (if YPS users previously lived outside Canterbury)

The above data were compatible with Porchlight’s existing data framework of Outcomes Star and

Department of Communities and Local Government ‘Supporting People’ measurement.

Accordingly, a comprehensive list of forecast outcomes for YPS Users was developed within the

Outcomes Star Framework. This is produced in Table 7 below. Numbers in parentheses refer to

number of individuals experiencing outcomes.


54 P a g e

Category Self-identified outcomes (SROI Stage 2: n = 12)

DCLG Outcome Scoring (from previous data collection: n=34) (No. users achieving/not achieving outcome)

Outcomes Star Scoring (from previous data collection: n=44) (No. users with better/worse scores after staying with PL)

Housing

Gained comfortable, safe accommodation (5)

Maintain accommodation (36/4) Managing tenancy & accommodation (20/17)

Secure settled accommodation (48/4)

Crime

Comply with statutory orders (8/1)

Offending (18/4)

Better manage harm to others (10/0)

Minimise risk of harm to others (15/0)

Reduced substance misuse (dealing and/or consuming) (2)

Manage substance misuse issues (15/19) Drug & alcohol misuse (18/9)

Keeping house

Gained domestic skills (shopping, cooking, etc) (10)

Self care & living skills (21/19)

Gained budget/finance skills (7)

Reduced debts / increased savings (4)

Maximise benefits/claims (31/1)

Managing money (23/12) Reduce debt (16/0)

Health

Manage physical health (26/8) Physical health (23/5)

Manage mental health (24/10)

Emotional & mental health (24/5)

Better manage self harm (7/1)

Category Self-identified outcomes (n = 10)

DCLG Outcome Scoring (n=34) (No. users achieving/not achieving outcome)

Outcomes Star Scoring (n=44) (No. users with better/worse scores after staying with PL)

Social

Made new/renewed contact with family members (2)

Establish contact with friends/family (0/0)

Social networks and relationships (22/7) (-ve) Moved town / location: lost contact with friends/family (5)

Establish contact with external groups (29/0)


55 P a g e

Category Self-identified outcomes (SROI Stage 2: n = 12)

DCLG Outcome Scoring (from previous data collection: n=34) (No. users achieving/not achieving outcome)

Outcomes Star Scoring (from previous data collection: n=44) (No. users with better/worse scores after staying with PL)

Made new friends

Participated in social activities (3)

(-ve) lived with disruptive/annoying people (1)

Meaningful use of time

(-ve) Moved town / location: Lost existing employment (1)

Informal learning (30/0)

Meaningful use of time (23/10)

Had material and emotion support to pursue training, education &/or employment (3)

Participate in training & education (22/6)

Gained education qualifications and/or employment (5)

Gain qualifications (8/4)

(-ve) made bad decisions for impacting negatively on future goals (e.g. stop schooling) (1)

Participate in work-like activities (11/3)

Obtain paid work (3/9)

Participate in paid work (6/6)

Social Made new/renewed contact with family members (2)

Establish contact with friends/family (0/0) Social networks and relationships (22/7)

Personal Goals Made plans /goals for the future(3) Developing confidence & choice (29/5) Motivation & taking responsibility (25/10)

Made decisions for oneself (2), including

Increased confidence & maturity (7)

Technology helping to maintain independence (0/4)

Increased ability to live independently in a sustainable way (2)

(-ve): Expelled from Porchlight (1)

Table 7. Summary Collation of potential outcomes for YPS users (Stage 2 data, existing Porchlight reporting)


56 P a g e

Very long-term outcomes:

- For the purposes of this Forecast SROI, it is assumed that improvements on the above

outcomes is likely to lead to reduction in the likelihood that YPS users will become long-term

homeless as adults, and further outcomes for government departments as outlined in

further sections below.

To summarise the foregoing, it is hypothesised that Porchlight’s outcomes for YPS Users are

connected together in a ‘theory of change’ as follows:

Through staying in Porchlight accommodation and receiving support from dedicated key-workers,

YPS Users first (1a) gain safe accommodation and a home ‘base’ from which to live their lives, (1b)

make new friends and gradually experience some improvements in the relationships with extended

family, and (1c) learn practical skills to mange money and live independently. Second, as a result of

these changes, some YPS users (2a) experience improvements in their mental and physical health and

(2b) use their time more productively by engaging in education, vocational and other forms of

training and employment. Third, at the same time some YPS users also (3a) gain in confidence and

maturity in their interactions with others and setting life goals for themselves, (3b) engage less in

anti-social and criminal behaviour that bring them into contact with the police and the justice

system, and (3c) more generally gain the strength and ‘resilience’ to respond to negative events and

‘life’s knocks’ without losing confidence or reverting to damaging behaviours that would result in

being excluded from independent housing or the family home. Fourth, some YPS users experience

negative outcomes: (4a) disruption to existing friendships and work when they move from outside

Canterbury into YPS Canterbury, and more significantly (4b) potentially becoming homeless again

following expulsion from Porchlight due to repeatedly breaking house rules. Fifth and finally, as a

result of the foregoing changes, a number of YPS users will (5a) have significantly increased likelihood

of achieving long-term employment in adulthood and (5b) avoid adult homelessness.

These outcomes are represented visually within a ‘theory of change’ outlined in Figure 10 below,

with numbers and percentages of YPS users experiencing each outcome..


57 P a g e

Figure 10. Theory of Change for YPS Users

7.2.3. Outcomes Star and academic evidence for long-term social impact


‘Outcomes Star – Homelessness’ (see (MacKeith, 2011)) is a predictor of long-term outcomes. For

example, a St Mungo’s briefing outlined that the use of the Outcomes Star system anecdotally

corresponds with other systems to measure qualitative outcomes (Triangle Consulting, 2013).

However, further St Mungo’s research notes that typically “Positive outcomes peak at 6 -12 months,

longer stays can be associated with a decline in progress and an increase in mental health problems.

Clients with multiple needs were most likely to slip backwards substantially.” (St Mungo's, 2012, p.

4). Furthermore, to date, the only formal, publicly available statistical data comparing Outcomes

Star data with ‘hard’ data connected with Department of Community and Local Government

Supporting People Funding (for homelessness), suggests that “there is currently no evidence that

there is a statistical linkage between the recorded by the Outcomes Star (the ‘soft’ outcomes) and

that recoded by the Link database (the ‘hard’ outcomes)” (Boswell & Skillicorn, 2009, p. 20).


58 P a g e

For these reasons, the use of Outcomes Star outcomes within the period that YPS Users remain at


appropriate for use with a Forecast SROI.

7.2.4. Outcome Indicators

Table 8 below outlines the indicators used to measure outcomes for Users. The primary source of

indicators is the Homelessness Outcomes Star, which is widely used in the sector and by Porchlight

for many years. See sub-section ‘YPS User – Existing Outcomes Data.’ on page 36 for more

information. Further, more ‘objective’ indicators were developed with this Forecast SROI for

triangulation purposes.

Accordingly, indicators are described as ‘subjective,’ ‘objective,’ or ‘composite’ when the data is

derived from respondents’ subjective views but guided by a survey with a specific number of items

(questions) and fixed responses on a scale. For exact survey questions, please see Appendix 11.4.1.

For the purpose of this Forecast SROI, the pre-YPS and post-YPS scores on surveys are estimates

only, made by YPS Users made at the one and same time – i.e., Users record what they believe there

score was prior to moving into Porchlight, and now (at current location or after leaving Porchlight).

This data was comparable with existing Porchlight data on improvements by YPS users against

Outcomes Star measured at multiple points during the course of each users stay.


59 P a g e

Outcome Indicators

Having a place to stay (safe accommodation)

Actual no. of weeks youth spend in YPS housing = 52 weeks x 100% Porchlight capacity (21 users)

Improved motivation and taking responsibility

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale) - Objective: Change in scores on survey measuring resilience & self-esteem (6 items, 5-point scale) - Objective: Change in scores on life satisfaction and happiness survey (2 items, 11-point scale)

Improved self-care and living skills, including cooking and shopping

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale) - Objective: Change in proportion of meals cook for self

Managing money and personal administration, including budgeting and reducing debt

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale) - Objective: Change in proportion of weeks break budget

Better friendships, relationships and social networks

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale) - Objective: Change in number of times per year speak with parents/guardians & siblings - Composite: Change in level of stress and enjoyment when seeing family on 7-point scale - Composite: Change in level of loneliness on 7-point scale

Reduced alcohol and drug misuse

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - 1. Subjective: change in scores on Outcomes Star (10-point scale) - 2. Objective Alcohol & Drugs : Change in number of times drink alcohol/take drugs per year - 3. Objective Misuse: Change in level of harm from substance use on a 3 point scale

Improved physical health Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale)

Improved mental and emotional health

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - 1. Subjective: Improved score on Outcomes Star sub-scale (10-point scale) - 2. Composite: Improved score on life happiness score (10-point scale)

More meaningful use of time including study, training/education and gaining formal qualifications, employment, volunteering,

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - 1. Subjective: Improved score on Outcomes Star sub-scale (10-point scale) - 2. Objective: Gained formal NVQ qualification while at Porchlight


60 P a g e

hobbies) - 3. Objective: Change in pre- & post-YPS hours weekly hours spent working

Better able to manage a private tenancy and your accommodation

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - 1. Subjective: change in scores on Outcomes Star (10-point scale) - 2. Objective: Score of at most 10 on a sub-scale measuring confidence in managing own tenancy (5-items, 5-point scale)

Reducing offending and interaction with police and justice system.

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Subjective: change in scores on Outcomes Star (10-point scale)

Increased long-term likelihood of employment

Number of YPS Users gaining a formal qualification while staying at Porchlight * 85% youth employment/employment rate in the quarter to March 2013.

(Negative) Reduced contact with friends due to excessive travel cost/time after moving to Porchlight

Number of YPS Users stating they stopped seeing friends due to excessive travel cost/time

(Negative) Reduced salary from work due to excessive travel cost/time after moving to Porchlight

Number of YPS Users stating their work salary was reduced due to excessive travel cost/time

(Negative) Expulsion from Porchlight for repeatedly breaking house rules

Estimated no. of YPS Users whose short-term lease is not renewed, with departure prior to originally plan.

Reduction in long-term homelessness

Estimated no. of youth who will avoid adult homelessness = Number of YPS Users who achieve average scores of at least 8/10 on Star Outcomes

Table 8. List of YPS User Outcomes and indicators.

Quantities according to each indicator are recorded in the Impact Map, using additional ‘quantity’

columns. Final quantities used for the SROI calculation are arrived at by triangulation, using

averages of quantities according to multiple indicators. See Quantity columns in Impact Map (in

appendix) for further details.

Justification for choice of selected indicators

1. Achieving NVQ Qualifications as an indicator for increased likelihood of long-term (adult)

employment

The authors of an SROI of another charity supporting homelessness, Crisis, use NVQ qualifications

attained as an indicator of employability. Their argument is as follows:

“Using evidence from Layard, McIntosh and Vignoles (2002) and McIntosh and

Vignoles (2000), The Princes Trust (2008) argues [that] the acquisition of Level 1

numeracy or literacy skills raises the probability of employment by about five

percentage points. For those in employment, it raises wages by about nine

percentage points in the case of numeracy skills and seven percentage points in

the case of literacy skills” (Oxford Economics, 2009, p. 8).


61 P a g e

Based on this evidence, and accounting for the fact that some YPS users gain further qualifications at

NVQ Level 2 and even 3 while at Porchlight, it is assumed that as a result of gaining qualifications YPS

users are 10% more likely to gain long-term employment.

2. Scoring average of 8/10 across all Star Outcomes as an indicator for complete avoidance of long-

term (adult) homelessness.

This indicator is based on Porchlight experience. The subsequent, evaluative SROI will require use of

long-term longitudinal data, either from existing academic literature or through Porchlight’s own

tracking of YPS users into adulthood.

7.2.5. Financial Proxies and Outcome Groupings

All outcomes in the YPS User theory of change are taken forward for valuation. However, to avoid

‘double counting,’ financial proxies are assigned not to each outcome, but groups of outcomes that

could be expected to have a significant amount of overlap are grouped, as outlined in Table 9 below.


62 P a g e

Individual Outcomes Outcome Group Financial Proxy selected

Having a place to stay (safe accommodation) Safe accommodation Median weekly rent for one room in a 4-bedroom share house: £1300 pcm for house / £76 pw per room. (Home.co.uk, 2013)

Improved self-care and living skills, including cooking and shopping

Increased independent living skills

Cost of attending a personal budgeting and management course: £59. (Matrec, 2013).

Managing money and personal administration, including budgeting and reducing debt *


Improved motivation and taking responsibility Mental health, confidence, responsible behaviour

Cost of weekly private counselling sessions for one year: £40 x 52 weeks = £2,080 pa. (NHS, 2013)


Improved mental and emotional health *

Reducing offending and interaction with police and justice system.

Improved physical health Physical health Average junior annual membership fees for a community sports club in the UK: £61 pa. (Sport + Recreation Alliance, 2011, p. 24)

Better friendships, relationships and social networks Positive social life

One quarter of the average annual expenditure on recreation and culture by two adult UK households with children: £80 per week / £1040 pa. (ONS, 2012, p. Table 3.6)

More meaningful use of time including study, training/education and gaining formal qualifications, employment, volunteering, hobbies)

Employment, Education, Training

Annual minimum wage for 21 year olds working full time: £6.19p per hour / £11,266 pa. (Directgov, 2013)

Increased long-term likelihood of employment *

(Negative) Reduced contact with friends due to excessive travel cost/time after moving to Porchlight *

Disruption to friends & employment to move into Porchlight

Kent Annual bus ‘Freedom Pass’ for unlimited free travel within Kent, for school studies or youth over 16 in care: £100 pa. (Kent County Council, 2013)

(Negative) Reduced salary from work due to excessive travel cost/time after moving to Porchlight

(Negative) Expulsion from Porchlight for repeatedly breaking house rules

(Negative) Expulsion Median cost of renting one room in 4-bedroom house in Canterbury for 6 months: £1300 pcm for house / £1,950 per room for 6 months. (Home.co.uk, 2013)

Reduction in long-term homelessness Avoided adult homelessness

Calculation of WTP for estimated 15 years Quality Adjusted life years difference between homeless and non-homeless in UK (see below for calculations and references).

Table 9. Outcome groupings for purposes of assigning financial Proxies

* Outcomes to which financial proxies are attached in the impact map (Appendix 11.10) for transparency.


63 P a g e

A Stated preference (willingness to pay) methodology was initially selected to determine the value

of the above outcomes/outcome groupings to YPS users and staff. However, the above financial

proxies were selected as being more appropriate, following discussion with SROI Network assurers.

Please see Appendix 11.9 on page 107 for discussion about the stated preference method originally

used. The only outcome for which the stated preference methodology was used indirectly is

outlined below.

Financial Proxy for value of avoiding adult homelessness

A recent multi-country academic study investigated 22,000 European’s stated Willingness to Pay

(WTP) for one quality adjusted life year (QALY). The study found a trimmed mean of WTP per QALY

ranging from $18,247 to $34,097 (Robinson, Gyrd-Hansen, Bacon, Baker, Pennington, & Donaldson,

2013). Using a year-long average exchange rate of £0.65p per US£1 (Oanda.com, 2013), this implies

a WTP per QALY of £28,072 – £52,457. For the purposes of this SROI, the minimum £28,072 WTP

per QALY was selected.

There is no available data on average differences between homeless and non-homeless people in

the UK in terms of ‘Quality adjusted life years’ (QALY). However, the average difference in actual life

expectancy is 30 years (Crisis, 2011, p. 2). For the purposes of this SROI, the average difference in

QALY is very conservatively estimated at 50% of 30 years = 15 years. In other words, despite dying

30 years younger than non-homeless people, it is assumed that adjusting for quality of life, homeless

people only live 15 less ‘quality’ years than the non-homeless.

Accordingly the total value of avoiding a reduction of 15 QALY through adult homelessness is

estimated at £323,317 (NPV of £28,072 p.a. for 15 years, using the standard 3.5% discount rate).


64 P a g e

7.3. YPS Users’ families


Inputs:

- It is assumed that YPS users’ families input is disruption to their relationships with YPS Users,

primarily as a result of the distance/cost of travel to Porchlight (Canterbury) from elsewhere

in Kent.

- However, as no YPS Users reported this input, it is assumed that no family provided this

input.

Outputs:

- No outputs are experienced by users’ families.


For the purpose of this Forecast SROI, in the absence of direct engagement with users’ families it is

assumed that users’ families may experience a slight net improvement in the quality of their

relationship with YPS users due to their stay at Porchlight. It is also assumed that users’ guardians

avoid costs associated with their children’s upkeep, including accommodation, food, clothing, etc.

Parents and guardians: Users’ relationships with their parents/guardians is typically highly negative

when they first come to Porchlight, as they have invariably either leave or have been ‘kicked out of

home.’ However, through their time at Porchlight, users typically become more responsible and re-

establish some amount of contact with parents/guardians. There is anecdotal evidence from YPS

User Stage SROI 2 & 3 interviews that this leads to gradually improved relationships with

parents/guardians.

Siblings and extended family (including grandparents): Users’ relationships with extended family

typically improves during their stay at Porchlight, except when they can no longer see them due to

excessive distance/travel (as outlined in Inputs in Section 7.2.1 on page 51 above). This is latter

point is a negative outcome (or more accurately, input) of YPS Users stay at Porchlight.

In the absence of direct data from users’ families, these outcomes remain hypotheses only and must

be further assessed.

7.3.3. Outcome Indicators and Financial proxies

Only two outcomes are included in this forecast SROI: improved relationship with YPS User and

avoided costs of YPS users’ upkeep.

These were measured via YPS Users only, using the same outcome indicator (see below) and

financial proxy (User willingness to pay as a very rough estimate) as used for YPS Users.


65 P a g e

Outcome Indicator Financial Proxy

Improved relationship with YPS Users

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators: - Objective: Change in number of times per year speak with parents/guardians & siblings - Composite: Change in level of enjoyment when seeing family on 7-point scale - Composite: Change in level of stress when seeing family on 7-point scale

One quarter of the average annual expenditure on recreation and culture by two adult UK households with children (ONS, 2102, Table 3.6) = £1,040

Avoided cost of child upkeep (accommodation/food, etc)

Actual no. of weeks youth spend in YPS housing, that would otherwise would be provided by local authority under legal obligation = 52 weeks x 100% Porchlight capacity (21 users)

Weekly cost of child upkeep (Liverpool Victoria (LV=), 2013, p. 3). = £204


66 P a g e

7.4. Partner Organisations

To better support its YPS users, Porchlight has working relationships with a range of charitable and

public agencies (its ‘partner organisations’). These relationships typically involve 2 aspects:

- Referrals:

o Inward referral into Porchlight, of prospective YPS users to YPS accommodation

o Outward referral to other organisations, of current YPS users who could benefit

from other services in addition to YPS accommodation and support.

- Ongoing liaison:

o For YPS users who access services provided by partner organisations on an ongoing

basis, Porchlight and partner organisation staff share information on a regular basis.

7.4.1. Inputs

Inputs for referral and partner organisations are:

- Time required by staff to cross-refer young people between YPS and other organisations.

o This includes staff helping their young clients to fill out application forms and

occasionally accompanying young people to interviews or other appointments to

assess eligibility.

- Time taken to maintain referral arrangements and train YPS staff on issues relevant to

making appropriate referrals.

- Time required for ongoing liaison with Porchlight staff regarding mutual clients

Summary inputs are provided in Table 10 below. For detailed calculations of time inputs, please see

Table 10 in Appendix 11.8 on page 105.

7.4.2. Outputs

Outputs for referral and partner organisations are the cross-referrals:

- A number of young people referred to Porchlight’s YPS and/or to partner organisation

services

- A number of young people accepted by YPS and/or partner organisation services

SROI Element Item

Organisation 1 – KCA (Drug & Alcohol Misuse Charity)

Organisation 2 – Canterbury College

Organisation 3 – NHS Foundation Trust (Children’s Community Mental Health)

Organisation 5 –Kent Social Services (Children and Families) – Homelessness Unit

Activity

Provide support to young people to tackle drug and alcohol dependency

Main college / education provider in Canterbury

Provide therapy for young people with mental health issues

Statutory support of young people and risk assessment (neglect, child protection, etc)

Outputs Inward referral to YPS: 3 per year 10 per year 10 per year 50 per year

Outward referral to partner organisation

~ 2-3 per year (constituting < 1% of total referrals) N/A 40 per year 10 per year

Input time (hours) Frontline 27 14.75 190 230

Management 8 8 8 8

Total 35 22.75 198 238

Estimated salary Frontline £25,000 £25,000 £25,000 £25,000

Management £35,000 £35,000 £35,000 £35,000

Estimated total cost Frontline £438.31 £239.45 £3,084.42 £3,733.77

Management £182 £182 £182 £182

Total £620.13 £421.27 £3,266.23 £3,915.58

Table 10. Summary of monetised inputs for YPS partner organisations.


The YPS produces outcomes for a range of ‘partner organisations,’ including:

- Local Authority social services, including children’s services and adult services - Other charity providers, particularly a youth drug & alcohol support agency - Canterbury College - NHS: mental health services, GPs and hospital A&E departments - Police and the justice system

The following two sections elaborate on these outcomes and indicators for short-term and longer-term outcomes respectively. 7.4.3.1. Short-term outcomes for partner organisations

- Housing cost-savings

o Local Authorities have a legal obligation to house homeless youth under the age of 18 who have been ‘thrown out of the family home’ (Shelter, 2009). However, through its accommodation, Porchlight saves Local Authorities the cost of needing to house YPS users in more expensive accommodation such as B&Bs and foster care.

- Retained/increased funding: o The college reports that 70-80% of Students who become homeless cannot find a

stable place to live and drop-out of school, with a financial cost to the College. However, having a stable tenancy contributes to a less chaotic life-style and increased attendance and engagement at College. According to Canterbury College, Porchlight significantly reduces the changes of students made homeless dropping out of college.

o Canterbury College receives government funding for each student under 19 years old.

- Resource re-allocation

due to time savings in dealing with homeless youth (short-term)

o Due to the regular contact and positive relationships users typically have with Porchlight key workers, referral and

Porchlight is quite well known as a service within Canterbury. They

sometimes house people that have quite severe mental health needs.

Referrals from PL are appropriate to our service. Whereas with GPs,

we get loads of inappropriate referrals.

[Also] Porchlight provide 24/7 access for the young people to their

staff in one of their houses. They are gate-keeping, they are they are

doing preventative work, preventing the young people’s mental

health deteriorating [e.g. and starting arguments with other people

or self-harming]. If Porchlight weren’t there, we would have to see

them [the young people] a lot more...”

– Community Children’s Mental Health Nurse, NHS Foundation Trust

“Porchlight is invaluable. There’s just no

other resources in our district for young

people who, for whatever reason, have left

home, and have become homeless.

– Lead on Homelessness for 17-18 year

olds, Canterbury Social Services (Children

and Families short-term Team)


69 P a g e

partner organisations also experience time-savings dealing with their young homeless clients in three ways:

o Highly-appropriate referrals of users that meet referral and partner organisation’s criteria. This saves time compared to organisations that make inappropriate referrals.

Estimated time saving per client is 20 minutes (child mental health team)

o Reduced case load for existing client outreach based on their housing situation. Social Services youth homelessness and child mental health teams do not spend any time liaising with YPS users, in contract with a range of hours for youth housed in other accommodation. Estimated time saving per client is:

Social services (homelessness unit): 1 hour weekly meetings with clients avoided. Number of weeks for social services – relevant clients is:

Direct Access accommodation: average length of stay = 12 weeks.

Medium-needs accommodation: average length of stay = 28 weeks NHS (child mental health team): 14 hours per client. See Table 18 and Table 19 in Appendix 11.8 on page 105 for more

calculations.

o Improved client attendance and engagement at meetings. Partner organisations

report that due to their clients who are YPS users are more likely than other clients

to both attend meetings and have greater motivation to engage constructively. This

often results in YPS users being less likely to require full support.

For example, the child mental

health team report that YPS users

typically only need 6 sessions of

therapy compared to 12 for other

clients with the same severity of

issues.

A summary of short-term outcomes is provided in Table 11 below.

“Anecdotally, our staff feedback is that

when there are professionals like

Porchlight staff involved, then... that

helps with [clients’] motivation, so

attendance [at meetings] goes up...

and [our] staff are able to access

clients.”

– KCA (Partner organisation working

in drug and alcohol)


70 P a g e

SROI Element Organisation 1 – KCA (Drug & Alcohol Misuse Charity)




Organisation 6 – Kent County Council(s) (Housing)

Service provided

Provide support to young people to tackle drug and alcohol dependency

Main college / education provider in Canterbury

Provide therapy for young people with mental health issues

Statutory support of young people and risk assessment (neglect, child protection, etc)

Legally obliged to provide housing for homeless youth

£ Cost Savings

Yes – Avoided loss of gov’t revenue. (£1,482 per student per year)

Yes – Avoided duty to provide housing (£2,767 per week per person)

Time saving from appropriate referrals – – Yes – estimated at 20 mins per YPS client (£39 per hour)

–

Time saving from reduced case-load of homeless youth, based on improved client (YPS User) attendance and engagement

– – 14 hours per YPS client (NHS Unit cost: £39 per hour)

1 hour per YPS client per week at YPS (Social services unit cost: £74 per hour)

More effective work (e.g. from knowledge sharing)

Yes (Not monetised)

– – – –

Table 11. Summary of short-term outcomes and financial proxies for YPS partner organisations.


71 P a g e

7.4.3.2. Medium- and long-term outcomes for public agencies

- Housing cost-savings (medium-term)

o Following their stay at Porchlight but prior to turning 21 years old, the majority of

YPS Users move into accommodation with family and friends, and do not require

support from local authority housing or social services.

- Resource re-allocation due to reduced burden of dealing with homeless adults (long-term) o During their stay at Porchlight and with their key workers, YPS Users make a series

of gradual improvements in a range areas, as outlined in Section 7.2.2 on page 52 above. These include:

Health Anti-social behaviour Employment (and education & training) Ability to hold a tenancy.

o These improvements will lead to reduced use of public services in the long-term, as

outlined in Table 12 below.


72 P a g e

Stakeholder Outcome Indicator of extent of outcome Local Authority Housing Services (medium-term)

Medium-term cost-savings (housing for youth immediately post-Porchlight): avoided legal obligation to house homeless youth

Estimated no. of weeks youth will spend in private housing post YPS/pre age 18, that would otherwise would be provided by local authority under legal obligation = 52 weeks * Estimated 65% of YPS Users that move into accommodation provided by family or friends rather than publicly-funded accommodation.

Local Authority housing & social services (long-term)

Long-term resource reallocation away from homeless adults due to reduced public housing and social services support (adult homelessness)

Estimated no. of youth who will avoid adult homelessness = Number of YPS Users who achieve average scores of at least 8/10 on Star Outcomes

NHS Long-term resource reallocation away from homeless adults due to improved physical & mental health outcomes and (projected) reductions in adult homelessness

Estimated number of visits by YPS Users avoided per year to GPs = Number of YPS Users who achieve improved Star Outcome (Physical Health) score x estimated 3 GP visits per year.

Long-term resource reallocation away from homeless adults due to improved physical & mental health outcomes and (projected) reductions in adult homelessness

Estimated number of visits by YPS Users avoided per year to Hospitals (self-harm) = Number of YPS users raising both Star Outcome Score (Mental & Emotional Health) Life happiness score from less than or equal to 3 / 10 to over 3 / 10 x 5 A&E visits per year.

Ministry of Justice Long-term resource reallocation away from homeless adults due to improved crime/ASB outcomes and (projected) reductions in adult homelessness

Estimated number of contacts with police from YPS Users avoided per year (emergency call-outs)

HMRC

Long-term tax revenue increase due to YPS Long-term improvement: employment

Estimated number of extra FTE positions (minimum wage) worked by YPS Users per year

Table 12. (Long-term) Outcomes and Indicators for public agencies.


73 P a g e

The foregoing description of short, medium- and long-term outcomes for partner organisations can

be described in a hypothesised theory of change as follows:

As a result of receiving accommodation and support from Porchlight, homeless youth:

(5) No longer fall under the local authority duty of care, which discharges Local Authorities from

their housing obligations.

(6) Engage more constructively with partner organisations, which results in YPS users:

a. Making quicker short term progress with partner organisations and thereby a short-

term time saving for partner organisations, and

b. Staying in college for longer or attending college, which results in increased

(government sourced) revenue for the college.

c. Making greater long-term improvements as described in the previous theory of

change for YPS users, which results

i. Increased likelihood of employment as adults, and increased tax revenue for

HMRC

ii. Reduced burden on a range of statutory and other public services including

social services, social housing, police and the justice system, the NHS and

DWP. This reduced burden results in reallocation of public resources away

from homeless youth and adults to other groups in need.

Furthermore:

(7) Partner organisations receive relatively more appropriate referrals from Porchlight than

elsewhere, resulting in a time saving and consequent resource reallocation to other clients.

(8) Porchlight and partner organisations share knowledge which results in partner organisations

more effectively being able to deliver on their own work and meet their own contractual

obligations.

This theory of change is outlined in visual format in Error! Reference source not found. below.


74 P a g e

Figure 11. Theory of Change for partner organisations and public agencies.

7.4.4. Local Authority housing and social services Outcome Indicators

Double counting is avoided between local authority housing departments’ short-term cash savings

and medium-term resource re-allocation, by carefully defining the relevant indicators to ensure the

two outcomes do not overlap:

- Outcome: housing services short-term cash savings while clients are at Porchlight

o Indicator: the number of weeks YPS users are in Porchlight accommodation

- Outcome: housing services medium-term resource reallocation after clients leave Porchlight

o Indicator: the number of weeks YPS users spend in private housing post Porchlight

but prior to age 18 only, which would otherwise have been funded by local authority

housing services.

- Outcome: social and housing services long-term resource reallocation after clients become

adults:

o Indicator: Estimated no. of youth who will avoid adult homelessness, i.e. after the

age of 18

Indicators for other outcomes experienced by partner organisations are outlined in the impact map.


75 P a g e

7.4.5. Financial Proxies and resource reallocation

The primary outcome for public agencies is resource reallocation away from YPS users, due to

reduced time spent with clients in the short-term and reduced burden on services in the medium-

and long-term.

The unit costs of delivering the relevant services that the homeless clients would access are selected

as financial proxies selected for public sector ‘resource reallocation. This follows guidance in the

SROI Network Guidance for financial proxies for resource reallocation and indirect cost-savings:

“[Unit] costs identified are proxies for this outcome and produce a way of valuing the resources

made available to the health [and other] service[s] with which it can now do other things” (Nicholls,

Lawlor, Neitzert, & Goodspeed, 2012, p. 51).

For examples specific to homelessness charities, see also (Oxford Economics, 2009, p. 11) and (New

Policy Institute, 2003).

The following financial proxies are selected to value outcomes:

- Cost-savings: o Local Authority – Avoided legal duty to provide housing to homeless youth

Establishment costs per resident for local authority care homes: £2,767 per week (PSSRU, 2011, p. 74).

Valuing this cost-saving does not involve ‘double counting’ with the ‘safe accommodation’ outcome for YPS users, as without Porchlight’s intervention, the Local Authority would still need to act as their legal guardian.

- Resource reallocation away from youth homelessness due to time savings (short-term): o Social Services (homelessness & mental health)

£74 per hour of client-related work, based on unit costs of a social worker (children’s services) (PSSRU, 2011, p. 157).

o Resource reallocation away from homelessness due to time savings (Long-term):Local Authority Housing:

Difference in costs between a first-stage hostel for homeless adults and

public housing rent cap on bed-sits: £24,323 (Oxford Economics, 2009, p. 9).

o NHS (long-term resource reallocation due to improved physical health):

Average cost of hospital admission for homeless person: £2,115 ( (Deloitte, 2012, p. 6) The following outcomes were excluded from the SROI analysis due to materiality considerations (see Section 6.1 Audit Trail on page 47). However, financial proxies are reproduced here and in the impact map for the sake of transparency:

o NHS Mental health Team (short-term resource reallocation due to improved

mental health and reduced mental health therapy)

£39 per hour, based on unit costs of a community mental health team (PSSRU, 2011, p. 168).

Long-term resource reallocation due to improved physical health)):

Cost of GP visit per clinic consultation: £53 (PSSRU, 2011, p. 149)


76 P a g e

o Justice system (resource re-allocation due to reduced ASB/offending):

Cost of police call-out: £629 (New Economy, 2012)

o Education providers (Canterbury College): Maintaining government-sourced revenue from (formerly homeless)

students remaining in college.

Fee charged for Further Education Courses (free for Ages 16-18/recipient of benefits): £1,482.00 p.a. (Canterbury College, 2013).

o Charity partner organisations: More effective working:

Not monetised.


77 P a g e

7.5. Funders


Funders typically provide two major inputs:

- Cash funding, as outlined in sub-section ‘Funders’ on page 38 above, totalling:

- Other funding, notably housing benefit, as outlined in sub-section ‘Funders’ on page 38

above.

Funders experience one type of output only, namely quarterly and annual reporting as required by

funders, on the YPS the use of their funds and the YPS as required by funders.

7.5.2. Outcomes

Discussions with funders concluded that they did not experience any material outcomes directly,

separate from securing outcomes for other stakeholders. These were identified by funders as:

- Empowering young people to be able to

manage a tenancy and avoid homelessness

- Reducing demand on statutory services,

including:

o Housing

o Health (mental, physical, and drug &

alcohol services)

o Justice, including youth offending

teams, police, and prisons/probation

services

o Social care and social services,

including youth services

o Schools & education

This generally accords with outcomes identified by

other stakeholders, and therefore no additional

outcomes are taken forward for valuation.

Porchlight are an A-grade provider, they are

constantly showing continuous improvement.

Of the 100 providers with ‘Supporting People’

contracts providing 500 services, only 10% are

A grade.

Porchlight have a good reputation, and being

Kent based, have experience in addressing

localised needs. This gives them an advantage

over national organisations.

It is estimated that every £1 of Supporting

People non-statutory funding, including to

Porchlight, is worth £3 to statutory agencies.

Supporting People Commissioner,

Kent County Council


78 P a g e

8. SROI Stage 4: Establishing impact

The fourth stage of an SROI involves identifying extents of ‘impact factors’ mitigating outcomes and

their valuations, namely:

1. Duration – how long outcomes are likely to last after intervention ceases.

2. Deadweight – the extent to which outcomes would occur in the absence of an intervention

3. Displacement – the extent to which outcomes are counterbalanced by negative/adverse

outcomes elsewhere due.

4. Attribution – the extent to which outcomes were caused by other organisations or people

rather than the intervention under analysis.

5. Drop off – the amount of the outcomes that disappears each year after an intervention

ceases without further support.

Methods used to estimate the impact factors mitigating outcome valuations are outlined for each

stakeholder group in turn.

8.1. YPS User Impact Factors

A review of prior academic literature relevant to the UK highlighted that some commonality in the

risk/prevention factors for long-term adult homelessness, and hence likely sources of attribution,

deadweight and displacement. See in particular (Joseph Rountree Foundation, 2000). However,

existing research does not provide robust determinations of the likely contribution of different

interventions in preventing homelessness.

Given prior research is of little guidance, impact factors were determined through consultation with

YPS Users and Porchlight staff. This is a weakness in the research. Recommendations for the

subsequent, evaluative SROI are outlined in Section 10.2 on page 85 below.

Displacement

Similar to previous homelessness SROI analyses for Crisis and Emmaus – see (Bagley, 2012) and

(Lawlor, 2012) – supporting homeless youth was judged to be subject to 0% displacement of

outcomes for YPS Users.

Other impact factors

All other impact factors for outcomes for users, with the exception of displacement and outcomes

listed below were primarily derived from user surveys. Survey respondents were asked to estimate

the level of impact factors as follows:

- Duration:

o Users: < 1 week, 1-2 months, 1 year, 2-3 years, 5+ years


79 P a g e

o Staff: Immediately, 1-2 weeks, 1-2 months, less than half a year, 1 year, 1-2

years, 3+ years

- Deadweight: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

- Attribution: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

- Drop off: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

Exceptional outcomes are:

- Having a place to stay (safe accommodation)

o Impact factors determined by SROI Working Group.

- Reducing offending and interaction with police and justice system


- Increased long-term likelihood of employment

o Impact factors determined by SROI Working Group, with reference to the

literature as outlined in the ‘justifications’ sub-section of section 7.3.3 from page

64 above.

o To reprise, a previous CRISIS homelessness SROI calculates NVQ Level 1

qualifications contribution an additionality of 5% – 9% increased likelihood of

long-term employment (Oxford Economics, 2009, p. 8). Given many Porchlight

YPS Users gain NVQ Level 2 & 3 qualifications, it is assumed that as a result of

gaining qualifications YPS users are 10% more likely to gain long-term

employment.

o Given that the youth employment rate is approximately 80% (McGuinness,

2013), we summarise the contribution of Porchlight as providing: attribution of

85% (employment boosted by 10% for NVQ qualifications), but deadweight of

75% (employment less 10% boost for NVQ qualifications)

- (Negative) Expulsion from Porchlight for repeatedly breaking house rules


- Reduction in long-term homelessness


8.2. Users’ families

The same impact factors for ‘improved relationship with YPS Users’ were used as the mirroring

outcome for Users (‘improved relationship with family’).

8.3. Referral and partner organisations

Impact factors for partner organisations were estimated by the SROI Working group, in consultation

with partner organisations.

Impact factors for statutory authorities/public agencies were estimated by the SROI Working group,

with reference to impact factors nominated by users (See Section 8.1 above).


80 P a g e

9. SROI Stage 5: Calculating the SROI

The fifth stage of an SROI involves several steps:

1. Integrating foregoing results on input costs, outcome valuations and mitigating factors, to

calculate the final SROI ratio.

2. Undertaking sensitivity analysis of calculation components

3. Further data collection using refined approaches to rectify major sensitivities of calculations

that could materially distort the final SROI ratio.

9.1. The SROI ratio

The final SROI Ratio is determined by dividing the sum of the Net Present Value for each material

outcome by the total value of inputs. Following methodology outlined in (Nicholls, Lawlor, Neitzert,

& Goodspeed, 2012), the net present value for each outcome is determined by multiplying the

quantity of each outcome by the relevant financial proxy, less any deadweight, attribution and/or

displacement, and then repeating for the number of years’ duration less any drop-off. A discount

rate is applied to the value of outcomes in each subsequent year to determine net present value. In

the absence of any better discount rate for social outcomes, a rate of 3.5% set by HM Treasury for

public sector accounting is used (HM Treasury, 2003, p. v).

9.1.1. Headline Ratio

It is forecasted that in the 2013/14 financial year, every £1 invested in Porchlight’s YPS will likely

produce £5.95 of social value, within a confidence range of £4.30p – £7.60p. This is the equivalent of

approximately £86,000 of social value per YPS user in 2013/14 on total inputs of approximately

£15,000 per YPS user.

It is forecast that YPS Service will require funds and social inputs valued at £726,000 . This includes

(approximately):

- in cash funding from DCLG / Kent County Council (Supporting People Programme)

- in cash funding from other sources, notably DWP / Housing Benefit.

- in costs for YPS Users, including nominal rent payments

- £7,800 worth of inputs from other stakeholders

Based on these inputs, it is forecast that the Canterbury YPS will create a total of approximately

£4,322,000 worth of social value. Of this amount:

- £1,900,000 of social value will accrue to YPS Users (£12,300 per user), who experience a

range of outcomes as outlined in section 1.3.1 on page 6 above.


81 P a g e

- £2,248,000 of social value will accrue to Local Authority Housing and Social Services (£45,000

per YPS User) who avoid costs of housing homeless individuals and can allocate resources

away from supporting homeless people to other groups in need.

o £1,232,000 of this will be accrued in the short term, due to cash savings from local


accommodation, such as bed and breakfasts or foster care, during the time they are

at Porchlight.

o £754,000 of this will be accrued in the medium term, due to cash savings from local


accommodation, such as bed and breakfasts or foster care, between the time they

leave Porchlight and the age of 18.

o £262,000 of this will be accrued in the long-term, due to resource reallocation away

from YPS preventing youth becoming adult homeless.

- £101,000 of social value will accrue to YPS Users’ families

- £73,000 of social value will accrued to the NHS.

Values and Proportions of social value are provided in Figure 12 and Table 13 below.

Figure 12. Breakdown of YPS Canterbury Forecast Social Value by major stakeholder group.


82 P a g e

Stakeholder Breakdown of social value

by stakeholder group

% of total

value

YPS Users £1,900,000 44%

Local Authority Housing Services (short-term) £1,232,000 29%

Local Authority Housing Services (medium-term) £754,000 17%

Local Authority Social & Housing Services (long-term) £262,000 6%

Users' families £101,000 2%

NHS £73,000 2%

Grand Total £4,322,000 100%

Table 13. Breakdown of YPS Canterbury Forecast Social Value by individual stakeholders.

9.1.2. Breakdown of social costs/inputs

Social costs/inputs are outlined in the table below.

Funding Source Likely £ Amount

(2013/14)

Est. per user % of Total

DCLG/Kent County Council (Supporting

People Programme)

£499,856 £9,997 68.8%

Other Funders (DWP Housing Benefit) £207,191 £4,144 28.5%

YPS Users £11,575 £232 1.6%

Local Authority Social Services £3,916 £78 0.5%

NHS £3,266 £65 0.4%

Education Providers £421 £8 0.1%

YPS Staff £248 £5 0.0%

Grand Total £726,473 £14,529 100.0%

Table 14. Breakdown of YPS Canterbury Forecast social inputs by individual stakeholders.

A visual breakdown of the value of inputs is produced in Figure 13 below.


83 P a g e

Figure 13. Breakdown of YPS Canterbury Forecast Social Costs by major stakeholder group.

9.1.3. Sensitivity Analysis

Varying the quantities, values and impact factors (deadweight, displacement, attribution, duration,

drop-off) for each outcome affects the final SROI ratio.

Table 15 below lists the outcomes that most influence the SROI ratio, in descending order of

‘sensitivity’.

A review of the top 5 outcomes was conducted, including consideration of margins for error,

resulting in a confidence range in the final SROI of £4.30p – £7.60p of social value for each £1

invested. The outcomes of most uncertainty (as opposed to outcomes of most variation in value),

are outcomes 1 & 4:

- YPS Users: Increased long-term likelihood of employment

- Local Authority Social & Housing Services (long-term): Long-term cost savings due to

reduced adult homelessness

Insights and recommendations (Section 10.2 from page 85 below) are provided to reduce

uncertainties with these outcomes.

69%

28%

2% 1%

Breakdown of social inputs by stakeholder group

DCLG/Kent County Council (Supporting People Programme)

Other Funders (DWP Housing Benefit)

YPS Users

Others


84 P a g e

Rank Stakeholder Outcome £ Social Value

(PV)

% of

Total

PV

1 YPS Users Reduction in long-term homelessness £1,597,416 37.0%

2 Local Authority Housing Services

(short-term)

Increased cash resources to dedicate to other youth, resulting from avoided legal

obligation to house homeless youth (as they are staying with Porchlight)

£1,231,616 28.5%

3 Local Authority Housing Services

(medium-term)

Medium-term resource reallocation due to avoided legal obligation to house

homeless youth (due to reduced need to housing for youth immediately post-

Porchlight):

£ 753,835 17.4%

4 Local Authority Social & Housing

Services (long-term)

Long-term resource reallocation due to reduced need to manage adult

homelessness

£ 261,915 6.1%

5 YPS Users Increased long-term likelihood of employment £ 184,244 4.3%

6 YPS Users Having a place to stay (safe accommodation) £ 60,007 1.4%

7 Users' families Avoided cost of accommodation/food £ 85,976 2.0%

8 NHS Long-term resource reallocation due to substantially improved physical & mental

health outcomes of homeless adults

£ 73,465 1.7%

9 YPS Users Improved mental and emotional health £ 43,073 1.0%

10 YPS Users Better friendships, relationships and social networks £ 17,706 0.4%

11 Users' families Improved relationship with YPS Users £ 15,493 0.4%

12 YPS Users (Negative) Expulsion from Porchlight for repeatedly breaking house rules -£ 3,551 0.1%

13 YPS Users Improved physical health £ 1,293 0.0%

14 YPS Users (Negative) Reduced contact with friends due to excessive travel cost/time after

moving to Porchlight

-£ 1,153 0.0%

15 YPS Users Managing money and personal administration, including budgeting and reducing

debt

£ 1,122 0.0%

Table 15. List of top 15 outcomes to which the final SROI ratio is most sensitive.


85 P a g e

10. SROI Stage 6: Reporting

The final stage of an SROI involves

1. Engaging with stakeholders on the findings

2. Embedding good outcomes processes

3. Verifying the report.

10.1. Engaging with Stakeholders

The SROI results were shared with stakeholders as outlined in Section 5.5 on page 42. These

resulted in the following changes being made to the report:

- YPS Users

o Legend added to Theory of Change for YPS Users.

o WTP Valuation of outcomes by YPS Users were amended as follows:

Better friendships, relationships and social networks: reduced from £550

to £250.

Reduced alcohol and drug misuse: increased from £100 to £250

Reduced Offending: increased from £100 to £250.

NB: changes made were small, to account for small size of feedback

group (n=5) compared to size of survey (n=22).

Please note these WTP financial proxies were not used in the final SROI

analysis.

- Others:

o Feedback with Commissioner and others did not result in any changes to the

SROI analysis.

10.2. Insights and recommendations

It is of interest that in this analysis (1) excluding the primary outcome of ‘avoiding adult

homelessness,’ 7% of the total value of the Young Persons’ Service accrues to the young homeless

clients themselves, whereas (2) 52% accrues to Local Authority housing and social services

departments in the short- and medium- term cost-savings through avoiding legal obligations to

house homeless youth. Furthermore, (3) in this analysis the proportion of value accruing to the NHS

and justice system was very small (2%).

The reason connecting these disparities is that local authorities will make cost-savings for all 50 YPS

users per year, whereas the SROI Working Group conservatively forecast that, for example, only 14

homeless youth per year will avoid adult homelessness, and that figures is subject to deadweight of

25% and attribution of 50%.


86 P a g e

As a result, a follow-up evaluative SROI should reassess valuations of outcomes for YPS users and,

importantly, more accurately track the number of YPS users that avoid adult homelessness as a

result of Porchlight’s intervention.

Porchlight already has well-embedded measurement of progress of YPS users against the Outcomes

Star Homelessness framework, involving regular check-ins and estimates of changes in outcomes on

the Outcomes Star 11-point scale.

However, several changes should be made to impact measurement systems to allow for effective

progress to an evaluative SROI:

10.2.1. Incorporate Objective Indicators into outcome measurement

The following, further outcome indicators should be used to measure YPS User Outcomes that are

most sensitive for SROI analysis (after number of YPS users avoiding adult homelessness).

Measurement should be integrated within existing outcome reporting processes by YPS Staff using

their key workers’ regular 1-1 evaluation sessions with YPS Users.

Stakeholder Outcome Indicator

YPS Users

Better friendships, relationships and social networks

Number of YPS Users scoring improvement between Pre-YPS & Post-YPS scores on multiple indicators:

- Subjective: change in scores on Outcomes Star (10-point scale)

- Objective: Change in number of times per year speak with parents/guardians & siblings

- Composite: Change in level of stress and enjoyment when seeing family on 7-point scale - Composite: Change in level of loneliness on 7-point scale



- 1. Subjective: change in scores on Outcomes Star (10-point scale)

- 2. Objective Alcohol & Drugs : Change in number of times drink alcohol/take drugs per year

- 3. Objective Misuse: Change in level of harm from substance use on a 3 point scale



- 1. Subjective: change in scores on Outcomes Star (10-point scale)

- 2. Objective: Score of at most 10 on a sub-scale measuring confidence in managing own tenancy (5-items, 5-point scale)

Increased long-term likelihood of employment

Number of YPS Users gaining a formal qualifications: - Number of qual’s (NVQ Level 1) - Number of qual’s (NVQ Level 2) - Number of qual’s (NVQ Level 3)

Table 16. List of additional indicators to be integrated into existing Porchlight measurement.


87 P a g e

10.2.2. Increased engagement with graduating and former YPS users

Engaging with YPS Users after they leave Porchlight is crucial to understanding the long-term impact

of Porchlight for its users, and for government agencies. A variety of means could be adopted to

boost contact and impact measurement with former YPS Users. These include:

- Data systems

o To the greatest extent possible, Porchlight should link datasets between its YPS

and adult homelessness services, to allow for long-term tracking of Porchlight

end-users that return to Porchlight as adults.

- Impact measurement and reporting

o Upon leaving Porchlight, YPS Users should be asked to:

Review SROI outcomes and values relevant to them

Provide observations/feedback on how to better collect and report on

impact that is relevant to them

- General engagement:

o Porchlight should organise annual social events for former YPS-users

o Porchlight should institute regular ‘news’ email bulletins to remain in contact

with former YPS-users.

o Porchlight should offer small token gestures/prizes for former YPS Users that

complete follow-up social impact surveys.

10.3. Verifying the Report

This report was reviewed pre-assurance in full by Porchlight staff and the Kent County Council

commissioner of Porchlight’s YPS.

This report has been submitted to the SROI Network for independent assurance that it was

developed in compliance with the 7 principles of SROI.


88 P a g e

11. Appendices

11.1. Appendix 1. List of Figures and Tables

Figure 2. Theory of Change of forecast outcomes for YPS Canterbury’s partner organisations. ......... 10

Figure 3. Breakdown of YPS Canterbury forecast social value by major stakeholder group ................ 13

Figure 4. Basic Demographic Profile of the 49 YPS Users (Canterbury) in 2012/13. ............................ 34

Figure 5. Length of stay profile of the 49 YPS Users (Canterbury) in 2012/13. .................................... 35

Figure 6. Inward referral profile of the 49 YPS Users (Canterbury) in 2012/13. .................................. 35

Figure 7. Housing Outcomes for 49 YPS users ...................................................................................... 36

Figure 8. Comparison of YPS Population and survey sample broken down by gender. ....................... 43

Figure 9. Comparison of YPS Population and survey sample broken down by duration of stay. ......... 43

Figure 10. Theory of Change for YPS Users .......................................................................................... 57

Figure 11. Theory of Change for partner organisations and public agencies. ..................................... 74

Figure 12. Breakdown of YPS Canterbury Forecast Social Value by major stakeholder group. ........... 81

Figure 13. Breakdown of YPS Canterbury Forecast Social Costs by major stakeholder group. ............ 83

Figure 12. Theory of Change for YPS Staff ......................................................................................... 101

Table 1. Outcomes in descending order of absolute value. ............................................................... 15

Table 2. Sources of YPS Funding .......................................................................................................... 38

Table 3: Range of hypothesised outcomes of Porchlight for different stakeholders ........................... 39

Table 4: Proposed Key Stakeholder Engagement ................................................................................. 40

Table 5. Excluded Stakeholders ............................................................................................................ 41

Table 6. List of stakeholders and outcomes excluded from SROI analysis. .......................................... 47

Table 7. Summary Collation of potential outcomes for YPS users (Stage 2 data, existing Porchlight

reporting) .............................................................................................................................................. 55

Table 8. List of YPS User Outcomes and indicators. .............................................................................. 60

Table 9. Outcome groupings for purposes of assigning financial Proxies ........................................... 62

Table 10. Summary of monetised inputs for YPS partner organisations. ............................................ 67

Table 11. Summary of short-term outcomes and financial proxies for YPS partner organisations. ... 70

Table 12. (Long-term) Outcomes and Indicators for public agencies. ................................................. 72

Table 13. Breakdown of YPS Canterbury Forecast Social Value by individual stakeholders. .............. 82

Table 14. Breakdown of YPS Canterbury Forecast Social Costs by individual stakeholders. ............... 82

Table 15. List of top 15 outcomes to which the final SROI ratio is most sensitive. ............................. 84

Table 18. List of Indicators for Staff Outcomes .................................................................................. 102

Table 18. Summary of time-based inputs for referral and partner organisations front-line staff .... 105

Table 19. Summary of per client case-load hours for youth in different housing situations. ........... 106

Table 20. Outline of potential limitations of research design and measures to mitigate. ................ 109


89 P a g e

11.2. Appendix 2. The Stages of an SROI

Carrying out an SROI analysis involves six stages:

1. Establishing scope and identifying key stakeholders. It is important to have clear

boundaries about what your SROI analysis will cover, who will be involved in the process and

how.

2. Mapping outcomes. Through engaging with your stakeholders you will develop an impact

map, or theory of change, which shows the relationship between inputs, outputs and

outcomes.

3. Evidencing outcomes and giving them a value. This stage involves finding data to show

whether outcomes have happened and then valuing them.

4. Establishing impact. Having collected evidence on outcomes and monetised them, those

aspects of change that would have happened anyway or are a result of other factors are

eliminated from consideration.

5. Calculating the SROI. This stage involves adding up all the benefits, subtracting any

negatives and comparing the result to the investment. This is also where the sensitivity of

the results can be tested.

6. Reporting, using and embedding. Easily forgotten, this vital last step involves sharing

findings with stakeholders and responding to them, embedding good outcomes processes

and verification of the report.


90 P a g e

11.3. Appendix 3. Stage 2 Interview questions

11.3.1. Script for SROI Stage 2 Interviews – YPS Users

Priming:

Porchlight wants to fully understand the social impact of its Young Persons Service (YPS) in

Canterbury for different people, including its service users. We are conducting a Social Return on

Investment (SROI) study, to understand how much Porchlight delivers on its mission through the

Young Persons Service. Would you be happy to participate in this focus group, to talk about the

impact of Porchlight in your life? We would ask you a series of questions, and I estimate the

interview would require 30 minutes.

Preparation for Focus Groups:

Here are 5 questions to help you prepare:

1. Situation prior to involvement with the YPS: Before you got involved with Porchlight’s YPS in

Canterbury, what were the things that you wanted to change in your life that made coming to

Porchlight part of your plan? What were you doing then?

2. Your expectations: What did you think you would get from participating in the YPS? What were

you expecting?

3. Your input: Other than your time, what did you have to do to be able to participate? Did you or

your family have to give up anything for you to come to YPS? Did it cost you anything in terms of

money?

4. Changes that occurred: What did actually change or happen for you by being with the YPS?

Examples? What do you do differently now that you have been involved with the YPS?

5. How important the change is: What are these changes worth to you? Can you compare these

changes to other things just as important to you?

Interview Questions:

(1) Background? Introduction to this study ... (2) Demographics

a. Names / Ages / Gender b. Which houses stayed at, for how long. c. Where went afterward

(3) How did you find Porchlight’s YPS?


91 P a g e

(4) Before you got involved with the YPS, what were the things that you wanted to change that

made coming to Porchlight part of your plan?

(5) What did you think you would get? Had you any particular expectations as to what you would gain by coming to Porchlight’s Young Persons Service?

(6) Change: What has changed in your life since coming to the YPS? Could you please give us some examples?

a. What happened first? Then what happened?

b. And what did that change allow you to do afterward?

Unintended: Were there any unexpected changes – any surprises?

Negative: Were all the changes positive? Were there any negative changes?

(7) How do you know that the changes have happened? What could you show us to prove that

these changes happened? In your opinion, how should we measure those changes?

(8) What are these changes worth to you? Can you compare these changes to other things just as important to you?

(9) What might have happened had you not been able to coming to YPS? What additional support have you made use of? Tell us about them.

(10) Who else helped you achieve these changes? (11) Are there other people that may have experienced changes because of YPS?

(12) Was there any additional expense or action you had to take to make the most of YPS?

(13) Overall, have your expectations been met by the YPS? If you feel there have not been changes

or that your expectations have not been met, have you any suggestions of what you would like to happen?


92 P a g e

11.3.2. Script for Stage 2 Interviews – YPS staff

Priming:

Porchlight wants to fully understand the social impact of its Young Persons Service (YPS) in

Canterbury for different people, including its service users. We are conducting a Social Return on

Investment (SROI) study, to understand how much Porchlight delivers on its mission through the

Young Persons Service. Would you be happy to participate in this focus group, to talk about the

impact of Porchlight in your life? We would ask you a series of questions, and I estimate the

interview would require 30 minutes.

Preparation for Focus Groups:

Here are 5 questions to help you prepare:

0 Situation prior to involvement with the YPS: Before you got involved with Porchlight’s YPS

in Canterbury, what were the things that you wanted to change in your life that made

coming to Porchlight part of your plan? What were you doing then?

1 Your expectations: What did you think you would get from participating in the YPS? What

were you expecting?

2 Your input: Other than your time, what did you have to do to be able to participate? Did

you or your family have to give up anything for you to come to YPS? Did it cost you anything

in terms of money?

3 Changes that occurred: What did actually change or happen for you by being with the YPS?

Examples? What do you do differently now that you have been involved with the YPS?

4 How important the change is: What are these changes worth to you? Can you compare

these changes to other things just as important to you?

Interview Questions:

(1) Background? Introduction to this study ...

(2) Demographics

a. Names / Gender b. Role c. How long with Porchlight. And the YPS?

(3) How did you come to work at Porchlight’s YPS?

(4) Did you want to work with the YPS? If so, why? If not, why not?


93 P a g e

(5) What did you think you would get from working with the YPS? Did you have any particular expectations?

(6) Input: Was there any additional expense or action you had to take to make the most of YPS? For example, other than your time, what did you have to do to be able to work with the YPS? Did you have to give up anything? Did it cost you anything in terms of money?

(7) Change: What has changed in your life since starting to work with the YPS? Could you please give us some examples?

c. What happened first? Then what happened?

d. And what did that change allow you to do afterward?

Unintended: Were there any unexpected changes – any surprises?

Negative: Were all the changes positive? Were there any negative changes?

(8) How do you know that the changes have happened? What could you show us to prove that

these changes happened? In your opinion, how should we measure those changes?

(9) What are these changes worth to you? Can you compare these changes to other things just as important to you?

(10) What might have happened had you not been able to coming to YPS? What additional support have you made use of?

(11) What else, or who else helped you achieve these changes?

(12) Apart from funding, do you have any thoughts on how the YPS could deliver more for all its different stakeholders?


94 P a g e

11.3.3. Script for Stage 2 Interviews – Partner organisations

Priming:

Thank you for agreeing to take part in our Social Return on Investment (SROI) study of Porchlight’s Young Persons Service (YPS). CAN Impact has been commissioned to evaluate the social impact of Porchlight’s YPS for different stakeholders, including commissioners, partner organisations, and of course the young people themselves. We are using a social return on investment (SROI) methodology to compete this evaluation. The SROI study will allow Porchlight to understand, communicate and hopefully enhance the social value created by its Young Persons Service. In preparation for a 30 minute conversation, please find below 6 main issues/questions will form the basis for our discussions. We would appreciate you considering these prior to our conversation, particularly the outcomes from your partnership with Porchlight, and any tangible evidence that demonstrates this.

1. Why does your organisation engage with Porchlight’s YPS? What are your organisation’s

priorities and projects that make Porchlight a relevant organisation to engage with?

2. Your input: Is there anything you and your organisation have to contribute or ‘give up’ to engage with Porchlight’s YPS? Can you estimate the exact contributions per year, in terms of costs/funding, staff time spent on liaising, coordinating data/information systems, etc?

3. Positive and negative outcomes – for end users: What actually changes or happens for your

end-users as a result of your engagement with Porchlight? What happens differently for your end-users because of your engagement with Porchlight?

4. Positive and negative outcomes – for your organisation: Separate from changes for your end-

users, what actually changes or happens for your organisation as a result of your engagement with Porchlight? What happens differently for your organisation because of your engagement with Porchlight?


95 P a g e

Script

1. Why do you engage with Porchlight’s YPS? What are your organisation’s priorities and projects

that make Porchlight a relevant organisation to engage with? a. What does your organisation do, practically speaking? b. What are your roles? How long been with Organisation? c. Who are your end-users? d. What is the nature of your engagement with Porchlight?

i. Referral orgs: 1. Numbers of referrals per year? 2. Share any information? 3. Joint activities? 4. Etc?

2. Your input: Is there anything your organisation has to contribute or ‘give up’ to engage with

Porchlight? This may be costs/funding, staff time spent on liaising, coordinating data/information systems, etc, or something else

a. Staff time: How many? At what levels? Doing what? b. ££/funding c. Other

3. Positive and negative outcomes – for end users: What actually changes or happens for your end-users as a result of your engagement with Porchlight? What happens differently for your end-users because of your engagement with Porchlight?

a. What happens first, then what happens, and then?

4. Positive and negative outcomes – for your organisation: What actually changes or happens for your organisation as a result of your engagement with Porchlight? What happens differently for your organisation because of your engagement with Porchlight?

a. What happens first, then what happens, and then?

5. How important the outcomes are: What are these changes worth to your organisation and end-users? Do you have any evidence of this worth in financial terms?

a. Rank outcomes in order of importance

i. Users ii. Organisation

b. £ Value of ranked outcomes

i. Users ii. Organisation

6. Involvement of others: How much of these outcomes would your organisation and end-users

experience without engaging with Porchlight? What might have happened had your organisation not engaged with Porchlight? What other organisations help you secure these outcomes


96 P a g e

a. Attribution b. Duration c. Deadweight d. Displacement e. Drop off

f. What evidence do you have for these figures?

7. General

a. Porchlight’ strengths and weaknesses as an organisation (separate from the service) i. What do you think Porchlight does well that no one else does?

ii. What would you suggest that the YPS could improve?

b. Strengths and weaknesses of Porchlight engagement with your organisation i. Any suggestions to improve Porchlight’ engagement with you?

c. Is there anything further you would like to tell me about the impact of Porchlight for your organisation and your end-users?


97 P a g e


11.4.1. Script for SROI Stage 3 – YPS Users

To compare the extent of outcomes on indicators before and after Porchlight’s intervention,

respondents were asked to measure the extent of outcomes ‘before’ and ‘after’ their stay at

Porchlight.

Respondents were asked all relevant indicator questions on the situation ‘before’ they stayed at

Porchlight, and then answered repeat questions on the situation ‘now, ‘

The actual interview script is provided in an attachment.

11.4.2. Script for SROI Stage 3 – YPS Staff

To compare the extent of outcomes on indicators before and after Porchlight’s intervention,

respondents were asked to measure the extent of outcomes ‘before’ and ‘now, since’ working at

Porchlight.

Respondents were asked all relevant indicator questions on the situation ‘before’ they stayed at

Porchlight, and then answered repeat questions on the situation ‘now, ‘

The actual interview script is provided in an attachment.


98 P a g e


The following scripts were used in conversations with stakeholders to engage feedback on theories

of change, final SROI ratios and value created, and individual valuation of outcomes:

Theory of Change

- [Show Visual theory of change]

- In general, does the visual map make sense? Do you understand it?

- About you in particular:

o Which bubbles are true for you?

o Does the map include the important changes for you?

o Anything important changes missing? What?

Social Value Created

- Overall, we estimated that each [user/partner organisation] gets approximately [£££] worth

of value per year for all the different changes they experience as a result of Porchlight.

- [YPS Users]

o YPS Users also estimated the following value for individual changes when asked the

‘maximum they would be willing to pay’ for each outcome

[Provide list]

- Partner Organisations:

o We estimate the following outcome numbers and ££ values for each outcome

experienced by partner organisations

[Provide list]

- Are these £ amounts too high? Too low? About right?


99 P a g e

11.6. Appendix 6. Audit Trail – Staff


The YPS is staffed by 9 full-time employees: one Service Manager, seven Support and Resettlement

Workers (i.e. key workers), and one maintenance worker (to refurbish and maintain YPS

accommodation/houses). These staff are responsible for delivering the YPS and its outcomes for its

users and other stakeholders. Staff typically provide three major inputs:

- Time at work (i.e. full-time work)

- Displaced care-duties

o Each YPS Staff member is required to complete 4 night shifts per month on average.

This displaces some staff members’ care duties for dependents onto their family and

friends.

Amount: 4 night shift per month for 1 of 9 staff members = 40 hours per

month (estimate from survey, n=9)

Financial Proxy: Minimum wage.

- Other consequences of night-shift work, such as disruption to family and social life, are

described and quantified in the outcomes section below.

Staff typically experience one output only, namely:

- Employment with Porchlight’s YPS, typically on a full-time basis.

11.6.2. Outcomes & Theory of Change

Through stakeholder engagement outlined in Section 5.5.4 on page 44 above, staff reported a range

of outcomes and changes in their lives as a result of working with the YPS. It is important to note

that Porchlight has a positive discrimination policy of employing some individuals that may be

unlikely to be employed elsewhere. This includes former users of Porchlight services. Outcomes for

Porchlight staff include:

Immediate Outcomes:

- Access to further training, both provided by Porchlight and as time-off to attend courses

from external providers.

- Higher Income: Above-sector average pay, albeit reduced by unpaid travel expenses &

untaken holiday/TOIL

- Work with young people (with chaotic lives)

- Act as a key worker to assist young homeless people.


100 P a g e


- Gain new skills & work experience, through first post-university position or within a new

sector

- Greater understanding of desired career direction, either through first position post-

university position or within a new sector

- Being better informed about social support systems, by engaging with public services on

behalf of clients.

o This also allows staff to pre-empt obstacles in client work, with outcomes for

clients/users

- Occasional ‘small wins’ with YPS users, by helping them clients materially changing their

personal situation and attitudes in various ways.

- A sense of satisfaction / feeling they are ‘making a difference,’ by helping clients improve

their situation and avoid becoming long-term homeless

- A sense of working within a supportive / close-knit team through the positive work

environment.

Longer-term outcomes:

- Self-affirmation and sense of self-worth, through pursuing work that allows staff to express

who they are and what they believe in.

- Becoming less naive about people and society in general, by seeing what can and cannot be

easily changed regarding homelessness and related issues.

Negative outcomes for staff were that they:

- Become ‘hardened’ and callous/less sensitive to people’s suffering

- Experience increased stress & the ‘occasional downer’ due to the pressures of working with

their clients (YPS users) whose lives cannot be cannot be easily improved

- Needed to undertake shift work and extra work hours for out-of-house activities with clients

- Experience disturbed sleep patterns, due to variable work hours and occasional work-related

stress

- Disrupted family/social lives

- Domestic partners needing to take on a greater emotional and practical burden due to

irregular hours

o Partner take on greater emotional/practical burden (family cost)

o Less time spent with friends

- Less patience and energy with family, due to occasional work-related stress

These outcomes are represented visually within a ‘theory of change’ outlined in Figure 14 below.


101 P a g e

Figure 14. Theory of Change for YPS Staff

11.6.3. Indicators

Table 17 below outlines the indicators used to measure outcomes for staff. For the purpose of this

Forecast SROI, the pre-YPS and post-YPS scores on surveys are estimates only, made by YPS staff

made at the one and same time – i.e., staff record what they believe there score was prior to

working on the YPS, and then at the time they completed the survey.


102 P a g e

Outcome Indicators

Occasional small wins with clients leading to a sense of satisfaction and self-worth that helping others

- Number of staff reporting they experience 'occasional small wins' and number of 'small wins' staff experience on average per month.'- Number of staff reporting change in sense of satisfaction and self worth (5 item survey, 7-point scale)

Gain greater skills/experience leading to better understanding of future career direction

- Number of staff reporting they experienced outcome '- Number of staff reporting change in level of confidence & certainty about career direction (1 item, 7-point scale) '- Number of staff reporting improved future job prospects (2 item survey, 7-point scale)

Feel like a member of team (mitigating some amount of stress)

- Number of staff reporting they experienced outcome '- Number of staff reporting change in extent to which working in a team mitigates work-related stress (1 item survey, 9-point scale)

Perception of salary being above sector average

- Number of staff reporting they experienced outcome - Amount of perceived difference in salary from sector average

Salaries Total take home salary of staff per year

(Negative) Disrupted family/social life

- Number of staff reporting they experienced outcome '- Number of staff reporting change in level of disruption to family and close friends (3 item survey, 5-point scale)

(Negative) Stress & disturbed sleeping patters

- Number of staff reporting they experienced outcome '- Number of staff reporting change in frequency, duration and severity of stress (3 items, 4-7 point scale)

Table 17. List of Indicators for Staff Outcomes


103 P a g e

11.6.4. Financial Proxies

Financial Proxies for YPS staff are as follows:

- Perception of above sector salary

o Average respondent response on the annual £ amount by which their salary is

above/below sector average for their type of role.

- Salaries

o Total take-home salary of YPS Staff, derived from Porchlight data. This is estimated

at: £25,000 p.a.

- All other outcomes

o Average response of staff to stated preference (wiliness to pay) questions. Interview

respondents were asked to state what the outcomes were worth to them, and given

several options on value, including an option to nominate a value themselves.

Please see the Stage 3 User survey (Appendix 11.4.1 on page 97) for further

information.

o For further detail and discussion on the stated preference methodology employed,

please see Section 7.2.5 on page 61.

11.6.5. Impact Factors

All impact factors, with the exception of displacement which is not relevant for the YPS Staff

outcomes and the exception of staff salaries (see below), were derived from surveys. Survey

respondents were asked to estimate the level of impact factors as follows:

1. Duration:

a. Users: < 1 week, 1-2 months, 1 year, 2-3 years, 5+ years

b. Staff: Immediately, 1-2 weeks, 1-2 months, less than half a year, 1 year, 1-2 years,

3+ years

2. Deadweight: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

3. Attribution: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

4. Drop off: None (0%), A little (25%), Some (50%), a lot (75), all (100%)

Exceptional outcomes are:

- Staff Salaries

o Impact factors determined by SROI Working Group

o Deadweight of only 80% (rather than 100%), due to its positive discrimination

policy of employing some individuals that may be unlikely to be employed

elsewhere. This includes former users of Porchlight services.

o Attribution and displacement: 0%


104 P a g e

11.6.6. Analysis

As outlined in the impact map (Appendix 11.9) and materiality audit trail (Section 6.1 on page 47

above), after consideration of impact factors, all outcomes for YPS staff were judged to be non-

material from an SROI perspective. For details on impact factors see Section 8 (SROI Stage 4:

Establishing impact) on page 78 below.

11.7. Appendix 7. Excel Workbook Calculations of YPS User Outcomes & Financial Proxies

[See attached]


105 P a g e

11.8. Appendix 8. Calculations for time inputs & short-term time-savings (Referral organisations).

SROI Element

Item Organisation 1 – KCA (Drug & Alcohol Misuse Charity)




Inputs from partner organisation

Time to make referrals to YPS

1 hour per referred person

15 mins per student x 15 applications. 1 hour x 1-2 students requiring accompaniment to appointments

1 hour per client

1 hour per application

Time to accept referrals from YPS

N/A N/A 1.5 hour assessment per client

2 hours per client

Time to maintain referral arrangements

1 day p.a. for training for YPS staff

–

Time for ongoing liaison with YPS re successfully referred clients

4.5 hours per client (15 mins per week for 18 weeks)

2-3 hours per year x 3-4 students per year dealing with students not abiding by Porchlight rules. Of these, if they leave Porchlight, , approximately 90% will eventually drop out of college.

3 hours per client ( 15mins per week for 12 weeks)

16 hours per client x 10 clients (2 hours per week for 2 months)

Total Input time

Frontline 8 hours fixed + 5.5 hours x 3 clients + 1 hour x 3 clients = 27.5 hours

.25 hrs x 15 + 1 hr x 2 + 3 hrs x 3 = 14.75 hours

1 hr x 10 + 1.5 hr x 40 +3 hrs x 40 = 190 hours

1 hr x 50 + 2 hrs x 10 + 16 hrs x 10 = 230 hours

Table 18. Summary of time-based inputs for referral and partner organisations front-line staff


106 P a g e

Immediate destination of client

Number (& %) of clients in this category

Total case load hours required (social services homelessness team)

Total case load hours required (social services child mental health team)

Return home ~ 1-2 (3%) 0 hours 52 hours (1 hour per week for a year)

Porchlight 6-10 (12-20%) 0 hours 0 hours

Sofa surfing with friends / extended family

33 (66%) 16 hours (2 hours per week for 2 months)

0 hours

Social services Long-term team

~ 3-5 (6%-10%) 16 hours (2 hours per week for 2 months)

Foster care ~ 3 (6%) 16 hours (2 hours per week for 2 months)

208 hours (half a day a week for a year)

Weighted Average (weighted by proportion)

12 hours per client 14 hours per client

Table 19. Summary of per client case-load hours for youth in different housing situations.


107 P a g e

11.9. Appendix 9. Discussion of Stated Preference methodology

The stated preference (willingness to pay/accept) methodology was used as part of stakeholder

engagement for YPS users and staff, to provide initial estimates of the value of different outcomes.

Interview respondents were asked to state what the outcomes were worth to them, and given

several options on value, including an option to nominate a value themselves. Please see the Stage

3 User survey in Appendix 11.4.1 for further information.

The stated preference methodology is a commonly accepted method to measure intangible

outcomes of subjective value (Fujiwara & Campbell, 2011), albeit for large sample sizes.

Discussion of Stated Preference Methodology and limitations

As outlined in (Nicholls, Lawlor, Neitzert, & Goodspeed, 2012) and (Fujiwara & Campbell, 2011),

there are a range of commonly used methodologies to measure non-market goods (such as

outcomes of social interventions). These include:

- Stated preference methods:

o Contingent valuation methods, which focus on the valuation of a non-market good

as a whole; and

o Choice modelling methods, which focus on valuing specific attributes of a non-

market good.

- Revealed preference methods

o Indirect valuation of an outcome by studying people’s real-life behaviour, and

calculating the premium/cost differentials in receiving two goods or services that are

the same in all respects except that one has the outcome of interest and one does

not. For example, the value of clean air could be determined by looking at the

difference in house prices in areas with high/low levels of air pollution. Further sub-

categories of revealed preference valuation methods include:

The ‘travel cost method’ that equates the value of an outcome to the cost of

the extra inconvenience (such as travelling an extra distance) to secure it

over a comparable outcome nearby.

Assessing average household spending on similar activities similar to the

outcome to be valued (be it ‘leisure,’ or ‘health’).

o A new, emerging method of valuation called ‘life satisfaction approach,’ which

“estimates the value of non-market goods by looking at how they impact on

people‘s reported well-being (life satisfaction)” (Fujiwara & Campbell, 2011, p. 10).

In practise, the life satisfaction approach involves measuring the correlation

between a given intervention and changes in personal attributes, such as health or

life satisfaction, that can be equated with increased or reduced income through

large data sets such as the British Household Panel Survey (see (Fujiwara &

Campbell, 2011) for more information).

For this forecast SROI, the Contingent Valuation (Stated Preference Method) was chosen to value

outcomes for YPS Users and their families (but not outcomes for government agencies). Specifically,


108 P a g e

YPS users were asked to nominate the appropriate ‘payment card’ to reflect the value of each

outcome they experienced.

This valuation method was chosen for several reasons:

- The majority of users can be categorised as low-income, so use of average household

spending (e.g. revealed preference) may over-estimate valuations. Furthermore, significant

variations in the extent of each outcome that users experience mean using average

expenditure may not adequately capture the range of value for different users.

- There were no ‘attributes’ of the YPS service that could be varied easily. Rather, different

users experienced different outcomes as a result of using the YPS depending on their

personal circumstances. Accordingly, a choice-modelling method was considered less

appropriate than a stated preference method.

- This Forecast SROI did not measure changes in life satisfaction over time, but only ‘one-off’

retrospectively. Accordingly, the life satisfaction approach was not appropriate, as it

requires accurate measurement of change in life satisfaction over time without

response/recall bias.

The Stated Preference (CV) method as used in this study is nonetheless subject to potential biases

and limitations. Table 20 below outlines some of the major potential limitations and measured

taken in this stage to mitigate them.


109 P a g e

Issue Details Measures taken to mitigate

Anchoring Responses may be biased toward any figures/numbers respondents are primed with.

To minimise over-estimation, respondents were provided with payment cards. Respondents were also given the option of providing alternative valuations. See Appendix 11.4.1for interview script (including payment cards and ‘open question’ of maximum WTP.

Joint effect and sub-additivity effects.

Responses may vary depending on whether the different outcomes are presented together, or sequentially.

None taken. Due to limitations in the survey (it had to be conducted within a short time), all outcomes needed to be presented at the same time for valuation, rather than individually. Therefore, no comparison could be made between ‘joint’ vs. ‘individual’ responses

Lacking information about the good

Responses may be biased if respondents do not understand the good

N/A – valuations only requested for outcomes each user experienced.

Scope insensitivity Responses do not vary with the size/ extent of the outcome.

None. The sample (n=22) was too small to conduct statistical analysis with a high level of confidence to determine whether scope insensitivity existed.

Recall bias There can be significant mismatch between people’s responses at the time of experiencing and outcome and subsequently (e.g. their recollection of how much they valued an outcome in the past changes).

None. The one-off survey for this Forecast SROI did not allow for control of recall bias.

Social desirability Respondents could consciously or unconsciously provided responses seen as socially desirable. This could include exaggerating the value of the service if they believe a higher valuation of outcomes would benefit Porchlight.

None taken.

General validity issues Survey samples must be sufficiently large and representative of the relevant population to ensure validity of results.

The largest possible sample for the survey was taken, resulting in a margin of error of approximately 16%. See Impact map for sample sizes of surveys for each financial proxy.

Table 20. Outline of potential limitations of research design and measures to mitigate.


110 P a g e

As the above table suggests, there are various limitations with the valuation method selected for

user outcomes, some of which were not mitigated in this study. As a result, both the validity and

reliability of the study could be questioned. For this reason, valuations are appropriate only as an

estimate for a Forecast SROI.

Accordingly, it is recommended that in the subsequent, evaluative SROI:

- The life-satisfaction approach or alternative, appropriate methodology is used in

addition to Stated Preference to provide for triangulation of data.

- Longitudinal tracking is used, particularly on objective indicators, to reduce recall bias.

- A significantly larger sample is used for increased robustness (i.e. all YPS users),

particularly to ensure that the ‘scope sensitivity’ issue of ensure the exact extent/size of

each outcome experienced by users is taken into account. In the absence of this

possibility, alternative financial proxies not based on the stated-preference method may

be required.


111 P a g e

11.10. Appendix 10. Social Impact Map


112 P a g e


113 P a g e

11.11. Appendix 11. Glossary

Attribution An assessment of how much of the outcome was caused by the contribution of other organisations or people.

Cost allocation

The allocation of costs or expenditure to activities related to a given programme, product or business.

Deadweight

A measure of the amount of outcome that would have happened even if the activity had not taken place.

Discounting

The process by which future financial costs and benefits are recalculated to present-day values.

Discount rate

The interest rate used to discount future costs and benefits to a present value. Displacement

An assessment of how much of the outcome has displaced other outcomes. Distance travelled

The progress that a beneficiary makes towards an outcome (also called ‘intermediate outcomes’).

Drop-off

The deterioration of an outcome over time. Duration

How long (usually in years) an outcome lasts after the intervention, such as length of time a participant remains in a new job.

Financial value

The financial surplus generated by an organisation in the course of its activities. Financial model

A set of relationships between financial variables that allow the effect of changes to variables to be tested.

Impact

The difference between the outcome for participants, taking into account what would have happened anyway, the contribution of others and the length of time the outcomes last.

Impact Map

A table that captures how an activity makes a difference: that is, how it uses its resources to provide activities that then lead to particular outcomes for different stakeholders.


114 P a g e

Income An organisation’s financial income from sales, donations, contracts or grants.

Resources Inputs

The contributions made by each stakeholder that are necessary for the activity to happen. Materiality

Information is material if its omission has the potential to affect the readers’ or stakeholders’ decisions.

Monetise

To assign a financial value to something. Net present value

The value in today’s currency of money that is expected in the future minus the investment required to generate the activity.

Net social return ratio

Net present value of the impact divided by total investment. Outcome

The changes resulting from an activity. The main types of change from the perspective of stakeholders are unintended (unexpected) and intended (expected), positive and negative change.

Outputs

A way of describing the activity in relation to each stakeholder’s inputs in quantitative terms. Outcome indicator

Well-defined measure of an outcome. Payback period

Time in months or years for the value of the impact to exceed the investment. Proxy

An approximation of value where an exact measure is impossible to obtain. Scope

The activities, timescale, boundaries and type of SROI analysis. Sensitivity analysis

Process by which the sensitivity of an SROI model to changes in different variables is assessed.

Social return ratio

Total present value of the impact divided by total investment. Stakeholders

People, organisations or entities that experience change, whether positive or negative, as a result of the activity that is being analysed.


115 P a g e

11.12. Appendix 12. References

AccountAbility. (2008). AA1000 AccountAbility Principles Standard.

Bagley, A. (2012). Leeds Survivor Led Crisis Service: Full SROI Evaluation Report. Real-Improvement.

Boswell, C., & Skillicorn, D. (2009). Validating the Outcomes Star as a data collection tool. Cardiff:

University of Wales Institute.

Bunn, F., Byrne, G., & Kendall, S. (2009). Telephone consultation and triage: effects on health care

use and patient satisfaction (Review). The Cochrane Library .

Canterbury College. (2013, 03 10). Apply Now & Fees. Retrieved 03 10, 2013, from Canterbury

College: http://www.cant-col.ac.uk/studying-with-us/apply

ComRes. (2013, 02 26). Margin of Error Calculator. Retrieved from Comres.co.uk:

http://www.comres.co.uk/poll-digest/11/margin-of-error-calculator.htm#

Crisis. (2011). Homelessness: A silent killer - A research briefing on mortality amongst homeless

people. London: Crisis.

Deloitte. (2012). Healthcare for the homeless: Homelessness is bad for your health. London: Deloitte

Consulting.

Directgov. (2013, 10 04). The National Minimum Wage Rates. Retrieved 20 04, 2012, from

Direct.gov.uk: https://www.gov.uk/national-minimum-wage-rates

Fujiwara, D., & Campbell, R. (2011). Valuation Techniques for Social Cost Benefit Analysis: Stated

Preference, Revealed Preference and Subjective Well-Being Approaches . HM Treasury.

HM Treasury. (2003). The Green Book: Appraisal and Evaluation in Central Government. London:

TSO.

HMRC. (2012, 10 5). Income Tax rates and allowances. Retrieved 10 5, 2012, from

www.hmrc.gov.uk: http://www.hmrc.gov.uk/rates/it.htm#2

Home.co.uk. (2013, 08 05). Canterbury Market Rent Summary. Retrieved 08 05, 2013, from

http://www.home.co.uk/for_rent/canterbury/current_rents?location=canterbury

Joseph Rountree Foundation. (2000). Single homelessness: An overview of research in Britain. Bristol:

Policy Press.

Kent County Council. (2013, 07 30). Kent Bus Freedom Pass. Retrieved 07 30, 2013, from

http://www.kent.gov.uk/roads_and_transport/getting_around/travel_by_bus/kent_freedom_pass.a

spx

Lawlor, E. (2012). A Social Return on Investment study of Emmaus UK 2011/12. London.


116 P a g e

Liverpool Victoria (LV=). (2013). Cost of a Child from Cradle to College. Bournemouth: Liverpool

Victoria.

MacKeith, J. (2011). The development of the Outcomes Star: a participatory approach to assessment

and outcome measurement. Housing, Care and Support , 14 (3), 98-106.

MacKeith, J., Burns, S., & Graham, K. (2008). The Outcomes Star: Supporting change in homelessness

and related services (2nd Edition). London: Homeless Link.

Matrec. (2013, 08 10). Personal budgeting and money management. Retrieved from

http://www.matrec.org.uk/Personal_budgeting.htm

McGuinness, F. (2013). Youth unemployment statistics - Commons Library Standard Note

(SN/EP/5871). London: UK Parliament.

More than Outputs. (2010). The economic and social return of Action for Children’s Family

Intervention Project, Northamptonshire. London: Action for Children.

New Economy. (2012). Troubled Families Cost Database. Retrieved 03 09, 2013, from

http://www.local.gov.uk/c/document_library/get_file?uuid=05090be1-8721-4264-864f-

307b95214431&groupId=10171

New Policy Institute. (2003). Crisis - How Many, How Much? Single homelessness and the question of

numbers and costs. London: Crisis.

NHS. (2013). Can I get free therapy or counselling. Retrieved 08 10, 2013, from

http://www.nhs.uk/Conditions/stress-anxiety-depression/Pages/free-therapy-or-counselling.aspx

Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2012). A Guide to Social Return on Investment

(2nd Ed). London: The SROI Network.

Oanda.com. (2013, 08 14). Historical exchange rates. Retrieved 08 14, 2013, from Oanda.com:

http://www.oanda.com/currency/historical-rates/

Office for National Statistics. (2012). Annual Survey of Hours and Earnings, 2012 Provisional Results.

London: Office for National Statistics.

ONS. (2012). Family Spending 2012 Edition. London: Office for National Statistics.

Oxford Economics. (2009). Explanation of the SROI calculation for Crisis Skylight Education, training

and employment centres for Homeless People. London: Crisis.

PSSRU. (2011). Unit Costs of Health and Social Care 2011. Canterbury: University of Kent.

Robinson, A., Gyrd-Hansen, D., Bacon, P., Baker, R., Pennington, M., & Donaldson, C. (2013).

Estimating a WTP-based value of a QALY: The ‘chained’ approach. Social Science & Medicine , 92

(September), 92-104.

Shelter. (2009). Responding to youth homelessness following the G v LB Southwark judgement:

Children's Legal Service Briefing. London: Shelter.


117 P a g e

Sport + Recreation Alliance. (2011). Survey of Sports Clubs 2011. London: Sport + Recreation Alliance.

St Mungo's. (2012). First Research Results From the Outcomes Star. London: St Mungo's .

The SROI Network. (2012). A Guide to Social Return on Investment. London: THE SROI Network.

Triangle Consulting. (2013, 02 20). Development of the Outcomes Star (Phase 4). Retrieved from

Outcomesstar.org.uk: http://www.outcomesstar.org.uk/how-was-the-outcomes-star-deve/

Wikipedia. (2013, 02 26). Margin of error. Retrieved 04 15, 2013, from Wikipedia:

http://en.wikipedia.org/wiki/Margin_of_error

porchlight yps canterbury forecast sroi - can invest · porchlight young persons’ service...

Documents