improving children’s social care services - corc · 2017-01-31 · improving children’s social...

IN PARTNERSHIP

WITH Child OutcomesResearch Consortium

Improving Children’s Social

Care ServicesResults of a

feasibility study

Ivana La Valle, Lisa Holmes, Chloe Gill, Rebecca Brown, Di Hart and Matt Barnard

November 2016

Improving Children’s Social Care Services: Results of a feasibility study

Referencing this report

This report should be referenced:

La Valle, I., Holmes, L., Gill, C., Brown, R., Hart, Di., Barnard, M. (2016). Improving Children’s Social Care Services: Results of a feasibility study. London: CAMHS Press.

ISBN: 978-0-9933436-3-6© CAMHSPress 2016

Contents

Acknowledgements 4Key findings 5 Executive summary 6

Background 6 Method 6

Rapid evidence review 6 Analysis of Ofsted and DfE data 6

Results 7Findings from the literature review 7 Findings from the analysis of DfE data and Ofsted ratings 7Conclusion 8

Main report 9Chapter 1: Introduction 10

1.1 Background 101.2 Aims and objectives 11

Chapter 2: Method 12 2.1 Rapid evidence review 122.2 Analysis of DfE data and Ofsted ratings 12

Chapter 3: Findings from the literature review 14

3.1 Introduction 143.2 Expected outcomes 14 3.3 The right services of the right quality 17 3.4 Social work models 18 3.5 Fit-for-purpose organisation 18 3.6 Discussion 19

Chapter 4: Findings from the analysis of DfE data and Ofsted Ratings 20 4.1 Percentile analysis 204.2 Regression analysis 22 4.3 Discussion 22

Chapter 5: Conclusion 23 References 25 Appendix 1: Summary of literature reviewed 29 Appendix 2: Department of Education variables 42

About the funderThe Nuffield Foundation is an endowed charitable trust that aims to improve social well-being in the widest sense. It funds research and innovation in education and social policy and also works to build capacity in education, science and social science research. The Nuffield Foundation has funded this project, but the views expressed are those of the authors and not necessarily those of the Foundation. More information is available at: www.nuffieldfoundation.org

ACKNOWLEDGEMENTS

The authors would like to thank all those who helped make the feasibility study possible. In particular, the authors would like to thank the Nuffield Foundation for funding the study and for providing support and guidance, in particular Teresa Williams, Ruth Maisey, Alison Rees and Tracey Budd. The authors would also like to thank colleagues from the Department for Education, in particularly Richard White, for their support and input and those from the Rees Centre for Research in Fostering and Education at the University of Oxford for sharing insights from the evaluations of the Innovation Programme. Finally, the authors would like to thank all those who attended the seminar where the results of the study were presented and who contributed their invaluable thoughts and knowledge.

KEY FINDINGS

This report sets out the findings of a feasibility study that aimed to understand how to define ‘good’ children’s social care services and how to assess whether they have improved. The intention was for the feasibility study to support a larger project exploring the processes involved in improving children’s social care services. The key findings of the study are:

• There is a lack of consistent expectations about outcomes for children’s social care services and what indicators should be used to monitor them. There is mixed evidence about the features that characterise good children’s social care services, and a significant proportion of it is based on expert opinion and has not been tested quantitatively.

• Analysis of the relationship between outcome data for children in need collated nationally by the Department for Education and Ofsted ratings of children’s services found very little association. There did not seem to be any pattern in terms of the local authorities that were in the top or bottom percentiles for child outcome. Only one child outcome variable and one workforce variable had a statistically significant relationship with the Ofsted ratings.

• Before proceeding with a study exploring how children’s social care services improve, it is important to identify an outcomes framework that a wide range of stakeholders agree is appropriate and to establish a set of indicators that reflect these outcomes and that could be collected and collated at a national level.

• The study team concluded that at this time it is not feasible to go ahead with the main project as originally envisaged, and instead to focus on how to create an outcomes framework and establish an appropriate set of indicators.

EXECUTIVE SUMMARY

BackgroundThis report sets out the findings of a feasibility study that aimed to understand how to define ‘good’ children’s social care services so that we could then select a sample of children’s social care services for a study exploring how high quality in service provision can be achieved and sustained.

Local authorities’ ability to safeguard and promote the welfare of children in need has been the subject of considerable comment over recent years. Every time a child is seriously abused or neglected, the spotlight falls on the services that are deemed to have failed them, followed by assertions that lessons have been learned and radical changes made to bring about improvement. Yet the regular analyses of Serious Case Reviews repeatedly identify the same weaknesses in practice where children have died or been seriously injured as a result of abuse or neglect.

In spite of the focus on improving practice and an apparent consistency in the problems that need to be addressed, some authorities appear to have become trapped in a cycle of failure according to the findings of Ofsted inspections. It appears that an understanding of the elements of success does not necessarily translate into the ability to implement them in practice. This raises the crucial question of how those services that have improved have done so and why others appear to have failed. In other words, what were the processes and approaches that were used in order to successfully implement changes in practice, structure and culture? This was the question that was at the heart of the main project that this feasibility study was designed to support.

However, the way in which a good service is defined and measured is contentious. Within this context it was clear that in order to study how children’s social care services improve, it was first necessary to define what was meant by ‘good’ children’s social care and to establish a robust approach to identifying services that have ‘improved’. This feasibility study was intended to address those two issues.

MethodRapid evidence reviewThe aims of the rapid evidence review were to explore existing evidence on what effective children’s social care looks like in terms of outcomes for children and their families, what indicators can be used to monitor these outcomes, how good/effective social work practice is defined and measured, and what factors support (or undermine) effective/good social work practice. A full systematic review was beyond the scope of the feasibility study: instead a pragmatic approach was used to identify the most relevant evidence, which was then summarised using a standard template.

Analysis of Ofsted and DfE data The Department for Education’s data on outcomes for children in need and the children’s social care workforce were compared with Ofsted ratings for children’s services1

to explore whether there are relationships between the judgements made by Ofsted and indicators for children’s outcomes and workforce stability. The aim was to assess whether Ofsted ratings could be used to purposively select authorities with higher and lower quality of services for further study.

To conduct the analysis, the team selected 11 child outcome variables from DFE’s Looked after Children in England dataset and the Characteristics of Children in Need dataset. A further three workforce variables were selected from DFE’s Children’s Social Work Workforce dataset. Data analysis comprised two phases: the first involving the identification of the ‘highest’ performing and ‘lowest’ performing local authorities according to each of the selected variables and then looking to see whether authorities appeared consistently in each group; the second an analysis of the associations between Ofsted ratings and child outcome and workforce variables.

1 Based on the single inspection framework (SIF) for local authority children’s social care services introduced in 2013

Results

Findings from the literature reviewIn order to structure the literature review we developed a framework including four dimensions that need to be explored to understand what good children’s social care looks like and what factors influence service quality and ffectiveness.

A shared understanding of expected outcomes for service users: Policy documents describe outcomes for children in need in very broad terms but do not describe in detail what improvements in outcomes are expected from intervention and how these should be measured. The research evidence provides more detailed descriptions of outcomes for social care users. However, these focus mainly on narrowly defined experiences of services and narrowly defined outcomes, and tend to focus on looked after children, rather than all children in need. More comprehensive assessments of outcomes were found for specific programmes, but there was little evidence that these broader outcomes are being used to monitor the effectiveness of the range of support children’s social care services provide.

The right services of the right quality: There is a considerable amount of evidence relating to how children’s social care services can deliver the right services of the right quality. In relation to the right services, the effectiveness of the delivery and management of the front door service is seen as crucial. There is also a great deal of evidence on the factors affecting the quality of an assessment. However, much of the evidence is based on professional and expert opinions and is not often validated with data linking particular features of assessments with data on outcomes for service users, and is largely qualitative. The research literature also abounds with narrative descriptions of what good social work practice should look like, but there is a scarcity of quantitative indicators that have been used to systematically and ‘objectively’ measure quality. A similar picture emerged when looking at the literature on what underpins effective and good quality social work practice. The quality of supervision and the need for transparency and scrutiny were consistently highlighted, though they were not supported by a strong evidence base.

Social work models: A number of social work models are being developed and tested, for example, through the DfE’s Children’s Social Care Innovation Programme, but they are mostly at an early stage and evaluation findings were not available at the time the feasibility study was carried out. Findings of programme evaluations that have been published are not conclusive in terms of the impacts on outcomes for users.

Fit-for-purpose organisation: There is a great deal of consistency in the messages from the literature about the kind of organisational features that influence the quality

of practice and the effectiveness of service delivery, that is: workforce stability and engagement, leadership, inter-agency working, organisational culture, and IT systems. However, there is little consistency on how these features should be measured and limited robust evidence that these are the ‘right’ features to monitor.

Findings from the analysis of DfE data and Ofsted ratingsThe analysis aimed to explore whether Ofsted Single Inspection Framework (SIF) ratings provide a useful basis on which to sample Local Authorities. In order to assess this we compared local authority Ofsted ratings with data from DfE on outcomes for children in need and workforce stability variables (the latter have been associated with effective service provision). An association was expected between Ofsted ratings and outcomes for children in need and workforce (in)stability. Furthermore, one would expect some consistency in the results relating to child outcomes and workforce stability: for example an authority would be expected to being doing well (or badly) in all/most child and workforce variables being explored.

The analysis of the DfE data shows that out of the eleven outcome variables:

• 93 (61%) local authorities ranked in the top 10% for at least one outcome indicator

• 59 (39%) authorities did not rank in the top 10% in any of the outcome indicators

• 6 local authorities ranked in the top 10% in four or five indicators

• there were no local authorities that ranked in the top 10% for more than five indicators

• 63 (41%) local authorities did not rank in the bottom 10% for any of the child outcome indicators

• 89 (59%) ranked in the bottom 10% for at least one indicator

• 6 local authorities ranked in the bottom10% in four indicators

• no local authority ranked in the bottom10% in more than four indicators.

We carried out a regression analysis, which showed that only one child outcome variable and one workforce variable had statistically significant relationships with Ofsted ratings, but the associations were weak.

It is important to bear in mind that we used Ofsted ratings generated after the new single assessment framework was introduced, and a third of local authorities were not included in the analysis comparing the DfE data with Ofsted rating because they had not been inspected under this new framework at the time of the study. Nevertheless, these results raise a number of questions about the robustness of the available national data on

ConclusionThe ultimate aim of the feasibility study was to assess if there was sufficiently robust national evidence on the quality of children’s services and their capacity to improve (or lack of it), and a framework that could be used to assess under what circumstances children’s services can make a positive difference to the lives of children in need and their families.

The different sources of evidence we used and a consultation with experts and policy makers showed that:

• There is currently no robust national data that could be used to select a sample of local authorities based on the quality of their children’s services (and how this may have changed over time).

• There is not a consistent and robustly tested framework for monitoring outcomes for service users. While there is agreement about broadly defined outcomes for children in need, there is not sufficient evidence on the range of indicators that should be used (e.g. intermediate and longer term; chid level and proxy) to monitor outcomes following

intervention.

• There is a lack of robust evidence on what can make a difference, that is: what service level inputs and outputs are associated with positive outcomes from intervention and how causal relationships can be established between the characteristics of a service and outcomes for children in need and their families.

The clear recommendation from the consultation with experts was that it was crucial to identify an outcomes framework that a wide range of stakeholders agreed was appropriate and to establish a set of indicators that reflected these outcomes and that could be collected and collated at a national level. They said that this should be done before proceeding with a study exploring how children’s social care services improve. The study team therefore concluded that at this time it was not feasible to go ahead with the study as originally envisaged, and instead to focus on how to create an outcomes framework and establish an appropriate set of indicators.

child outcomes and/or Ofsted ratings. An exploration of the factors that may undermine the quality of these data sources was beyond the scope of this feasibility study, but these findings were discussed at a seminar attended by experts and government policy officials. The results could also indicate the need to consider contextual variables that could help to explain the lack of associations one would have expected between data on outcomes from

services and the quality of these services. However, as consistent data on these contextual local factors is not available, in consultation with experts, we concluded that it is not possible to use the existing national data to sample authorities on the basis of the quality of their children’s service and their capacity (or lack of it) to improve as we planned to do for the main study.

MAIN REPORT

This report sets out the findings of a feasibility study that aimed to understand how to define ‘good’ children’s social care services and how to assess whether they have improved. This first chapter describes the background to the study along with its aims and objectives.

1.1 BackgroundLocal authorities’ ability to safeguard and promote the welfare of children in need has been the subject of considerable comment over recent years. Every time a child is seriously abused or neglected, the spotlight falls on the services that are deemed to have failed them, followed by assertions that lessons have been learned and radical changes made to bring about improvement. Yet the regular analyses of Serious Case Reviews (Brandon et al, 2010; Brandon et al, 2012; DfE, 2014a) repeatedly identify the same weaknesses in practice where children have died or been seriously injured as a result of abuse or neglect, such as a failure to share information, poor quality assessments and drift2. This analysis chimes with the problems identified in the Munro review (Munro, 2011). The review particularly highlighted the issue of an excess of bureaucracy, which meant that practitioners and managers found it difficult to remain child centred and which also undermined the degree to which social workers could develop their own knowledge and expertise (Munro, 2011).

In parallel with the analysis of serious case reviews and Munro’s investigation, Ofsted has analysed the results of individual inspections in an attempt to describe the

features of local authorities that are judged to provide a ‘good’ or ‘outstanding’ service (Ofsted 2013, 2014). This analysis indicates that authorities are more likely to serve their children well if they: offer effective early help; engage well with children; focus on direct work with children (rather than form-filling); and have manageable caseloads and good supervision.

Yet, in spite of the focus on improving practice and an apparent consistency in the problems that need to be addressed, some authorities appear to have become trapped in a cycle of failure. Following an Ofsted inspection that raises concerns about their ability to keep children safe, there may be a period of intense activity, including bringing in new managers and creating new systems and processes, but without any apparent sustained improvement (e.g. Le Grand, 2014). It appears that an understanding of the elements of success does not necessarily translate into the ability to implement them in practice. This raises the crucial question of how those services that have improved have done so and why others have failed. In other words, what were the processes and approaches that were used in order to successfully implement changes in practice, structure and culture? This was the question that was at the heart of the main project that this feasibility study was designed to support.

However, the way in which a good service is defined and measured is contentious. The current systems for measuring and inspecting children’s social care services

Chapter 1

Introduction

2 The Government is proposing to replace serious case reviews with a new national learning framework in response to the Wood review of LSCBs

have been blamed for creating perverse incentives, for relying on inaccurate data or missing the true nature of provision. In addition, inspection has been described as a blocker to developing child-focused services and high quality front-line practice while impacting negatively on staff morale and retention (Gibson and O’Donovan, 2013; Searle and Patent, 2013; Clarkson et al, 2009). Areas with higher levels of need have also been found to receive lower inspection ratings (Clarkson et al, 2009), suggesting that these authorities have a harder job to do. These arguments give weight to the call by ADCS, LGA and Solace to apply a new, differentiated inspection model of children’s social care that genuinely involves all agencies (ADCS, 2015).

Within this context it was clear that in order to study how children’s social care services improve, it was first necessary to define what was meant by ‘good’ children’s social care and to establish a robust approach to identifying services that have ‘improved’. This feasibility study was intended to address those two issues.

1.2 Aims and objectivesThe overarching aim of the feasibility study was to develop the methodology for a study which addressed four key questions in relation to children’s social care in England:

• What can be done to prevent a local authority’s children’s social services from failing?

• How can failing children’s social services be turned around?

• How can service improvement, including effective prevention, be embedded and sustained?

• What are the costs, economic benefits and potential costs avoided of different ways of improving children’s social care and sustaining improvement over time?

The key original objectives of the feasibility study were to:

1. review the relevant literature on children’s social care and organisational change to ensure the main study builds on the existing body of evidence of what works in delivering effective children’s social care and how service improvement can be achieved and embedded

2. provide a critical review of current national measures used to assess the effectiveness of children’s social care, identify gaps and suggest how the national statistical evidence could be improved

3. using the evidence from 1 and 2 to develop a typology of local authorities based on their children’s social care performance and capacity to improve (or lack of it), and then use this typology to select a sample of authorities to be included in the main study

4. test the feasibility of collecting from local authorities the data required to answer the research questions outlined above

5. obtain an agreement, in principle, from the sampled authorities to take part in the main study.

Having completed objectives 1 and 2, it became apparent that the evidence base was not sufficient to develop a typology of authorities based on their performance for the reasons explained later on, so in consultation with a range of experts we revised both the focus and the methodology for the main study. This will focus on establishing a theory of change for children’s social care services, developing an outcomes framework and investigating the range of data there is or could be collected to use as indicators within the framework.

This chapter describes the methodology for the study, which included a rapid evidence review and analysis of Department for Education (DfE) social care statistics and Ofsted SIF ratings.

2.1 Rapid evidence reviewThe aim of the review was to develop definitions of key concepts and to identify ways of operationalising them, including:

• effective children’s social care: what does it look like in terms of children and young people’s outcomes and what indicators can be used to monitor these outcomes?

• key ingredients of effective children’s social care: what activities and processes are associated with achieving and maintaining an effective children’s social care service and how these activities and processes can be effectively implemented and embedded?

A full systematic review was beyond the scope of the feasibility study: instead a pragmatic approach was used (see Box 1) to identify the most relevant and useful evidence, which was then summarised using a standard template.

2.2 Analysis of DfE data and Ofsted ratings This element of the feasibility study comprised secondary analysis of the DfE data on child outcomes and children’s social care workforce. These data were compared with Ofsted SIF ratings to explore whether there are relationships between the judgements made by Ofsted and

children’s outcomes and workforce stability. The aim was to assess whether we could find an association between Ofsted ratings and data monitored by DfE and they could therefore be used to purposively select authorities with higher and lower quality of services for the main study.

The choice of variables for the analysis was guided by methodological considerations (e.g. variables with a high proportion of missing cases were excluded) as well as the literature review, in particular a number of recent studies that have explored outcomes for children in need. The latter included Bywaters and colleagues (2016) who explored the relationship between poverty, child abuse and neglect, which concluded that inadequate attention is given to developing an evidence base for quantifying and understanding the relationship between poverty and child abuse and neglect in the UK; Sebba and colleagues (2015) who explored the educational outcomes of looked-after children, including through a detailed secondary analysis of the SSDA 903 statistical return for Looked after Children and the National Pupil Database. In addition, the team was grateful to have access to an unpublished study by Rick Hood3 that used publicly available data to explore national trends in intervention rates, variations between local authorities, the effects of local characteristics such as deprivation, and the connection between these variables and Ofsted ratings.

Eleven child outcome variables were selected from DFE’s Looked after Children in England including adoption (year ending 31st March 2015) dataset and the Characteristics of Children in Need (year ending March 31st 2015) dataset. A further three workforce variables were identified from DFE’s Children’s Social Work Workforce (year ending 30th September 2015) dataset. As described above the

Chapter 2

Method

3 The full study has not yet been published, however access was provided to a research summary.

Rapid review methodology • The NSPCC Library and Information service

undertook a comprehensive search of their library catalogue. The library is the only UK database that specialises in published material on child protection and related subjects. Information specialists catalogue articles published in a range of academic journals, books and grey literature (http://www.nspcc.org.uk/library).

• Key words used for the search were:

− Social Policy

− Children’s services, social services, child protection services, public sector

− Evaluation, commissioning, codes of practice, performance evaluation, performance indicators

− Change, innovation, organisational management, organisational behaviour, organisational change

• The search covered the 2005-2015 period, but we summarised primarily evidence from 2010-15, older evidence was included in the review when it seemed particularly relevant as more up to date evidence was not available

• Similarly, while the search covered the international (English language) literature, we summarised primarily the English evidence drawing on the international literature when it seemed particularly relevant to the English context.

• A total of 57 documents were summarised and included in the review.

selection of variables involved an iterative process and was informed by the work of others. The key criterion for selection was that the variables measured child outcomes rather than social work processes, and that the selected workforce variables were indicative of workforce stability, which was hypothesised to be closely related to children’s outcomes and the least ambiguous in terms of ‘good’ or ‘bad’ outcomes. The variables that were eventually included are listed below.

Child outcome indicators:

• Return home from care

• Care leavers in education, employment or training at age 21

• Care leavers in suitable accommodation at age 21

• Missing from care

• Absences from care

• Offending of looked after children

• Substance misuse of looked after children

• Emotional and behavioural health of looked after children (this included two variables)

• Referrals within the past 12 months of a

previous referral

• Repeat children protection plans

Workforce indicators

• Children in Need per social worker

• Social worker turnover rate

• Agency worker rate

A full explanation of the variables included in the analysis is included in appendix two.

The data analysis comprised two phases:

Phase One: this involved the identification of the ‘highest’ performing and ‘lowest’ performing local authorities (the top and bottom 10% respectively) according to each of the indicators and then looking to see whether authorities appeared consistently in each group.

Phase Two: this involved an analysis of the association between Ofsted ratings and each child outcome and workforce indicator and workforce (using regression analysis).

http://www.nspcc.org.uk/library

This chapter sets out the findings from the literature as grouped under the four areas used in the theoretical framework described in the methodology.

3.1 Introduction The literature review was guided by a framework developed by the research team (Figure 3.1), which indicates the different elements of the system that are needed to understand what good/effective children’s social care looks like and the factors influencing service quality and effectiveness:

1. Expected outcomes: first, we need to understand what outcomes are expected for children and families supported by children’s services. Given that a substantial proportion of public money is invested in children’s social care (relative to the number of ‘service users’ defined as children in need), one would expect: 1)national policy and local services to clearly spell out what kind of improvements in outcomes are expected in line with an overarching framework for children’s policy; 2)some evidence of whether these outcomes are being achieved (or not), which in turn requires a degree of consensus on what kind of indicators should be measured.

2. Right services of the right quality: in order to be effective, national policy levers (e.g. funding, guidance, regulations, ‘nudge’ messaging, specific programmes) must be based on an understanding and robust evidence of how one can ensure that ‘the right services of the right quality’ are delivered to children in need (and achieve the expected outcomes). One would therefore expect a degree of consensus and evidence on: consistent processes for identifying children in need; the kind of support children in need in different circumstances require from a range of

Chapter 3

Findings from the literature review

agencies (e.g. education, health as well as children’s social care); and, what kind of service quality is required to achieve the expected outcomes for service users.

3. Effective social work models: like most professions, the social work profession has developed a range of models and approaches. An understanding of which models work, for whom and how is important to guide choices about the organisation of local services and to some extent national policy decisions as well, as through various funding streams DfE supports the development of different social work approaches.

4. Fit-for-purpose organisation: an understanding of what organisational features are more likely to support (or undermine) the quality of practice and effective service delivery would seem important to guide local decisions.

In the rest of the section we discuss what evidence we found on these different topics focusing in particular on quantitative indicators that can be used to monitor: changes in outcomes for service users, service quality and effectiveness, the impacts of different social work models on users, and organisational features associated with high quality and effective services. While qualitative research can play a very important role in mapping what needs to be measured and in helping to explain and refine quantitative evidence, robust and consistent quantitative indicators are needed to conclusively answer the questions of what works and what does not work.

3.2 Expected outcomes In reviewing the evidence on expected outcomes for children social care users, we looked at recent

Figure 3.1 Good children’s social care: what we need to know and what we know

policy documents and data produced by government departments and agencies, as well as research evidence.

The policy documents described outcomes for children in need in very broad terms. For example, the revised Working Together to Safeguard Children Framework (HM Government 2015) describes the universal dimensions of healthy child development, parenting capacity and wider family and community support. In relation to children in need specifically, it talks about protecting children from maltreatment, preventing impairment in children’s health and development, ensuring children have safe and effective care and achieve the best outcomes. Similarly, the Children and Social Work Bill (2016) says local authorities’ corporate parent responsibility means they must: ‘…promote high aspirations, and seek to secure the best outcomes, for those children and young people; … for those children and young people to be safe, and for stability in their home lives, relationships and education or work; … prepare those children and young people for adulthood and independent living.’ The recent DfE document Children’s Social Care Reform: A vision for Change (DfE 2016a) focuses on improvements in: people and leadership, practice and systems, governance and accountability, but does not detail what improvements in outcomes should result from

this vision beyond ‘transforming’ the lives of vulnerable children. Similarly, the most recent report on DfE’s Children’s Social Care Innovation Programme (DfE 2016b) focuses on expectations relating to practice and service improvements and does not specify expected outcomes, although we understand that an outcomes framework has been compiled reflecting expected improvements for children and their families. While the theoretical basis for children in need policy, the Assessment Framework triangle developed in 2002, did clearly set out the outcome domains summarised below (Department of Health 2002). However, there is little discussion of how these outcome domains could be further developed and operationalised, how policy levers are meant to support these different outcome domains, or how the outcomes are meant to be monitored.

As discussed in chapter 4, DfE regularly publishes statistics on children’s social care. However, only a limited number of those relate directly to outcomes and the findings discussed later raise the question of whether the right indicators are being used to monitor outcomes for children in need and their families. A key aim of inspections of children’s social care carried out by Ofsted is to explore children in need’s experiences and their progress. However, progress is very broadly defined: for example,

Indicators of expectedoutcomes for service users

•Reflecng naonal priories and baxed on research evidence

Right services of the right quality

•Observable and measurable features ofquality linked to outcomes for users

•Proxy measure correlated to observable measures

Efecve social work models

•Tested models with evidence of impacteson users

Fit-for-purposeorganisaon

•Organisaonal features linked to highquality pracce and effecve service delivery

one of the criteria for awarding an outstanding rating is that services deliver: ‘…measurably improved outcomes. For some children and families, their progress exceeds expectations’ (Ofsted 2015). These outcomes are not described in the guidance and while examples of more specific outcomes are provided in inspection reports of individual authorities, these are not operationalised and measured in a consistent way across different local uthorities.

The absence of detailed expectations of what children’s social care is for has been recently highlighted by Forrester, who has argued that there is an absence of ‘…a vision for what children’s services should be striving to achieve…the ultimate aims we might strive for...’ (quoted in Solomon ed 2016:11). He goes on to suggest that this has undermined repeated government efforts to improve children’s social care.

The research evidence on children’s social care provides more detailed descriptions of outcomes for children’s social care users. However:

Assessment Framework

Family and Environmental Factors

Safeguardand

Promoting Welfare

Parenting Capacity

Child

’s D

evel

omen

tal N

eeds

CHILD

Family’s S

ocial

Integratio

n

Family H

istory

and F

unctioning

Wid

er Family

Ho

using

Em

ploym

ent

Incom

e

Co

mm

unity R

esources

Basic Care

Ensuring Safety

Emotional Warmth

Stimulation

Guidance and Boundaries

Stability

Education

Emotional and Behavioural Development

Identity

Family and Social Relationships

Social Presentation

Selfcare Skills

Health

• the most commonly explored outcomes seem to focus mainly on narrowly defined experiences of services and narrowly defined outcomes, with the latter reflecting a balance towards negative expectations (e.g. drug use, involvement with the criminal justice system) rather than the broader range of outcomes more typically explored when assessing children’s development, and described in the Assessment Framework (Department of Health 2002);

• the more comprehensive assessments of outcomes were found for specific programmes, such as Multi-Systemic Therapy (MST) and Enhanced Foster Care, and as far as we know no attempt has been made to collect and link these outcomes to a wider range of services provided by children’s social care;

• outcome measures tend to focus on looked after children, with more limited evidence on other children in need and measures to monitor parents’ capacity to care for their children.

The research on outcomes for children in need that was reviewed for this study is outlined in Table 1 in appendix one. A summary of the literature is included below.

• Commonly monitored aspects of services appear to be placement stability (Brodie 2009; Dickson et al 2009; Sebba et al 2015; Stein 2009) and child protection referrals and re-referrals (ADCS 2015; Fauth et al 2010) with a focus on the number of placements/re-referrals. These measures are also part of the DfE statistics discussed later. A recent study has also found associations between the age of entry into care, length of time spent in care and school instability (i.e. frequent school changes and absences) and educational outcomes (Sebba et al 2015). Other quantitative measures of experiences of children’s services seem far less common. They were only reported in one piece of research (Brodie, 2009), where it was found that children’s satisfaction with a placement was associated with better educational outcomes, and in one review of effective child protection practice (Fauth et al 2009), which found that a good relationship between the social worker and parent was associated with positive child outcomes.

• Key national educational statistics (e.g. key stage and GCSE results, SEN status, attendance, exclusion) for looked after children are reported in the literature (ADCS 2015; Brodie 2009; Seeba at al 2015) and some are included in the DfE looked after children statistics discussed later on. Educational statistics for children in need (who are not looked after) seem less common but were reported in a recent comprehensive study on educational outcomes (Sebba et al 2015). However, quantitative data on children’s views of their education experiences has been identified as a gap in the literature (Brodie 2009). Furthermore, looked after children’s involvement in extracurricular activities does not seem to be commonly monitored (Brodie 2009). In contrast, involvement in extracurricular activities, views of educational experiences and aspirations for the future are commonly measured by child development studies such as the birth cohort studies and the DfE longitudinal survey of young people.

• A recent report on looked after children from ADCS (2015) provides statistical evidence on teenage pregnancies and offending behaviour, and the latter was also a commonly monitored outcome in a review of MST, Enhanced Foster Care and mentoring schemes (Dickson et al 2009), and is included in the DfE looked after children statistics.

• The ADCS (2015) report has also explored looked

after children’s emotional and behavioural health, based on data collected using the Strengths and Difficulties Questionnaire. This was also an outcome assessed by some of the interventions included in an international review of effective child protection practice (Fauth et al 2010), evaluations of specific looked after children programmes (Dickson et al 2009) and a recent study on the educational performance of looked after children (Sebba et al 2015).

• A wider range of outcomes for looked after children was only found in an international review of evaluations of specific programmes i.e. MST, Enhanced Foster Care and mentoring schemes (Dickson et al 2009). Outcomes reported in the review included: self-harming/violent behaviour; substance misuse; self-esteem and feeling valued; community engagement; family and peer relationships. DfE looked after children statistics also include a measure of substance misuse, but a high proportion of LAs do not provide this information. A gap even among studies using a broad range of outcome indicators seems to be physical health (apart from substance abuse), which is again an outcome domain typically assessed by child development studies and identified in the Assessment Framework (Department of Health 2002).

• While parenting capacity is a key outcome domain identified in the Assessment Framework (Department of Health 2002), quantitative measures of parents’ experiences and outcomes were only found in an international review of effective child protection practice (Fauth et al 2010). Parent outcomes reported in this review included: physical and emotional care of the child; discipline and parental coping; level of service use (with high service use associated with positive child outcomes); and, as mentioned earlier, the parent’s relationship with the social worker.

3.3 The right services of the right quality There is a considerable amount of evidence relating to how children’s social care services can deliver the right services of the right quality (see table 2 in appendix one). In relation to the right services, the effectiveness of the delivery and management of the front door service (i.e. assessments, case load management) is seen as crucial (ISOS 2016; Le Grand et al 2013, Stein 2009), yet the most recent Ofsted report on children’s social care found considerable weaknesses in many local authorities’ assessment processes (Ofsted 2015b). There is also a great deal of evidence on the factors affecting the quality of an assessment (Brandon et al 2010, Fauth et al 2010; Kirman and Melrose 2014, Ofsted 2015c). However,

much of the evidence is based on professional and expert opinions and is not often validated with data linking particular features of assessments with data on outcomes for service users, and it is largely qualitative.

The research literature abounds with narrative descriptions of what good social work practice should look like, but there is a scarcity of quantitative indicators that have been used to systematically and ‘objectively’ measure quality even for fundamental aspects of practice highlighted by most of the literature, namely adopting a child-centred and evidence-based approach (e.g. Brandon et al 2010, Fauth et al 2010, Munro 2011, Stein 2009). No measures were found that have been systematically used to monitor child-centred practice, nor any used to measure user engagement, which is also considered an important aspects of service quality. In relation to evidence-based practice, much of the literature highlights insufficient use of evidence in social work practice (Dettlaff 2015, Fauth et al 2010, ISOS 2016, Kirman and Melrose 2014). The literature also notes the use of the ‘wrong’ evidence, such as indicators that encourage a focus on processes rather than quality (Stein 2009) and those that can negatively affect the development of social work expertise, learning from evidence and the application of professional judgement (Munro 2011). However, there was no consensus on how evidence-based practice can be defined, operationalised and systematically measured.

A similar picture emerged when looking at the literature on what underpins effective and good quality social work practice. Two key features were consistently highlighted, though they were not supported by a strong evidence base:

• The quality of supervision is seen as crucial to support good practice (Brandon et al 2010, Fauth et al 2010, ISOS 2016, Ofsted 2015, Singleton 2010), but there is limited evidence on what constitutes effective supervision, how it can be measured, and through which mechanisms effective supervision is linked to positive outcomes for users.

• The need for transparency and scrutiny through quality assurance and auditing mechanisms is also often mentioned (Fish 2009, Easton et al 2013, ISOS 2016), but again there is a lack of systematic evidence on mechanisms for achieving transparency and scrutiny, and how these are linked to positive outcomes for users.

A final consideration in relation to how to monitor whether the right services of the right quality are being delivered is that the focus is almost exclusively on children’s social care services, yet many of the outcomes they are trying to achieve (e.g. education, family functioning) are crucially dependent on other parts of the system (e.g. early years,

education, health) delivering appropriate services to meet the needs of children in need and their families.

3.4 Social work models A number of social work models are being developed and tested, for example, through the DfE’s Children’s Social Care Innovation Programme (DfE 2016b), but they are mostly at an early stage and evaluation findings were not available when the feasibility study was carried out (see table 3 in appendix one). Findings of programme evaluations that have been published (e.g. Reclaiming Social Work, Social Work Practices, Vanguard Method ) are not conclusive in terms of the impacts on outcomes for users (Forrester et al 2013, Gibson and O’Donovan 2014, Stanley et al 2012). While it may be too early to assess the impacts of some of these programmes, the methodologies used to evaluate them may not produce definitive evidence of impacts on users even at a later stage. Evaluations of more established models, such as Signs of Safety, similarly highlight the difficulties associated with robustly evaluating social work models (Turnell and Edwards 2004).

3.5 Fit-for-purpose organisationThere is a great deal of consistency in the literature about the kind of organisational features required to support good quality practice and effective service delivery (see table 4 in appendix one), but again little consistency on how these features should be measured and limited robust evidence that these are the ‘right’ features to monitor. The five key features are described below.

• Workforce stability and engagement 4:‘failing departments’ (defined in terms of Ofsted ratings) are reported to be typically characterised by staff instability including at leadership level (ISOS 2016, Le Grand et al 2013 and 2014). Staff stability has also been reported to affect aspects of practice (Ofsted 2015a, Stein 2009). However, the strongest evidence identified relates to staff engagement. Through statistical modelling Glisson et al (2011) explored the relationship between staff engagement and satisfaction and individual-level service user outcomes. They found that behaviour scores for children were significantly and clinically improved in welfare organisations where staff had higher scores for engagement (measured through feelings of personal accomplishment and involvement in their work), while practitioner accounts of service quality and the number of services provided were not associated with child outcomes.

• Leadership: there is a strong emphasis in the literature on the need for effective leadership to underpin good quality practice and effective service delivery (e.g.

4 Social work competency and social work training are also expected to be closely linked to the quality of practice, however, they were beyond the scope of our study which focused on local responses to service improvement and the role of national policy levers in influencing a range of more of less effective local responses.

C4EO 2011, Hay Group Holdings 2015, ISOS 2016), and concerns that too many children’s social care departments lack effective leadership (Ofsted 2015a). Components of effective leadership have been mapped (e.g. C4EO 2011, Hay Group Holdings 2015), but do not seem to have been systematically and consistently monitored across local authorities to establish what impacts they have on social work practice and/or service delivery.

• Inter-agency working: virtually every report on children’s social care highlights the importance of effective inter-agency working, and serious failures in service delivery are typically associated with poor collaboration between different agencies (e.g. Bachmann et al 2009, Home Office 2013, ISOS 2016, Le Grand et al 2013 and 2014, Ofsted 2015b). Views in the literature vary between those who directly link effective inter-agency working to positive outcomes for users, based on the views of safeguarding experts (e.g. Home Office 2013), and those who argue that inter-agency working can help to improve efficiency (Bachmann et al 2009), but on its own is probably insufficient to improve services and outcomes for users (Rowlands 2010). There is considerable consistency in the literature about the broadly defined components of effective inter-agency working, but no consistency in how these should be measured and monitored, with qualitative accounts of these components and their relationship to practice and service delivery being predominant.

• Organisational culture: some of the literature has focused on the role of the organizational culture to support (or undermine) a culture of continuous learning and service improvement. The Munro review of child protection (2011) pointed to the repeated failure of organisational learning, a systemic problem linked to a culture which prioritises adherence to top-down managerial process and regulation, instead of focusing on practice, learning and outcomes for families. Similar messages have been highlighted in other work (e.g. DH 2000, Horwath and Morrison 2000, Munro and Hubbard 2011, Stein 2009). Key themes about features of organisational cultures seem to have been explored mainly qualitatively, but it should be possible to adapt indicators from the organisational management literature to measure children social services’ cultures more systematically.

• Effective (IT) systems: a small number of reports reviewed found links between effective IT systems and the quality of practice (Ofsted 2015b) and effective service delivery (Gibson and O’Donovan 2014). One could argue that an assessment of a wider range of support functions (e.g. HR functions, admin support) is likely to be required to ensure social work staff can operate effectively.

3.6 DiscussionOur review has identified a lack of explicit and detailed expectations in recent policy documents about outcomes for children’s social care users, and a tendency in research to focus on a limited range of narrowly defined outcomes which do not fully reflect experiences of services, nor the range of outcomes that would be typically explored to monitor the impact of public services on children’s development and envisaged in the Assessment Framework (Department of Health 2002).

In relation to monitoring the delivery of the right services of the right quality, we found that much of the evidence is based on narrative descriptions, with a scarcity of quantitative indicators to systematically and consistently measure assessment processes and the quality of practice, and explore associations with outcomes for service users.

Home grown social work models are at an early stage of development and testing, so we do not know if, how and for whom they work. The difficulties associated with robust impact assessments of these models should not be underestimated, but this does not mean that they should not be attempted.

We identified some broadly defined organisational features that are believed to support good quality practice and effective service delivery, but very little on how these should be measured and whether these are the ‘right’ features to monitor. While it may not be possible to directly link organisational features to most outcomes for service users, it should be possible to test if and how organisational features impact on social work practice and/or service delivery, which in turn will affect outcomes for users.

The quantitative analysis of Department for Education (DfE) data this chapter focuses on first sets out the degree to which there was consistency in the local authorities appearing in the top and bottom percentiles, and how that compared with Ofsted SIF ratings. The chapter then presents the findings of the regression analysis between the Ofsted ratings and DfE data. As explained earlier analysis of DfE data included:

11 child outcome variables from the Looked after Children in England including adoption (year ending 31st March 2015) dataset and the Characteristics of Children in Need (year ending March 31st 2015) dataset; and

3 workforce variables from the Children’s Social Work Workforce (year ending 30th September 2015) dataset.

The Ofsted ratings were based on Ofsted’s Single Inspection Framework (SIF) for inspecting services for children in need of help and protection, children in care and care leavers. Inspections included in this analysis were carried out between November 2013 and March 2016.

4.1 Percentile analysisThe table 1 shows the number of best performing local authorities according to the child outcome indicators.5

Overall, 93 (61%) local authorities ranked in the best performing 10% for at least one outcome indicator, and 59 (39%) did not rank in the best performing 10% in any of the outcome indicators. Six local authorities ranked in the best performing 10% in four or five indicators (these are identified in Table 2), and there were no local authorities that ranked in the top 10% for more than five indicators.

Table 3 shows the number of local authorities which ranked in the worst performing 10% for child outcome indicators. A total of 63 (41%) local authorities did not rank in the worst performing 10% for any of the child outcome indicators, and 89 (59%) ranked in the worst performing 10% for at least one indicator. Six local authorities ranked in the worst performing 10% in four indicators (displayed in Table 4) and no local authority ranked in the worst performing 10% in more than four indicators.

Chapter 4

Findings from the analysis of DfE data and Ofsted Ratings

5 The authors acknowledge that ‘best’ and ‘worst’ are contentious judgements, and that the context of each indicator would need exploring further to add to the rigour of this analysis. The authors plan to examine the contextual factors in detail for the next phase of this study.

Table 1: Number of Local Authorities ranked in best performing 10% for child outcome indicators

Number of LA’s with no child outcome indicators ranked in best performing 10% 59 39%

Number of LA’s with one child outcome indicator ranked in best performing 10% 47 31%

Number of LA’s with two child outcome indicators ranked best performing 10% 26 17%

Number of LA’s with three child outcome indicators ranked best performing 10% 14 9%

Number of LA’s with four child outcome indicators ranked in best performing 10% 3 2%

Number of LA’s with five child outcome indicators ranked in best performing 10% 3 2%

Total 152 100%

Table 2: Local authorities with four or five child outcome indicators ranked in best performing 10% and corresponding Ofsted ratings

LA Ofsted rating: Date of Ofsted

Brighton and Hove Requires improvement 22/06/2015

Lambeth Inadequate 12/05/2015

Salford Good 22/06/2015

Cheshire West and Chester Good 19/02/2016

Croydon Not yet completed

Sandwell Inadequate 05/06/2016

Table 3: Number of Local Authorities ranked in worst performing 10% for child outcome indicators

Number of LA’s with no child outcome indicators ranked in the worst performing 10% 63 41%

Number of LA’s with one child outcome indicator ranked in the worst performing 10% 42 28%

Number of LA’s with two child outcome indicators ranked in the worst performing 10% 27 18%

Number of LA’s with three child outcome indicators ranked in the worst performing 10% 14 9%

Number of LA’s with four child outcome indicators ranked in the worst performing 10% 6 4%

Total 152 100%

Table 4: Local authorities with four child outcome indicators ranked in worst performing 10% and Ofsted ratings

Las Ofsted rating Date of Ofsted

Central Bedfordshire Not yet completed

East Sussex Good 21/03/2014

Leicester Inadequate 20/03/2015

Plymouth Requires improvement 06/01/2015

Reading Not yet completed

Worcestershire Not yet completed

6 Note: while these correlations are statistically significant, they are considered weak.

4.2 Regression analysis

Regression analysis was carried out to explore possible relationships between the selected indicators and Ofsted ratings. The results of this analysis are included in the table below, they show6:

• Only one child outcome indicator had a statistically significant relationship with Ofsted ratings. This indicator was the percentage of looked after children who had a missing incident during the year. However, the analysis showed a positive relationship i.e. the better the Ofsted rating the more missing incidents.

• One workforce indicator had a statistically significant relationship with Ofsted ratings. This indicator was the agency worker rate, and showed a negative relationship, i.e. the lower the agency worker rate the better the Ofsted rating.

A detailed breakdown of the regression analysis can be found in appendix two.

4.3 DiscussionThis analysis was part of a small feasibility study, and was not meant to be exhaustive but to provide an indication of whether Ofsted ratings could provide a reliable sampling criterion for research. Based on these findings we have concluded that Ofsted ratings (alone) would not provide a reliable way of selecting children’s social care services based on their performance for research purposes. A number of issues also arose during this analysis which should be considered further if future, more in-depth, analysis were to be conducted.

The results indicate the importance of the consideration of the context of each indicator within local authorities. For instance, the analysis shows that higher numbers of missing incidents per looked after children in a local authority is predictive of a better Ofsted judgement. This result is counter intuitive and would need analysing in greater detail to uncover the context of these findings. The data would benefit from being linked with other measures, for example the age of the looked after population in a local authority area (one could hypothesise a correlation between missing incidents and a larger adolescent population of looked after children). Furthermore, do better performing local authorities according to Ofsted having better methods of tracking looked after children, and knowing when they are indeed missing, or do the better performing local authorities according to Ofsted have better management information systems that can more accurately record missing incidents? These nuances would need exploring in further detail, if we are to be confident about the meaning of this finding.

During this small-scale, feasibility study it was not possible to incorporate an analysis of trends over time. Trend

data was not available for all of the selected indicators, either because they were newly introduced data items, or because their definition had changed since previous data releases. It was also not possible within the remit of this feasibility study to incorporate an analysis of financial data. The released Section 251 expenditure data would have required substantial re-structuring to align with the other datasets that were being used. Any secondary analysis of the Section 251 data would also require consideration of the limitations and comparability of the data (Holmes and McDermid, 2012; National Audit Office; 2015).

The most recent round of Ofsted inspections was not complete at the time of analysis. Only about two-thirds of local authorities had been inspected using the new single assessment framework for inspection. Therefore one-third of local authorities were not included in this analysis. Future analysis should consider an examination of the proportion of the eligible population for which data is returned by local authorities. While a thorough analysis of this was not possible for this feasibility study, it was noted that there were variations between the proportions of data submitted by local authorities. This additional consideration would greatly improve the rigour of the current analysis.

This feasibility study focused on fundamental questions about the definition of a ‘good’ service for children in need and ways of accurately assessing whether that standard has been reached. The ultimate aim of the feasibility study was to assess if there is sufficiently robust national evidence to select children’s social care services based on the quality of their provision and their capacity (or lack of it) to improve services and ultimately make a positive difference to the lives of children in need and their families. Some of the findings reflected what might have been expected, but others were more surprising and have made the study team reconsider whether the main project this study was feeding in to is possible.

The rapid literature review was structured using a clear theoretical framework, the findings across the different domains were united by the fact that despite the large number of documents assessed, there was relatively little robust, empirical quantitative evidence in this area. Of particular relevance for the feasibility study is the fact that there was a lack of consistent expectations about outcomes for children’s social care users and what indicators should be used to consistently monitor these. In addition, the narrowly defined set of outcomes does not reflect all those that would be hoped for children, young people and their families. The evidence on factors that underpin quality of practice and effective service delivery was largely qualitative with little systematic attempt to operationalise and measure key concepts in a consistent way and explore associations with outcomes for users. Similarly, the evidence about organisational features that are believed to support good practice and effective service delivery was largely qualitative with little consensus on how they should be measured or indeed if they are in fact the most appropriate mechanisms to focus on.

While the findings from the literature review, and in particular the fact that there was limited robust evidence, was perhaps not surprising, the findings of the analysis of the quantitative data was more unexpected. There did not seem to be any pattern in terms of the local authorities that were in the top or bottom percentiles for the DfE child outcome variables. This was illustrated by the fact that there were no local authorities that ranked in the top 10% for more than five out of 11 outcome variables and even more strikingly no authorities that ranked in the worst performing 10% in more than four out of 11 variables. These findings were reinforced by the regression analysis, which found that only one child outcome variable and only one workforce variable had a statistically significant relationship with the Ofsted ratings, and these associations were very weak.

As discussed in chapter four, the reason for this apparent lack of pattern could be because of the quality of the data or because a range of contextual factors need to be taken into account in the analysis. However, it could also be because the child outcome data collected at a national level is not the right data to use to assess the effectiveness of children’s social care services, or/and that Ofsted ratings do not reflect the quality of services being provided, or that the quality of children’s social care services is not sufficient in itself to affect children and young people’s outcomes.

In order to explore these different interpretations further, the team held a seminar attended by academic, policy and social work experts, including those from the DfE, Ofsted, the Association of Directors of Children’s Services, the Local Government Association, local authorities, the NSPCC, and researchers from the universities Oxford, Coventry and Bedfordshire, among others. The strong consensus at the seminar was that the DfE data we analysed and Ofsted ratings could

Chapter 5

Conclusion

not be relied upon to assess the quality of children’s social care services and design a study that required us to select authorities according to the quality of their children’s services and their capacity to improve (or lack of it). The clear recommendation was that before proceeding with a study exploring how children’s social care services improved, it was crucial to identify a framework that a wide range of stakeholders agreed was appropriate which identified expected outcomes for users of children’s services, which could then be used to measure and assess how good or effective a service is. The study team therefore concluded that at this time it was not feasible to go ahead with the study design originally envisaged, which would require a robust way of selecting authorities according to trends in service quality and performance in order to understand why and how service improvement can be achieved and sustained. Instead it agreed to focus on how to create the outcomes framework and establish an appropriate set of indicators. The team is currently seeking funding to take this work forward.

REFERENCES

ADCS, LGA, & Solace (2015) Multi-agency Inspection of Child Protection: A Position Paper from ADCS, LGA and Solace, Manchester: ADCS.

Axford, N. & Little, M. (2006) Refocusing children’s services towards prevention: lessons from the literature, Children and Society, vol. 20, pp. 299- 312.

Bachmann, M. O., O’Brien, M., Husbands, C., Shreeve, A., Jones, N., Watson, J. & Mugford, M. (2009) Integrating children’s services in England: national evaluation of children’s trusts, Child: care health and development, vol. 35, pp. 257- 265.

Brandon, M., Bailey, S. & Belderson, P. (2010) Building on the learning from serious case reviews: a two-year analysis of child protection database notifications 2007-2009. London: Department for Education.

Brandon, M., Sidebotham, P., Bailey, S., Belderson, P., Hawley, C., Ellis, C. & Megson, M. (2012) New learning from serious case reviews: a two-year report for 2009-2011. London: Department for Education.

Brodie. I. (2010) Improving educational outcomes for looked after children and young people, London: C4EO.

Bywaters, P., Bunting, L., Davidson, G., Haratty, J., Mason, W., McCarten, C. and Steils, N. (2016) The relationship between poverty, child abuse and neglect: an evidence review, London: Joseph Roundtree Foundation.

Centre for Excellence and Outcomes in Children and Young People’s Services (C4EO) & National College for Leadership of Schools and Children’s Services (2011) Resourceful leadership: how directors of children’s services improve outcomes for children: full report, Nottingham: National College for Leadership of Schools and Children’s Services.

Children and social work bill (2016) http://www.publications.parliament.uk/pa/bills/lbill/2016-2017/0001/17001.pdf

Clarkson, P., Davies, S., Challis, D., Donnelly, M., & Beech, R. (2009) Has social care performance in England improved? An analysis of performance ratings across social services organisations Policy Studies vol. 30 (4) pp. 403-422.

Department for Children Schools and Families (2008) Better Outcomes for Children and Young People- From Talk to Action, London: DCSF.

Department for Education and Skills (2007) Care Matters: Time for Change. Norwich: The Stationery Office.

Department for Education (2016a) A vision for change, London: DfE. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/491968/Childrens_social_care_reform_a_vision_for_change.pdf

Department for Education (2016b) Children’s Social Care Innovation Programme, London: DfE. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/513390/Children_s_Social_Care_Innovation_Programme.pdf

Department of Health (2000) An organisation with a memory. Report of an expert group on learning from adverse events in the NHS chaired by the Chief Medical Officer, London: The Stationary Office.

Department of Health (2002) Framework for the Assessment of Children in Need and their Families, London: The Stationary Office. http://webarchive.nationalarchives.gov.uk/20130401151715/https:/www.education.gov.uk/publications/eOrderingDownload/Framework%20for%20the%20assessment%20of%20children%20in%20need%20and%20their%20families.pdf

http://www.publications.parliament.uk/pa/bills/lbill/2016-2017/0001/17001.pdf

http://www.publications.parliament.uk/pa/bills/lbill/2016-2017/0001/17001.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/491968/Childrens_social_care_reform_a_vision_for_change.pdf



https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/513390/Children_s_Social_Care_Innovation_Programme.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/513390/Children_s_Social_Care_Innovation_Programme.pdf

http://webarchive.nationalarchives.gov.uk/20130401151715/https:/www.education.gov.uk/publications/eOrderingDownload/Framework%20for%20the%20assessment%20of%20children%20in%20need%20and%20their%20families.pdf




Dettlaff, A. J. & Rycraft, J. R. (2009) Culturally competent systems of care with Latino children and families, Child Welfare, vol. 88(6), pp. 109- 126.

Dickson, K., Sutcliffe, K., Gough, D. & Statham, J. (2009) Improving the emotional and behavioural health of looked after children and young people, London: C4EO.

Douglas, D., Jenkins, W. & Kennedy, J. (2012) Understanding continuous improvement in an English local authority, International Journal of Public Sector Management, vol. 25 (1), pp. 17-33.

Easton, C., Hetherington, M., Lamont, E., Phillips, L. & Walker, F. (2013) Evaluation of the sector-led peer challenge programme 2012/13, Slough: National Foundation for Educational Research.

Fauth, R., Jelicic, H., Hart, D., Burton, S. & Shemmings, D. (2010) Effective practice to protect children living in ‘highly resistant’ families, London: C4EO.

Fish, S. (2009). What are the key questions for audit of child protection systems and decision-making? London: Centre for Excellence and Outcomes in Children and Young People’s Services.

Forrester, D., Westlake, D., McCann, M., Thurnham, A., Shefer, G., Glynn, G. & Killian, M. (2013) Reclaiming social work? An evaluation of systemic units as an approach to delivering children‘s services: final report of a comparative study of practice and the factors shaping it in three local authorities, Luton: Tilda Goldberg Centre for Social Work and Social Care.

Gibson, J. & O’Donovan, B. (2014) The Vanguard Method as Applied to the Design and Management of English and Welsh Children’s Services Departments, Systemic Practice and Action Research, vol. 27(1), pp. 39-55.

Glisson, C. & Green, P. (2011) Organizational climate, services, and outcomes in child welfare systems, Child Abuse and Neglect, vol. 35(8), pp. 582- 591.

Glisson, C., Green, P. & Williams, N. (2012) Assessing the Organizational Social Context (OSC) of child welfare systems: Implications for research and practice, Child Abuse and Neglect, vol. 36(9), 621- 632.

Hay Group Holdings (2015) Developing Outstanding Social Workers. What makes great first time leadership? London: Frontline.

HM Government (2015) Working together to safeguard children. A guide to inter-agency working to safeguard and promote the welfare of children, London: Department for Education. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/419595/Working_Together_to_Safeguard_Children.pdf

Home Office. (2013) Multi-agency working and information sharing project: early findings, London: Home Office.

Horwath, J. & Morrison, T. (2000) Identifying and implementing pathways for organizational change: using the “Framework for the assessment of children in need and their families” as a case example, Child and Family Social Work, vol. 5, pp. 245- 254.

ISOS Partnership (2016) Action research into improvement in local children’s services- interim report, London: ISOS.

Jas, P. (2013) The role of interim managers in performance improvement: evidence from English local authorities, Public Money & Management, vol. 33(1), pp. 15-22.

Kirkman, E. & Melrose, K. (2014) Clinical judgement and decision-making in children’s social work: an analysis of the “front door” system: research report, London: Department for

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/419595/Working_Together_to_Safeguard_Children.pdf

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/419595/Working_Together_to_Safeguard_Children.pdf

Education (DfE).

Le Grand, J., Wood, A. & Gibb, M. (2013) Report to the Secretary of State for Education on ways forward for children’s services in Doncaster. London: Department for Education.

Le Grand, J., Wood, A. & Trowler, I. (2014) Report to the Secretary of State for Education and the Minister for Children and Families on ways forward for children’s social care services in Birmingham. London: Department for Education.

Lwin, K., Versanov, A., Cheung, C., Goodman, D. & Andrews, N. (2014) The use of mapping in child welfare investigations: a strength-based hybrid intervention, Child care in practice, vol. 20(1), pp. 81-97.

Malin, N. & Tunmore, J. (2013) Assessing the impact and implications for social workers of an innovative children’s services programme aimed to support workforce reform and integrated working Social Work and Social Sciences Review vol. 16 (3) pp. 81-90.

Munro, E. (2010a) Learning to reduce risk in child protection, British Journal of Social Work, vol. 40:11, pp. 1135- 1151.

Munro, E. (2010b) The Munro review of child protection part one: a systems analysis. London: Department for Education.

Munro, E. (2011) The Munro review of child protection: final report. A child-centred system, London: Department for Education.

Munro, E. & Hubbard, A. (2011) A systems approach to evaluating organisational change in children’s social care.

Ofsted (2013) Social Care Annual Report 2012/2013. London: Ofsted.

Ofsted (2014) An overview of inspection findings. An overview report of the inspection findings from Ofsted, the Care Quality Commission, Her Majesty’s Inspectorate of Constabulary, Her Majesty’s Inspectorate of Probation and Her Majesty’s Inspectorate of Prisons in relation to the help, care and protection of children. London: Ofsted.

Ofsted (2015) Framework and evaluation schedule for the inspections of services for children in need of help and protection, children looked after and care leavers, London: Ofsted. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/466010/Framework_and_evaluation_schedule_children_in_need_of_help_and_protection_CLA_and_care_leavers_LSCBs.pdf

Ofsted (2015a) Joining the dots. Effective Leadership in children’s services, Manchester: Ofsted.

Ofsted (2015b) Social care: the report of Her Majesty’s Chief Inspector of Education, Children’s Services and Skills 2013–14, Manchester: Ofsted.

Ofsted (2015c) The quality of assessment for children in need of help. Manchester: Ofsted.

Prochaska, J. & DiClemente, C. (1982) Transtheoretical Therapy: Toward a more integrative model of change, Psychotherapy: Theory, Research and Practice, vol. 19(3), pp. 276- 288.

Rowlands, J. (2010) Services are not enough: child well-being in a very unequal society, Journal of Children’s Services, vol. 5(3), pp. 80-88.

Searle, R. H. & Patent, V. (2013) Recruitment, retention and role slumping in child protection: the evaluation of in-service training initiatives, British Journal of Social Work, vol. 43(6), pp. 1111-1129 .

Sebba, J. Berridge, D., Luke, N., Fletcher, J., Bell, K., Strand, S., Thomas, S., Sinclair, I. & O’Higgins, A. (2015) The Educational Progress of Looked After Children in England: Linking

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/466010/Framework_and_evaluation_schedule_children_in_need_of_help_and_protection_CLA_and_care_leavers_LSCBs.pdf



Care and Educational Data, Oxford, Bristol and London: The Rees Centre, The University of Bristol and The Nuffield Foundation.

Singleton, R. (2010) The chief adviser of the safety of children: first annual report to parliament: 2010, London: Department for Children, Schools and Families.

Solace (2014) Reclaiming Children’s Services. Policy Report Number Three. London: Solace.

Solomon, E. (ed) (2016) Re-thinking Children’s Services - Fit for the Future? London: NCB and Catch 22.

Stanley, N., Austerberry, H., Bilson, A., Farrelly, N., Hargreaves, K., Hussein, S. & Strange, V. (2014) Establishing Social Work Practices in England: the early evidence, British Journal of Social Work, vol. 44(2), pp. 367-383.

Stein, M. (2009) Quality Matters in Children’s Services. Messages from research, London: Jessica Kingsley Publishers Ltd.

Turnell (2012) The Signs of Safety: Comprehensive Briefing Paper, Resolutions Consultancy Pty Ltd: Perth, Australia.

Turner, D., Skelcher, C., Whiteman, P., Hughes, M. & Jas, P. (2004) ‘Intervention or Persuasion? Strategies for Turnaround of Poorly-Performing Councils’. Public Money and Management, vol. 24(4), pp. 217-226.

Wiggins, M., Austerberry, H., & Ward, H. (2012) Implementing evidence-based programmes

in children’s services: key issues for success. London: Childhood Wellbeing Research Centre, Department for Education.

Appendix 1: Summary of literature reviewed

Table A1a: Outcomes for children in need and families

Variables Summary of evidence

Looked after children placement stability, placement rates, length of time in care, satisfaction with placement

ADCS report on looked after children (ADCS, 2015) looked at: looked after children with 3+ placements, stability of looked after children placements, time between court and adoption match, care leavers in suitable accommodation.Satisfaction with placement and placement stability associated with better educational outcomes for looked after children (Brodie 2009).Used for evaluations of enhanced foster care, MST and mentoring for looked after children (Dickson et al 2009) and a review of child protection practice (Fauth et al 2010).Sebba et al (2015) found an association between educational outcomes and placement stability, length of time in care, and age when entering care.Placement stability identified by Stein (2009) as key factor supporting positive outcomes for looked after children, with three measures of stability considered: 3+ placements per year; proportion of children looked after for four+ years who had been in the same foster placement for 2+ years; proportion of children under 16 looked after for 2.5+ years and placed for adoption.

Cp referrals/re-referrals ADCS report on looked after children (ADCS, 2015) looked at: repeat referrals, second/ subsequent CP plans, CP plans last 2+yearsCP referrals and re-referrals reported in review of child protection practice (Fauth et al 2010)

Educational outcomes e.G. Attendance, exclusion, qualifications

ADCS report on looked after children (ADCS, 2015) looked at: NEET 16-18, % half days missed, permanent exclusions- secondary, Early Years/ Foundation Stage attainment, Key Stage 2 Maths and English, GCSE A-C, level 2 and 3 qualification age 19, (looked after children) looked after children GCSE proportion, care leavers in EET, looked after children school exclusions, sessions missed by looked after children.Educational outcomes included in Brodie (2009) on what works in improving educational outcomes for looked after children.Sebba et al (2015) found the following to be associated with educational outcomes for looked after children:Type of school, i.e. mainstream versus other Length of time in care and age at entry into careGenderSDQ scoresSchool instability i.e. frequent school changes and unauthorized absences

Cyp’ views on educational experiences

Identified as an evidence gap (Brodie 2009)


Involvement in extra curricula activities

looked after children programmes involving extra curricula activities or school related support proved popular and sustainable – but not direct evidence of impact (Brodie 2009)

Cautions, convictions, offending behaviour

ADCS report on looked after children (ADCS, 2015) looked at: First time entrants to YOS, % (looked after children) looked after children offending aged 10-17.Used for evaluations of enhanced foster care, MST and mentoring for looked after children (Dickson et al 2009).

Self-harming, violent behaviour

Used for evaluations of enhanced foster care, MST and mentoring for looked after children (Dickson et al 2009)

Emotional difficulties, behavioural problems


Self-esteem and feel valued


Substance/alcohol misuse Used for evaluations of enhanced foster care, MST and mentoring for looked after children (Dickson et al 2009)

Community engagement Used for evaluations of enhanced foster care, MST and mentoring for looked after children (Dickson et al 2009)

Family and peer relationships


Physical and emotional care of child

ADCS report on looked after children (ADCS, 2015) looked at: hospital admissions, low birth weight rate and breastfeeding at 6-8 weeks excess weight in reception and year 6, (looked after children) looked after children emotional and behavioural health (ADCS, 2015)Physical and emotional care explored in review of child protection practice (Fauth et al 2010)

Teenage pregnancy Included in ADCS report on looked after children (ADCS, 2015)

Parental attitudes and behaviour

Reported in review of child protection practice (Fauth et al 2010)

Discipline and parental coping

Explored in review of child protection practice (Fauth et al 2010)


Relationship with social worker

An interim outcome explored a review of child protection practice (Fauth et al 2010) was good relationship between parent and social worked which was found to be associated with positive child outcomes.

Level of service use An interim outcome explored a review of child protection practice (Fauth et al 2010) was increase in family service which was found to be associated with positive child outcomes.

Table A1b: The right services of the right quality


Assessments & referral systems

Brandon et al (2010) described two factors that positively skewed practitioners appraisal of families, firstly adult-focused thinking that was sympathetic to social excluded and disadvantaged parents. Secondly, a tendency to see new circumstances (e.g. a new pregnancy or partner) in long-standing case as a ‘clean slate’ and new opportunity for change, this was particularly evident where practitioners were overwhelmed.An international review of child protection practice (Fauth et al 2010) found that practitioners’ ability to make more accurate assessments can enhance services, more accurate assessments require: 1)range of data sources in addition to talking to mothers (e.g. consult other family members and observed parent-child interaction) and examination of children’s attachment behaviours and detailed family histories; 2)identification of weaknesses and strengths and analyse complex interactions of different factors to determine family’s capacity to change. Ofsted (2015c) links good quality assessments with relevant training e.g. on using assessment tools and theoretical models of practice.ISOS (2016) found that one of the initial actions taken by LAs’ embarking on a improvement programme was to ensure an effective front-door system of managing referrals and assessments defined as: safe way of managing referrals into the service, allocating cases efficiently, focusing both on timelessness and quality of assessments and achieving securing in decision making. The front-door system was considered vital and a good service would become poor (not defined what poor means) within a matter of 3 months if it lacked a sufficient grip on the referral assessment process.Kirman and Melrose (2014) in a review of decision making in child protection cases found that assessments, particularly when workloads were high, were based on intuition rather than evidence-informed intuition, with decision-fatigue evident many decisions were required in a short period of time. Social workers could also place undue importance on information that confirmed pre-existing views or that came to mind easily. Authors also noted the low quality of information available to social workers to support decision making and an overarching lack of evidence about how to make decisions in particular circumstances. The latest Ofsted review of social care (Ofsted 2015b) found following areas for improvement in assessments for both early help and statutory work:

• In 8 of the 10 LAs inspected, assessment was undertaken as a standalone process to be ‘done’ to a family, rather than one part of an ongoing process to provide continued support and improve outcomes for children.

• Analysis in assessments did not clearly identify risks and strengths of individual families or indicate the potential for a family to achieve change.

• The language used in written documents was often unclear, overcomplicated, detracted from the concerns raised and was unhelpful to amilies.


• The quality of chronologies varied considerably, key information to inform assessments and plans was included in only a small number of cases.

• Often assessments did not fully include the views of extended family members, including grandparents, aunts and uncles.

• Professionals did not always update assessments to reflect changing circumstances and to inform planning for the child and family.

Assessments & referral systems

• Social workers did not always share the findings from assessments with children and their families to help them understand what was happening and the rationale for decisions made. Professionals in early help work shared assessments more readily.

• In a third of the 123 cases tracked, the written plans resulting from assessments were not good enough to drive improvement in children’s circumstances.

The report identified the following features of a good assessment: • done promptly • clearly identify risks and strengths of individual families and indicate potential for a

family to achieve change, clear analysis of findings to inform next steps• written in clear language and format suitable for sharing with children and families to

help them understand what was happening and the rationale for decisions made• good quality of chronologies with info required to inform plans

child’s voice and parents’ and extended family’s views reflected in assessment • views of range of relevant professionals • written plans resulting from assessments focus on the most important needs of

children, improve their experiences and include clear contingency plans• continuing to assess as new information comes to light and circumstances change.

Stein (2009) identified effective assessment as key to support positive outcomes for children and as being largely dependent on: availability of adequate information (from a range of agencies); social workers’ adherence to quality core assessment and recordings protocols.

Size of caseload Brandon et al (2010) describe the impact of practitioners feeling ‘overwhelmed’ by the size and scope of their casework on their ability of focus on the child, to undertake analytical thinking and make decisions.Research by Dettlaff et al (2015) supported other research that linked perceptions of ‘unmanageable caseload’ with poor case work. A key factor associated with successful implementation of the Reclaiming Social Work model was the size of the case load, which was half the size in the most successful LA compared (caseload size is not specified however) (Forrester et al 2013).ISOS (2016) cite one LA in their study that had found clear links between improve staff retention and caseloads of manageable size.Ofsted (2015c) links good quality assessments with manageable workloads.Kirman and Melrose (2014) in a review of decision making in CP cases found that heavy workloads meant social workers relied on intuition rather than evidence-informed intuition and could experience decision-fatigue when a high number of consecutive decisions were required in a short period of time. Solace (2014) found in its survey of members that increased demand was the key barrier to improvement and success.


Caseload management ISOS (2016) found that alongside an effective front-door system (see above) a good service requires timely flow of cases between teams e.g. through an area-based management system with all teams related to a child’s journey being under a single manager; a performance management approach with clear procedures and close monitoring of workflows and caseloads.In their report to the Secretary of State on future options for improvement in Doncaster children’s services, Le Grand et al (2013) cite the high number of unallocated cases as a historical cause for concern.

Child/family focus and engagement

Brandon et al (2012) found highlighted instances where practitioners fail to, or avoid, really engaging with or getting to know a child. This is linked to poor supervision and knowledge of child physical and emotional development.An international review of child protection practice (Fauth et al 2010) found that the child protection system is seen as a powerful tool, and approaches that share power with parents (e.g. to advocate for them) and involve them in their treatment are said to be helpful in increasing engagement. They also found that the voice of children is often missing within practice.The Munro review of child protection (2011) cites evidence that the needs of parents are sometimes prioritised over that of the child, and the views of children neither sought nor heard. The report explains that a culture which emphasises compliance and managerial process has taken the spotlight off relationships with children and families and developing effective social work expertise.Stein (2009) argues that high quality services require a culture of participation and dvocacy.

Proportion of time spent working directly with children and families

This was one of the positive outcomes from the initial findings from the Reclaiming Social Work evaluation (Forrester et al 2013)

Effective supervision Brandon et al (2010) summary of serious case reviews links regular and challenging supervision to the ability of practitioners to maintain effectively a compassionate yet critical stance when working with families.An international review of child protection practice (Fauth et al 2010) reported effective supervision is very important, but there is little evidence about its key components, possible indicators are: regular independent case audits and clinical supervision. The review also identified as possible indicators of ineffective supervision as: feeling overwhelmed, lacking confidence and ability to make decisions; practitioners simply ‘acting out’ their own strong feelings.ISOS (2016) link success in retaining workforce with good supervisory arrangements and active caseload management.Ofsted (2015c) links good quality assessments with robust, reflective managerial supervision and oversight of workers carrying out assessments.Singleton (2010) in its 1st report to parliament as Chief Adviser on the Safety of Children argued that adequate case management supervision and advice from more experienced colleagues key to effective practice.


Quality assurance/ auditing/ transparency/ scrutiny

Fish (2009) describes the shift towards local quality auditing of front line practice in assessment and decision-making as a means to improving quality and outlines most common models:

• Section 11 audit tools, which stem from the Children’s Act 2004 and focus on standards relating to staffing responsibilities and development at operational and management levels, as well as organisational processes linked to participation, multiagency working, safety and accessibility.

• Welsh LSCB selfassessment and improvement tool, which focuses on operational and managerial responsibilities for strategic focus, governance, staffing capacity, effective outputs all linked to good organisational and child outputs.

• Sheffield safeguarding evaluation programme which includes a number of methods for self-audit, case file analysis and feedback from partners, staff and service-users

These models are characterised by managerial systems and are contrasted with systems-based performance management models from other sectors, where groups of front-line staff have ownership of production problems and co-produce solutions. The paper debates the usefulness of performance indicators. On one hand, over-reliance on these can detract from a focus on outcomes and quality, on the other their role can facilitate accountability, and can support scrutiny and challenge.The evaluation of the peer-challenge programme (Easton et al 2013) which evolved on a regional basis, found that differing peer-challenge methodologies and governance structures were in place. Some regions operated formal quality assurance boards and regular operating cycles whereas others undertook more ad-hoc and developmental projects or aligned peer challenge with existing systems and processes. Topics for challenge were: child poverty, looked after children and corporate parenting, diverting children from care, domestic abuse, early intervention, children’s centres and early years, integrated disability services and adopted of disabled children, integration of public health in to council settings, justice reviews and troubled families, special educational needs, and NEET/ youth services. Areas of focus were selected individually by councils, usually based on self-assessment, or were agreed based on regional priorities. Host and peer-challenge authorities within regions were matched based on a number of factors including existing relationships, compatible strengths and weaknesses, geographic or demographic factors.

Qualitative and quantitative data was gathered and/or requested by authorities in preparation for the challenge exercise (e.g. self-assessment reports, Ofsted inspection reports, local performance data and qualitative feedback information). DCS and representative from varying management or practice tiers were involved, depending on relevance of roles and expertise. The challenges themselves were usually supported by a framework and handbook. The programme was largely popular and it was believed to provide a cost effective way of supporting good. The research gave examples where peer-challenge had led to local improvements, including: development of good self-awareness, improved relationships, and increased trust across and within agencies. It also instigated processes by which problematic performance and the need for early help could be identified and addressed within the region by sharing good practice. Peer-challenge was reported to help better define problems that required improvement, which in turn improved practice e.g. in front-door referral systems, commissioning of complex placements for looked after children. The peer-challenge programme had bene used in some cases in pre or post Ofsted inspection preparation.


Quality assurance/ auditing/ transparency/ scrutiny

ISOS (2016) found that:• children’s services with sustained good practice (=Ofsted rating?) were characterised

by openness to challenge, commitment to internal/external peer evaluations • staff in children services which were ‘securely good’ (with consistent high Ofsted

ratings?) understood what good quality looked like – indicators of latter mentioned related to processes eg commitment to ongoing training and development, healthy competition between teams, internal peer review structures

• LAs subject to externally chaired improvement boards following a poor inspection outcome often spoke highly about the benefits of this

Evidence based practice Axford and Little (2006) says EBP characterised by: • Standardised diagnostic tools with data regularly and effectively analysed to inform

service improvement• Implementation and evaluations of new programmes with strong evidence base • Funding mechanisms that incentivise early intervention

Dettlaff’s exploration of decision-making ecology (2015) summarised a range of factors identified as influential on decision making:

− Case factors: demographic (particularly race and ethnicity), physical and mental health, substance misuse, history of maltreatment.

− Individual factors: risk-aversion vs risk-acceptance, professional identification, experience, education and response to confrontation.

Organisational/ external factors: caseload, role clarity, supervision, concerns of personal liability, stress and turnoverDCSF (2008) reported on the implementation of Outcomes Based Accountability toolkit in several local authorities. Factors that supported the culture change required to focus on outcomes included sustained interest at senior level, ownership at all organisational levels, championing of outcomes and committed partnerships.An international review of child protection practice (Fauth et al 2010) argued that agencies and practitioners need to be more evidence based as there are some widespread beliefs about factors associated with abuse/maltreatment with no evidence to support them (or even to support the opposite). Greater use of theory to guide practice was one of the positive outcomes from the initial findings from the Reclaiming Social Work evaluation (Forrester et al 2013).ISOS (2016) found that:- effective use of data to improve the service was associated with: clarity about the ‘vital indicators’ that need to be monitored to assess the ‘health of a service’; regular data scrutiny as the different levels; a strong audit system which allowed triangulation of quant and qual performance indicators

• innovation is important but must be ‘disciplined and controlled’ with testing of new ideas and systems and consulting staff before they are scaled up

• while relying on what works seen as important, no real consensus on the level of ‘fidelity’ this would require with some high performing authorities strongly adhering to a particular social care model of practice (no examples provided), and others cherry- picking from a range of different approaches


Evidence based practice The Munro review of child protection (2011) highlights how a culture of compliance has negatively affected the development of social work expertise, learning from evidence and the application of professional judgement.Kirman and Melrose (2014) in a review of decision making in child protection cases found low awareness among social work staff of ‘what works’ and tools that can support decision-making.

Ofsted (2015c) links good quality assessments with use of theoretical models of practice (Signs of Safety and Reclaiming Social Work were by LAs reviewed). Stein (2009) identifies research as key to delivering effective services and support good outcome for children and families, use of performance indicators can also support good practice if the ‘right’ indicators are selected, if indicators are not fit for purpose they may encourage a focus on processes rather than quality.Wiggins et al (2012) identify several factors that affect the local replication of evidence-based interventions:

• Similarity or difference of previous practice• Fit and complexity of the intended population• Effective leadership• Resources• Staff approach and commitment to adopting a new programme model• Staff competency• Cross organisational support.

Stages of implementation moved from exploring population needs, resources and intervention fit, to staff and systems for delivery, to monitoring, evaluation and scale up.

Scope to use professional judgement

The Munro review of child protection (2011) highlights how a culture of compliance has negatively affected the development of social work expertise, learning from evidence and the application of professional judgement.For most vulnerable children many factors interact with one another, and successful intervention must understand the whole system of interaction of factors and how a minor change could have a big knock on effect. Intervention cannot follow a linear path, with predicable change-effect pattern, therefore frontline workers must be freed from prescriptive instructions and use their professional judgement to work with the complexities they face – opinion rather than evidence based statement (Rowlands 2010) Singleton (2010) in its 1st report to parliament as Chief Adviser on the Safety of Children said practitioners felt government devotes too much attention to process, some of it unhelpfully prescriptive, and not enough to strengthening professional practice. Practitioners felt some targets and indicators are too narrowly defined, don’t tell the ‘whole story’ and can create perverse incentives eg assessments which are rushed through and not done properly in order to meet requirement to do them within 35 days.

Quality of placements for looked after children

Stein’s review (2009) identifies placement quality as key for good child outcomes, defined broadly as ‘a placement that provides good social and psychological parenting’. In addition, good placements were described as those that help children and young people with behaviour and that reflect a child’s age behaviour and acceptance of care. Two measures of placement quality were constructed using ratings by social workers and supervising social workers. Ofsted (2015b) summary of children’s home found an overall slight decline in the number of homes that were rated good or better than the previous inspection (11% outstanding, -1%; 53% good, -4%; 27% adequate, +2%; 6% inadequate, =3%).

Length of intervention An international review of child protection practice (Fauth 2010) found that focused, long-term services appear to achieve better outcomes for children than episodic intervention.

Table A1c: Social work models

Programme Summary of evidence

Reclaiming Social Work

Reclaiming Social Work model involves the restructuring of social care departments into small teams, headed up by a consultant social worker and comprising social worker, child practitioner, therapist and a co-ordinator. The underpinning approach is that of family systems functioning. This study included observation, surveys and in-depth interviews to compare three authorities, one of which had adopted the RSW model. The study found several factors within the RSW authority that enabled implementation and generally improved the working environment for staff: the removal workplace blockers (such as IT and space issues), streamlining of decision making and bureaucratic processes, and reduced caseloads per unit. The evaluation linked these factors to improvements in practice, including, more direct work with children and families, better assessments, ongoing timely decision making, and increased confidence/ reduced stress amongst staff. Families reported higher satisfaction levels, but no higher perception of improvements in their family life. The authors note limitations linked to the small sample size and evidence of outcomes (Forrester et al, 2013).

Signs of Safety

Signs of Safety (Turnell and Edwards 1997) is a strength-based and collaborative approach to assessment and child protection casework. It was developed in Australia but since replicated internationally in the UK, USA, Canada, Sweden, Netherlands, New Zealand and Japan. Signs of Safety is described as a continually evolving model, which was based on, and continues to draw on, examples of good and collaborative practice. A number of evaluations point to potential benefits associated with the approach. Turnell’s in-depth briefing (2012) summarises international research that reports increased worker satisfaction, high levels of service user satisfaction and engagement, and a reduction in child removals and re-referral rates. Lwin et al (2014) found that cases where the Signs of Safety mapping process was used were significantly more likely to be referred to ongoing service provision compared to those that did not. A number of limitations in the literature are noted, some rely on trends data without comparators. Control groups feature in some studies but authors note differences that may account for differences in outcomes.

Social Work Practices

A quasi-experimental comparison between small, independent social work-led organisations and local authority-led children’s services was undertaken. The independent units were run by private, voluntary and social enterprise organisations and had differing set up processes and histories in providing social care services. The study found no conclusive evidence that the new units experienced improved placement change rates or quality relationships with children and families. Some staff reported increased opportunities for direct and preventative work with children and young people (Stanley et al, 2012).

Vanguard method

Gibson and O’Donovan (2014) explored the application of the Vanguard Method (Seddon, 2003) in children’s social care. They found that the extensive system re-design- using an ‘inside-out’ approach rather top-down or externally defined framework- led to reduced times between referral and service provision, and reduced the number of referrals. The system redesign involved applying the ‘check-plan-do’ model:

• The ‘check’ step involves mapping the system from a ‘customer’ perspective and conducting an analysis of demand, capability, flow, waste and underlying principles.

• The ‘plan’ and ‘do’ steps involved an ongoing process of experiment and improvement process to reconfigure jobs roles and focus effort on the key system points and work towards newly defined customer and organisational outcomes.

Over the period of implementing the redesign the local authorities witnessed a series of improvements, for example a reduction in the time between referral and service provision and a reduction of referrals.

Table A1d: Fit-for-purpose organisation Variables Summary of evidence

Workforce stability and engagement

Glisson et al (2011) used statistical modelling techniques to explore the relationship between staff engagement and satisfaction and individual-level service user outcomes. They found that behaviour scores for children were significantly and clinically improved in welfare organisations where staff had higher scores for engagement (measured through feelings of personal accomplishment and involvement in their work), and not practitioner accounts of service quality or number of services provided.

Jas (2013) found interim managers could be effective agents for change in poorly performing authorities by participating in strategic planning, change management or by bringing legitimacy to existing strategies. However, their impact could be hampered where staff were experiencing ‘consultant fatigue’ or were mistrustful of ‘outsiders’.

ISOS (2016) identified a number of workforce issues including:

• -links service effectiveness (=Ofsted rating?) to stable leadership and effective group of permanently employed heads of service and team managers

• explore workforce planning in different local circumstances (e.g. with higher/lower competition for SW staff) and analysis of retention benefits. –found a lack of staff engagement tended to characterise LAs which experienced failure or poor aspects of services (as assessed by Ofsted?)

• report an LA which had improved from an inadequate Ofsted rating and which had reduced reliance on agency staff from 50% to 4%.

In their reports to the Secretary of State on future options for improvement in Doncaster and Birmingham children’s services, Le Grand et al (2013, 2014) link successive and unpopular changes to structure and senior workforce to poor performance and delays in improvement. For example between 2005 and 2013 Doncaster experienced six DCSs and four chief executives.

Ofsted (2015c) links good quality assessments with continuity and consistency in workers for children and families.

Searle and Patent (2012) found that involving frontline staff in decision making had a positive impact on morale and subsequently staff retention.

Singleton (2010) highlights need to explore how LAs forecast demand for different children’s services, how they use this info plan demand for different types/level of staff, considering ratios between newly qualified and more experienced SWs, use of agency staff, retention and sickness levels.

High staff turnover linked to disruption in implementing care plans (Stein 2009).

Leadership C4EO (2011) conducted a review of good leadership in high performing children’s social care services. Three sources of publically available data (CAA ratings-2009, Children’s Services performance profile- Ofsted 2009 and National Indicator Set of CYP Outcomes- Ofsted) were used to identify authorities that demonstrated consistent good performance, had shown considerable improvement or were demonstrating a ‘trajectory of improvement. DCS’s behaviours associated with high performance included:

• Openness to possibilities• Ability to collaborate• Demonstrating belief in their team• Tenacity and resilience• Ability to generate commitment• Focusing on results and outcomes• Ability to simplify


Leadership • Ability to learn continuously• Resourcefulness i.e. ability to actively expand resources once can draw on to address

challengesHay Group Holdings (2015) were funded by DfE to develop ‘Firstline’, a leadership programme for good social worker managers. Initial research to inform the leadership programme found that good social work managers exhibited characteristics of good leadership (intelligence, decision-making capabilities, energising and empowering social workers) as well as exhibiting high emotional intelligence. Outstanding leaders went further, demonstrating leadership vision, enabling reflective practice, make evidence-based decisions, managing staff relationships and empowering staff within the children’s services system. Contextual enablers included stable and skills teams with clear roles, ongoing leadership development and effective senior leadership.

ISOS (2016) found lack of strategic vision tended to characterise LAs which experienced failure or poor aspects of services (as assessed by Ofsted?). An example of leadership ‘failure’ provided was over-commitment to a particularly initiative which could lead to neglect of other areas of work, or disruption to vital services and systems. The author argue that long-term sustainable improvement must start at the top, with a clear and constituent vision, based on honest self-appraisal and a forensic assessment of areas that require improvement. They also link service effectiveness (=Ofsted rating?) with some key leadership characteristics, including: relentless in pursuit of quality; demonstrably engaged and interested in frontline practice; able to focus on details without losing sight of the big picture; able to model the behaviours and standards expected from the organisation; attention to innovation.

Of the 59 inspections carried out using the SIF since November 2013, Ofsted (2015b) concluded that half of local authorities required improvement overall. They found a great deal of variability in relation to leadership, management and governance. Leadership was the only area where authorities (n=3) were judged to be outstanding, but was also the area where the highest number of inadequate judgements were made (n=13). In their thematic report Ofsted described good leaders as being open, honest, collaborative, moral and professionally strong. Also visible, credible and decisive, yet trusting. Amongst their workforce they encouraged creativity whilst ensuring consistency, encouraged reflective supervision and inspired confidence, they set- and modelled- high expectations, and developed a culture where support and challenge were balanced. They developed clear lines of accountability to support decision-making and used data on performance and hard and soft outcomes to inform their work. Good leaders also managed workforce issues well by monitoring caseloads and workloads, and staff vacancies. Strategies were developed to recruit, retain and develop staff.

Stein (2009) argues that high quality services require effective strategic planning, structures and policies.

Turner et al (2004) found that complacency, denial, inertia and silo management were all evident in their study of poor performing authorities where little evidence of improvement was found after change programmes were attempted.

Inter-agency work The evaluation of Children’s Trusts pathfinders (Bachmann et al 2009), which aimed to improve inter-agency working, largely focused on implementation and provided no evidence that better interagency working (where it was achieved) was linked to observable improved outcomes for children, although professionals believed that there had been some improvements, including: improved access to services for families; new and innovative service provision; more efficient service management; improved relationships with other agencies. It was noted that the impact of changes had largely been visible in structural, managerial and bureaucratic arenas, rather than in service delivery.


Inter-agency work A review of effective child protection practice (Fauth et al 2010) identified possible indicators of effective multi-agency working: having in place mechanisms to exchange of information and channels of communication; clarity in relation to roles and boundaries; multi-disciplinary training; co-location.

A Home Office (2013) review of MASH found that those interviewed (safeguarding experts from various agencies) felt that the new arrangements led to improved outcomes for children and families by facilitating more reliable and informed risk assessments, which led to earlier intervention and more efficient referrals, improved professional relationships and better case management. Factors linked to good multi-agency arrangements included: good relationships and information sharing; co-location (but not all in favour of this); strategic level buy-in to multi-agency working; shared tools; shared reviews and scrutiny processes; possibly integrated IT systems but not all agreed.

ISOS (2016) found that LAs with MASH teams with strong engagement from partner agencies strongly advocated for the benefits of MASH and that poor interagency working was associated with lack of alignment in thresholds.

In their reports to the Secretary of State on future options for improvement in Doncaster and Birmingham children’s services, Le Grand et al (2013, 2014) suggest poor relationships, or the isolation of children’s services, within the rest of the authority are linked to poor children’s services performance.

Ofsted (2015c) links good quality assessments with strong partnership working and information sharing.

While important to improve co-ordination and efficiency between agencies, not a ‘magic bullet’ and on its own not sufficient to improve outcomes for children – commentary not evidence based claim (Rowlands 2010).

Staff skills, training and development

A review of effective child protection practice (Fauth et al 2010) found that the attitudes and behaviour of individual practitioners have a major effect on whether families engage or not – empathy and established relationship skills (e.g. active listening, demonstrable genuineness and respect) combined with an eyes-wide-open, boundaried, authoritative approach aimed at containing anxiety and ensuring that the child’s needs and outcomes stay in sharp focus.

ISOS (2016) cite one LA in their study that had found clear links between staff retention and improved training and development, including clear pathways to advanced practitioner status/team manager and opportunities to specialise.

Stein (2009) goes beyond just training and development opportunities and argues that high quality services requires a learning organisation and innovation that uses performance indicator data, and open discussion and debate, as tools to improve quality.


Organisational culture, organisational learning and continuous improvement

Brandon et al (2012) review of SCRs highlighted the high volume of procedural recommendations made. Though these may be achievable, Brandon suggests these are superficial and fail to address deeper issues and learning.

Literature reviewed by DH (2000) highlighted how organisational culture can support or impede organisational learning. An organisation that prevents blame and cultivates openness, reflection and flexibility is more likely to successfully maintain a system-wide ‘learning loop’ where analysis of events, leads to change and embedding of change.

Douglas et al (2012) studied drivers for high performance in the public sector and identified organisations with ‘regenerational capabilities’. These authorities had strong tangible financial and professional ‘assets’, but also leaders who were dynamic and flexible, and focused on improvement.

Horwath and Morrison (2000) write that, where required, a focus on deep and cultural change is complex and sophisticated but brings better results than a superficial focus on policies and procedures. Internal and external drivers for change can compete against each other and be counterproductive- clarity can improve morale and organisational strength.

Munro’s literature review (2010) outlined factors that help and hinder improvement. She writes that a focus on critical change rather than policies and procedures is likely to bring about change, though this is difficult to achieve in the current top-down improvement culture of children’s social care. Further, a focus on systems rather than individuals could facilitate improvement by minimising the occurrence of blame, alienation and a reluctance to use professional judgement. Munro and Hubbard (2011) also stress the importance of focusing on system rather than individual error when looking at performance and improvement.

The Munro review of child protection (2011) points to the repeated failure of organisational learning. This systemic problem is linked to culture which prioritises adherence to top-down managerial process and regulation, instead of focusing on practice, learning and outcomes for families. Inspection can play a role in developing a learning culture.

Effective IT systems Ofsted (2015c) links good quality assessments with electronic recording systems that support good practice.

Gibson and O’Donovan (2014) explored the application of the Vanguard method (systems re-design process outlined by Seddon, 2003) in several local authority children’s services departments. By studying work flow they found practitioners spent around 80% of their time administering the national IT system. This minimised direct work with families and led to high re-referral rates as a result of unmet need. This system wastage was in turn linked with higher referral thresholds.

Appendix 2: Department of Education variables

Looked After Children in England including adoption variables (year ending 31st March 2015)

1. Children who returned to live with parents or relatives as part of a care planning process as a percentage of the sum of children who returned to live with parents or relatives as part of a care planning process and children who returned not as part of a care planning process

2. Care leavers aged 21 - percentage in education, employment or training

3. Care leavers now aged 21 – percentage in suitable accommodation

4. Children looked after: percentage of whom had a missing incident during the year

5. Children looked after: percentage who were away from placement without authorisation during the year

6. Offending by children who had been looked after continuously for at least 12 months: percentage convicted or subject to a final warning or reprimand during the year

7. Substance misuse by children who had been looked after continuously for at least 12 months: percentage identified as having a substance misuse problem during the year

8. Emotional and behavioural health of children looked after continuously for 12 months at 31 March for whom a Strengths and Difficulties Questionnaire (SDQ) was completed: banded SDQ score: normal

9. Emotional and behavioural health of children looked after continuously for 12 months at 31 March for whom a Strengths and Difficulties Questionnaire (SDQ) was completed: banded SDQ score: concern

Children in Need variables (year ending 31st March 2015)

10. Number of referrals in year ending 31 March 2015: Were within 12 months of a previous referral

11. Number of children who became the subject of a child protection plan during the year ending 31 March 2015 who became the subject of a plan for a second or subsequent time

Children’s social work workforce variables (year ending 30th Sept 2015)

12. Headcount: number of children in need per children’s social worker

13. Turnover Rate (%)

14. Agency worker rate (%)

TableA2a: Regression analysis: Ofsted ratings with DFE variables

Child outcome indicators

Looked After Children in England including adoption (year ending 31st March 2015)

Multiple R value

Significance F value

Children who returned to live with parents or relatives as part of a care planning process as a percentage of the sum of children who returned to live with parents or relatives as part of a care planning process and children who returned not as part of a care planning process

0.026301635 0.813412

Care leavers aged 21 - percentage in education, employment or training 0.05072353 0.652922

Care leavers now aged 21 – percentage in suitable accommodation 0.010736985 0.923729

Children looked after: percentage of whom had a missing incident during the year 0.35181615 0.001475

Children looked after: percentage who were away from placement without authorisation during the year

Offending by children who had been looked after continuously for at least 12 months: percentage convicted or subject to a final warning or reprimand during the year

0.187421372 0.138075

Substance misuse by children who had been looked after continuously for at least 12 months: percentage identified as having a substance misuse problem during the year

0.136037979 0.336242

Emotional and behavioural health of children looked after continuously for 12 months at 31 March for whom a Strengths and Difficulties Questionnaire (SDQ) was completed: banded SDQ score: concern

0.011249889 0.924211

Characteristics of Children in Need (year ending 31st March 2015)

Number of referrals in year ending 31 March 2015: Were within 12 months of a previous referral

0.042843958 0.700546

Number of children who became the subject of a child protection plan during the year ending 31 March 2015 who became the subject of a plan for a second or subsequent time

0.158428602 0.155141

Children’s social care workforce indicators

Children’s social work workforce (year ending 30th Sept 2015)

Headcount: number of children in need per children’s social worker 0.184723557 0.094562

Turnover Rate (%) 0.112336769 0.311959

Agency worker rate (%) 0.417130524 0.000118

The Child Outcomes Research Consortium (CORC) is the UK’s leading membership organisation that collects and uses evidence to improve children and young people’s mental health and wellbeing.

Founded in 2002 by a group of mental health professionals determined to understand the impact of their work, today our members include mental health service providers, schools, professional bodies and research institutions from across Europe and beyond.

We hold data relating to mental health and wellbeing outcomes of more than 400,000 children and young people in the UK, representing the largest data set of this kind worldwide.

Contact:[email protected]

improving children’s social care services - corc · 2017-01-31 · improving children’s social...

Documents