newsletter special edition - data and feedback

7/29/2019 Newsletter Special Edition - Data and Feedback

1/11

Children and Young Peoples Project

1For more information please see: http://www.iapt.nhs.uk/cyp-iapt

Children and Young Peoples

Improving Access to

Psychological Therapies

Outcomes and Feeedback

Bulletin - March 2013
http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/


2/11



Welcome and Introduction

Welcome to the CYP IAPT data bulletin, a special edition newsletter dedi-cated to data, feedback, outcomes and evaluation. This issue will give youan overview of what has been happening and what were setting out toachieve in 2013.

Id like to start by thanking all staff in the collaborating sites for their hard work

and dedication, particularly the data managers whose efforts have made this

possible. All phase 1 sites have successfully uploaded data during year 1 of the

project and the phase 2 sites have been incredibly busy getting their IT systemsin place and ready for the start of their Routine Outcomes Monitoring (ROM).

The collection and analysis of outcomes data is crucial in the CYP IAPT project.

Outcomes monitoring helps us to understand, quantify and demonstrate how

our treatment impacts on the lives of the young people using our services. This

information enables our young clients to reect on and track their progress over the course of therapy. It

also enables clinicians, young people and families to make decisions about what treatment is needed and

how helpful particular treatments have been, as well as helping individual practitioners and services review

their impact and make changes as necessary. This will ensure that we are able to deliver the best possible

treatment to children and young people.

We have learnt a lot during the rst year of the project. To harness this knowledge we surveyed part-

nerships about their experiences so that we can make improvements. You can nd a report from Anne

OHerlihy, the Extended Scope Project Lead, on page 3.

We also ran a consultation on data collection, the specication and the measures in the rst version of

the dataset. Thank you to everyone who took the time to complete the survey, your feedback has been

invaluable. Weve taken all of the comments into account, and a number of important changes have been

made to version 2 of the specication. Version 3, which includes a number of new measures, is due to be

implemented in April 2013 please see page 7 for more details.

Many of the collaboratives and partnerships have already been promoting the use of ROM in their services.

Hertfordshire, for example, ran a one-day training with Scott Miller, one of the pioneers of service user

feedback and outcomes in clinical practice. You can read a report about this day on page 5. The Salford

collaborative also ran a series of workshops on ROM for all its partnerships; you can nd further details

on page 6.

We are just beginning to analyse data from the rst three quarters from phase 1 sites. Please click here to

see the CORC report which outlines the results so far You can nd an overview of what weve achieved

to date, as well as the timeline for future uploads on page 10.

As ever, we very much welcome feedback and comment so please do not hesitate to get in touch.

Miranda Wolpert

National Informatics Lead,

CYP IAPT
http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/


3/11



Outcome measures in CYP IAPT

The CYP IAPT dataset identies a number of validated outcomes

measures tools and species when these should be used. Some tools

are completed by young people, some by parents and some by both.

We selected measures that have a proven value, are quick to com-

plete and are clinically useful, and which are compatible with and build

on the Child and Adolescent Mental Health Services Minimum Dataset

(CAMHS MDS).

In particular, the CYP IAPT programme set out to choose outcomemeasures that could be used to:

l Inform and support meaningful clinical conversations for

individual cases.

l Provide rigorous outcome data to allow consideration of service

level outcomes.

Measures were selected by the Outcomes and Evaluation Group and were agreed by the CYP IAPT Expert

Reference Group. Please see http://www.iapt.nhs.uk/cyp-iapt/ for more details.

Some measures were selected to be primarily used as clinical tools, others to be used to allow meaningful

comparison across cases. A range of options was provided for session by session tracking to allow clinicians

to select measures that were most relevant and meaningful collaboratively with young people and families.

Please share your top tips, advice and expertise on using feedback and outcomes tools

The next version of A Practical Guide to Using Service User Feedback & Outcome Tools to Inform Clinical

Practice in Child and Adolescent Mental Health is due to come out over the next couple of months. We

would really welcome contributions from clinicians with experience of using the tools. The next edition

will include tips, advice and expertise to help other clinicians and practitioners make best use of the CYP

IAPT feedback and outcomes tool kit to enhance clinical practice. Contributions could be (anonymised)

examples of where the tools had been particularly helpful, tips on helpful language to introduce the tools

to families and young people, ideas on feeding back information, using information from the tools in super-

vision etc. The next guide will be used for CYP IAPT and beyond to develop other useful resources and

training materials for clinicians and practitioners, all of which will be made available free of charge.

Please send contributions by the end of March 2013 to Duncan Law at [email protected]

The Editors reserve the right to edit and adapt contributions. Please title your emails CYP-IAPT feedback

and outcomes guide (and add your name).

Call for contributions
http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/


4/11



Anne OHerlihy, Extended Scope Project Lead, reports on feedback from

partnerships about the implementation of ROM

Over the last year, 18 CAMHS partnerships from year 1 have reported progress inthe implementation of routine outcome monitoring (ROM). The quarterly reportsfrom sites show that between 73-98% of CBT and parenting trainees are collectingand using ROM and the percentage of the wider clinical team also using ROM hasincreased from an average of 20% in September to 54% in December 2012. This

includes ve partnerships where 80-100% of their clinicians are reported to beusing ROM.

Concerns about ROMOne of the main difculties for ROM implementation is posed by IT infrastructure. All partnerships would

like an electronic system that allows data to be entered once with immediate upload to the patient recordsystem, and that can provide a data report with graphs in real-time.

Staff have expressed concerns about the time ROM use will add to their clinical sessions and the knockon effect this will have on waiting times, plus feeling overwhelmed by the additional paperwork. They are

also concerned about how the ROM data would be used by managers and supervisors, and a number ofclinicians remain unconvinced about the clinical benets. A further issue emerged about identifying the

most appropriate measures when working with young people who present with a range of issues and theclinicians tendency to opt for global measures that were not always clinically relevant.

New technologies and web-based entrySites are reporting that the capital funds given by the Depart-ment of Health for handheld technology and the changes to allowchildren and young people to enter data via the web will mitigate

some of the IT infrastructure risks, not least because IT depart-ments have engaged with clinical teams to support their bid.

Reported benets of using ROM on clinical practiceDespite the difculties and concerns, most partnerships describe a shift in culture and a growing under-standing of how ROM can benet clinical practice and service development. Foremost are the comments

from young people and their families, who value the opportunity to give immediate feedback and reporttheir concerns and issues regularly, as well as being able to visualise how they are doing on a graph.

Progress in using ROM

It means if we go off track or get a bit lost along the way, we can both

gure out how to nd the way back again.Young person from YoungMinds Very Important Kids Group


5/11



Services report that ROM supports clinical team discussions and decisions about the appropriate level ofcare needed by presenting an unbiased view of the young persons wellbeing, and has enhanced supervi-sory practice, further encouraging reective practice.

The service has been able to demonstrate to commissioners

collaborative engagement with young people and their families, which

has contributed to securing the contract for the next 3 years.

Hertfordshire CAMHS partnership

Reported benets of using ROM on service developmentROM is being used to support service design and development plans. Some partnerships are in discussionswith commissioners about how they can build outcomes into a review of their performance. The patientexperience measures such as how are you doing SRS and Chi are being used to inform consultations withyoung people and guide service development.

Developing practitioner skills at the same time as changing the way

services are delivered through ROM has kick started a change in

culture.

Gloucester CAMHS

Feedback is that they [trainees] are nding it very useful and it has

helped improve patient outcomes and end treatment at an agreed

time, based on the presenting outcome data.

Derby CAMHS

[It] makes us feel like this is more of a shared experience between us

and the clinician... like were in this together.

Young person from YoungMinds Very Important Kids Group

It supports us to feel more in control of its direction to understand when

things are working, when things are not and what we can do about it.

Young person from YoungMinds Very Important Kids Group


6/11



Example of service experience

Duncan Law, Hertfordshire Specialist CAMHS, talks about his partnershipsexperience of the implementation of CYP IAPT

It has certainly been a huge challenge rolling out the service user feedback and out-

comes framework across Hertfordshire, but the data is starting to come through

thanks to a great team effort, in particular from Amy Turner, a very committed and

competent data manger, and CYP IAPT Project Manger, Janet Arris.

The ow of data is testament to the way our services have begun to embrace the CYP IAPT philosophy,supported by trainees, supervsiors and service managers. Commissioners who see the value in having clini-

cians trained to use service user feedback and outcomes in clinical work have supported additional training

and clinical discussion forums. At a local team level, Siri Wooster, Divisional Manager of East Hertfordshire

Specialist CAMHS, where the Hertfordshire CYP IAPT roll out began, has ensured that CYP IAPT and

transformation have been high on the agenda and, most importantly, allowed time for discussion within the

team about the anxieties, practicalities and clinical issues around all aspects of service user feedback and

outcomes.

Just over a year on, we have made good progress but still have a way to go to get and keep clinicians on

board and spread the message that feedback is about positive collaborative practice and not just question-naires. The feeling in the partnership is that all this effort will be worth it.

In January Hertfordshire CAMHS ew over Scott Miller, one of the pioneers of service user feedback and

outcomes in clinical practice, to present an Outcomes Masterclass. Simon Shattock, a Systemic Family

Therapist from Hertfordshire Partnership NHS Foundation Trust, attended the training. Heres what he

thought of it.

I was rather sceptical about what this day would achieve for me as I have been using regular outcome

monitoring in my work since participating in the CYP IAPT course. The day helped me understand how

to use the session by session tools in a meaningful way to achieve feedback on the therapeutic relation-

ship. Scott Millers work challenges the orthodox view about the dominant discourse of evidence-based

practice. This is not to say we should not undertake treatments that are useful to children and families, but

should consider that real change with clients should happen in the rst 4-6 sessions, and that the greatest

predictor of positive outcome for clients is the therapeutic relationship regardless of the model. The most

effective therapists are those who seek negative feedback from clients earlier on in treatment. and then

tailor his or her approach to the client accordingly. I have used his ideas in therapy with a teenage girl and

was able to nd out that she did not like the silences in the room between us and would prefer if I talked

more and asked lots of questions. I have altered my position as a therapist accordingly and hope now to be

more effective in my therapy now.

Outcomes workshops


7/11



Recognising routine outcomes monitoring as a key driver for the successful implemen-

tation of CYP IAPT, the Manchester Collaborative ran a series of workshops, delivered

to all Partnerships.

The key aims were to:

l Raise awareness and support implementation of ROMs in practice

l Ensure a baseline knowledge of CYP IAPT across the workforce

l Introduce and increase awareness of nationally agreed CYP IAPT ROMs

l

Provide an opportunity to experience and test application in practicel Identify relevant internal/external networks and the necessary resources for

successful implementation

The workshops were delivered over a full day with a total of152 staff members across

the participating partnerships. The attendance of senior managers and clinicians was

particularly valuable in ensuring visible strategic and clinical ownership and commit-

ment to the successful implementation of CYP IAPT. Whilst challenging, the implemen-

tation of ROM is an exciting part of the process to support service transformation in

CAMHS and all should be commended for starting this work.

Thanks go to all the partnerships within the Manchester Learning Collaborative fortheir support, recognising that continued successful implementation will to a great

degree depend upon the goodwill of the committed workforce. The implementation of

ROMs requires a cultural shift for many organisations; it is a process that will take time.

There is clearly a signicant degree of enthusiasm for the use of ROMs in practice.

Gill Walker Service

TransformationLead; Manchester

collaborative

Barry Nixon NHS

North West CYP

IAPT Lead

The session made me excited & passionate about the transformation

of our service.

Trainee attending the workshop

Really informative, much better understanding of what IAPT means in

real terms and outcome measures.


Helped me understand the national context of IAPT and the use/

importance of routine outcome measures.


Outcomes workshops


8/11



Changes to the dataset

Version 2Following feedback from the consultation we have made several changes, and version 2 of the dataset wasreleased in December 2012.

Changes include:

l Addition of a rened assessment questionnaire (Current View) which includes additional presenting

problems, contextual factors, complexity factors and educational informationl Updating of event records to improve the quality of data recorded about resource need, for instance

indirect contacts are now captured, event duration, and how many professionals were present atappointments

Version 3Version 3 of the dataset will be released in April 2013, and the following measures will be added (subject

to nal sign off by measure developers) to track change in:

l Family relationships (SCORE 15)l Eating disorders (EDE-Q)l Parenting ( Parental Efcacy Scale)

l Management of learning disability (SLDOM)l General wellbeing (Warwick-Edinburgh scale)

In addition, the group are looking at whether a) some costed measures can be supported (but this requiresdiscussion on copyright issues); b) whether CORE YP and CORE OM can be used (but this requires discus-sion around data sharing.)

Consultation on the CYP IAPT dataset

To see what those working at all of the CYP IAPT sites thought of the data we carried out a consultationin July 2012 about the process, data specication and measures. A full report on the consultation can be

found here.We also asked for feedback via emails and in meetings. The majority of responses to the formalconsultation came from CAMHS clinicians (80%) and below is a selection of comments.

A child was able to highlight in a session the issues we were NOT

talking about more easily than if we had not had it clearly identied on

the SRS/ORS Family members like the visual aspects of this process -it shows the change not just talking

Feedback from clinician
http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/silo/files/cyp-iapt-measures-consultation-271112.pdfhttp://www.iapt.nhs.uk/silo/files/cyp-iapt-measures-consultation-271112.pdfhttp://www.iapt.nhs.uk/silo/files/cyp-iapt-measures-consultation-271112.pdfhttp://www.iapt.nhs.uk/silo/files/cyp-iapt-measures-consultation-271112.pdfhttp://www.iapt.nhs.uk/cyp-iapt/


9/11



Data Uploads and Analysis

Jenna Bradley

Senior Research

Ofcer and Project

Co-ordinator

Rachel Argent

Research Assistant

Andy Fugard

Research Lead

The CORC Central Team

Miranda Wolpert

CORC Director

The CORC Central Team provides a range of support for the collection, analysis and reporting of data, aswell as general support for members. The team provide site visits, and telephone and email support wheremembers have queries, need information, or have problems with implementation.

The CORCNexus consortium have been commissioned by the DH to collate and analyse data for CYPIAPT and support and train sites to allow appropriate data ow. CORCNexus is a collaboration between

the CAMHS Outcomes Research Consortium (CORC), the CAMHS Evidence Based Practice Unit (EBPU),and the secure data storage company MegaNexus (MN). Every quarter, outcome and activity data ows to

the secure data store, as outlined below.


10/11



Since January 2012 a total of 789 children, young people and/

or their families have been seen by CYP IAPT trainees in phase1 sites or others using the CYP IAPT measurement approach.

Six hundred and sixty six of these cases are still open and 121

have been closed (as of 30th September 2012).

What are we doing with the data?The CYP IAPT data will be used by the central CYP IAPT

project team to inform our understanding of the outcomes

achieved by young people and their families after accessing

services. We intend to make appropriate data publically avail-

able as soon as possible, but at this stage in the project we arefocussing on reviewing data quality and identifying any issues

or areas for further work.

CORC have carried out preliminary analysis of very high level summary intake data. Click here to view

the CORC report. Please be aware that the data analysis must be taken as provisional until all data quality

checks have been complete. Moreover, when considering information such as presenting problems, com-

plexity, participation in education or training, it is important to note that most of the data is from trainees

working with less complex cases which are appropriate for their skills and training as they proceed through

the course. Trainees have been working with cases that suit the therapies in which they are being trained. It

is very likely that aspects of the data will change once all practitioners in CYP IAPT sites use the outcomemonitoring.

Number of cases by presenting problems of cases

seen as identied at rst contact

The frequency of child and young person-rated session by session measures completed at rst time

point (Q1-Q3 2012).
http://www.iapt.nhs.uk/cyp-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/routine-outcome-monitoring-as-part-of-iapt/http://www.iapt.nhs.uk/cyp-iapt/


11/11


Data collectionperiod

Datasetversion

Phase 1sites

Phase 2 sites Centralsubmission period

End of year col-lection (Jan to

March 2013)

v2 All service Trainees only 1st to 15th April 2013

Verication of the outcomes tools proposed in the data set

consultation and development of associated guidanceDec 2012 to Feb 28th2013

Q1 (Apr to Jun)2013

v3 All service Trainees only 1st to 15th July 2013

Q2 (Jul to Sep)2013

v3 All service Trainees only 1st to 15th Oct 2013

Q3 (Oct to

Dec) 2013

v3 All service Trainees only. All service

gearing up 1st Dec 2013

1st to 15th Jan 2014

The CYP IAPT project has achieved a great deal in a very short time-frame, but theres lots more work to

be done. Below is a schedule for data uploads going forward.

Data collection going forward

Any Questions for CORC?

We very much welcome feedback, so please get in touch with comments and/or questions. Our email is

[email protected].

CYP IAPT YouTube Channel

On the CYP IAPT YouTube channel you can nd a range of vid-

eos of children and clinicians talking about routine outcomes

monitoring and service user feedback. The latest video is of

Duncan Law discussing how goal setting and sacking your ther-apist has encouraged better service user participation.

Please click here to view the videos.
http://www.youtube.com/channel/UCVr_XGnjA229P5bUgfZ-y3whttp://www.youtube.com/channel/UCVr_XGnjA229P5bUgfZ-y3w

newsletter special edition - data and feedback

Documents