1

People’s voice pilots

Final evaluation report

January 2013

Agenda item 11

Paper No: CM/01/13/10

Annex A

2

Background

In between inspections, CQC gathers the views of individual members of the public about the quality and safety of care services.

In its 2011/12 public communications strategy the public communications team proposed a ‘Tell us about your care’ campaign to encourage people to feedback to CQC about their individual experiences of care.

There was concern that this approach would stimulate unmanageable volumes of information, much of which would be outside our regulatory remit.

In addition there were no processes in place for identifying and reporting back on regulatory action taken in response to info received from the public.

3

Background

A cross-directorate project group led by the public communications team carried out three related pilot projects to test the following hypotheses:

Information we receive from the public is valuable

We can stimulate the amount of information we receive without increasing wastage (information of no regulatory value)

We can cope with the scale of information generated through stimulation

The project looked solely at individual feedback about specific services.

It did not attempt to explore the value of individual feedback received via other methods (e.g. through third party websites such as NHS Choices and Patients Opinion) or the value of collective feedback of groups (e.g LINks OSCs, local CVS groups etc).

4

The pilots

Two of the three pilots were carried out in partnership with voluntary organisations: Relatives and Residents Association (R&RA) and Patients Association (PA)

These orgs were targeted because they had reported receiving concerns about services that would be of value to CQC and had relatively small volumes so we felt comfortable we could manage the anticipated volumes

We undertook joint marketing and supported their helplines with guidance and training

R&RA and PA completed CQC ‘Share Your Experience’ webforms on behalf of callers to their helplines.

The third pilot used a variety of different channels to stimulate information from the public in the North West of England.

Advert and leaflets in GPs surgeries in BoltonFace to face attendance at local carer and older people engagement groups and events in WarringtonInterview with a compliance manager on local radio and articles in local newspapers in Lancashire Distribution of CQC materials following inspections of care homes with nursing in LancashireDistribution of CQC materials to day care centre clients in LiverpoolCQC article and leaflet insert in Lesbian & Gay Foundation magazineAdvertising on local radio in Lancashire

A phased approach was used in the North West to manage anticipated volumes

5

Approach

In order to understand the value of information received from members of the

public each piece of information received through the pilots was evaluated by an

inspector using a pre-agreed matrix, and by the Intelligence team for use in the

QRP.

Inspectors were also asked to report on any action they took in response to the

information received.

Possible actions a CQC inspector could take following receipt of a piece of informationNot serious enough to trigger any immediate action but information will be used when planning the next inspectionContact provider Check with another agency whether incident known and any action takenBring forward scheduled inspection Undertake responsive inspectionRaise with local authority as a safeguarding incident – safeguarding investigation underway – awaiting outcome

NCSC provided a customer support ‘track and trace’ function to inspectors prompting them to complete the evaluation

6

Success criteria

750 extra Share Your Experience forms per month (all pilots total)

More than 50% of feedback forms are included in the Quality and Risk Profile

Information source is of high value to CQC if 50% of information was rated valuable by Operations

Information source is of clear value to CQC if 25% of information was rated valuable by Operations

7

Findings

8

Volumes of information received

We received 62 forms a month through the pilots (average). This is a 10% increase on BAU.

8 a month from R&RA (45 in total across 6 months)

28 a month from PA (168 in total across 6 months)

26 a month from the North West activities combined (154 in total – across 6 months)

Lower volumes received than anticipated – across all pilots

9

Volume of information received by sector

North West pilot

Relatives and Residents Association

Patients Association

Care home 61% 78% 19%

NHS 26% 7% 75%

Dom care agency

3% 9% Less than 1%

Dental practice 4% 4% 5%

Independent healthcare

4% 2% Less than 1%

Prison healthcare

2% 0% 0%

10

Value of information received

45% (the majority) of information we received through the pilots didn’t prompt any immediate action but was judged by inspectors to be valuable intelligence to inform monitoring and inspection planning

10% prompted contact with the provider

4% prompted contact with another agency to check whether the incident was already known about and action taken

2% prompted a safeguarding referral

2% prompted a responsive inspection

3% brought forward a scheduled inspection

26% of information received was positive comments about care

Only 11% of information received across all pilots was of no value

11

Additional benefits of the pilot activities

Increased public awareness of CQC in activity areas

North West pilot activity has resulted in a 7% increase in people being aware they can share their concerns with us direct

Joint communications and marketing materials with the voluntary organisations involved in the pilots have helped us to reach a key audience which we would otherwise not have reached.

12

Conclusions

Information from the public is valuable All pilots have clearly shown that information we receive from the public is valuable

R&RA and Patients Association are both sources of valuable information

We can stimulate volume without compromising quality We have shown that we can stimulate the amount of information we receive and still

retain the quality (low wastage of non-valuable information that has been processed)However we have not stimulated volumes to the extent we anticipated

We can cope with the scale of information generated throughstimulation We have shown that we can cope with the scale of information generated when we

proactively seek to increase volumesWe have also shown we can increase public awareness without being inundated with

huge volumes of informationHowever – we still don’t know at what volumes we’d struggle to cope

13

Next steps

14

Next steps

Enable MI reporting on value of information received on Share Your Experience web forms, via CRM change (due April 2013)

Continue work with Patients Association and Relatives and Residents Association (phase 2 projects start February 2013 tbc) We will increase marketing, PR and engagement to increase the volume of information we receive via these organisations. With Patients Association we intend to particularly focus on gathering the experiences of older people.

Work with other third sector organisations, testing their value as an information source (from March 2013)

Use learning from North West pilot to inform marketing and awareness-raising activities across England

GP provider welcome packs (April 2013)TV advertising in GP practices reception areas (April 2013)Distribution of ‘What standards to expect’ leaflets to care homes (March 2013)Other sector provider welcome packs (from May 2013)

15

Next steps continued

Project to improve customer service for members of the public contacting CQC (will help us explore anecdotal feedback about negative experiences)(January – May 2013)

Explore what action we can take to address people’s fears that they, or their loved one, will experience negative consequences as a result of sharing their experiences of care with us. (February 2013)

Suggest the People’s Voice pilots work to inform:

Information management projectQRP improvement projectWork to develop role of Ops IntelligenceNCSC planning

16

Appendix: 1 Value of information received

CQC action prompted R&RA Patients Assoc.

North West (pilot identifiable)

North West (all)

ALL

Use to inform next inspection / provides a marker for future monitoring

42% 70% 25% 42% 45%

Contact provider to raise this issue and seek their response

7% 7% 10% 12% 10%

Check with other agency whether incident known and any action taken

9% 1% 2% 2% 4%

Bring forward scheduled inspection 7% 1% 2% 2% 3%

Responsive inspection 4% 2% 1% 2%

Raise with local authority as a safeguarding incident – safeguarding investigation underway – awaiting outcome

1% 2% 4% 2%

Other 1%

No value at all 18% 12% 5% 5% 11%

Positive feedback 13% 6% 53% 32% 26%