the acute co-ordination centre for the nhs patient survey programme use of inpatient survey results...

The Acute Co-ordination Centre for the NHS Patient Survey Programme

Use of Inpatient Survey Results

Results of an online survey of acute hospital trust survey leads for 2005 inpatient survey

The Acute Co-ordination Centre for the NHS Patient Survey Programme: Online survey of use of Inpatient Survey

Results

Background

• Objectives

– To establish how trusts currently use the survey results– To identify ways in which we might be able to help them:

• to use the results more effectively • and to disseminate them more widely

• Methodology

– The survey was conducted online– All 2005 survey leads (n.169) were e-mailed in early November

2006 to invite them to take part (with follow up mailing to replacement contacts, where notified)

– Two targeted reminders were sent to non-responders at weekly intervals to encourage participation (with a final, third, reminder included in a routine e-bulletin from the Acute Co-ordination Centre)


Results

Response Rates

– 76 trust survey leads completed the questionnaire, giving a “raw” response rate of 45%

– After adjustment for known “undelivered” e-mails (i.e. “bounce-backs” due to invalid e-mail addresses/rejection by SPAM filters) response rate was 47%

– Response rate amongst “in-house” trusts was marginally higher than for those using a contractor, hence all quantitative results have been adjusted at the analysis stage to give appropriate weight to the views of each group

– The job functions of those responding, and giving a job title, were: • PALS/PPI 40%• clinical governance managers 22%• senior managers, including clinical directors 16%• audit and information staff 11%• others 11%.

– A chart showing dates of returned questionnaires is appended (Appendix 1)


Results

Key findings: Current use/dissemination of results

• All survey outputs are widely used. However, the initial results (from a contractor or in-house team) are used the most. This is probably because they are:– available earlier– easier to analyse/manipulate

• Most trusts (82%) do additional analysis of the data – particularly looking at trends and comparing across sites/specialties or with other trusts– nine out of 10 also use, and value, the open-ended patient comments

• Benchmark reports are generally found more useful than the national tables – mainly because the latter are felt to be difficult to find and manipulate

• All results are widely disseminated throughout the trust, with presentations made to a number of groups and findings also shared through intranet, newsletters and various other fora


Results

Key findings: Help in analysing/using the data

Trusts would particularly welcome:

• Case studies (how others have used the data)

• And [statistical?] help in:

– comparing with other trusts– understanding trends– understanding the weighting used (e.g. in benchmark

reports/national tables)


Results

Key findings: Help in publicising/disseminating the data

The most popular suggestions were:

• Generic fact-sheets for patients and staff explaining the purpose/use of patient surveys

• Templates for fact-sheets and presentations for trusts to slot in their own results

• It was also felt by some (around 4 in 10) that the presentation of the results on the Healthcare Commission website could be improved, e.g.:

– More historical comparisons– More explanation of results– Better use of graphics– In general, make the results easier to find


Results

Survey Outputs Used (%)

3%

15%

20%

61%

68%

86%

26%

81%

National tables

Benchmarkreport

Data/ reportfrom in-house

team

Data/ reportfrom external

contractor

Used at allUsed most

Which, if any, or the survey outputs listed below does your trust use?Which one of these outputs do you use the most?


Results

Are any additional analyses carried out?

21%

30%

14%

4%

31%

No, but there is noneed

No, but it issomething would liketo do

Done by contractor(trust not involved)

Done by bothcontractor and withintrust

Done within the trust(contractor notinvolved)


Results

Types of additional analysis done(% mentioning each type of analysis)

6%

7%

17%

18%

27%

28%

33%

43%

51%

55%

88%

Other

Health status

Level of disability

Gender

Age group

Ethnic group

Other performance data

Specialty

Other trusts

Hospital site

Previous years

Based on 82% of trusts who do additional analysis


Results

Use of patient comments

6

1

9

10

22

23

23

Other

To congratulate staff

Used in training

Link to other data sources

As input to actionplans/strategies

Identify themes/areas forimprovement

Used inreports/publications/other

dissemination

Asked of those who analyse/use the open-ended comments

89% of trusts say they use the patient (open-end) comments. The number of trusts mentioning each “theme” is as follows:


Results

Use of patient comments: some examples

• “These are the 'diamonds' in the patients survey because they make the statistics real and meaningful…. The comments help stop any tendency to challenge the data on the stats.”

• “Break down into specialty and then into different themes – food,

cleanliness, staff attitude & service provision positive/negative, medication issues, clinical issues, information & communication.”

• “ We produce posters with comments on for use at action planning workshops.”

• “We use the comments for a wide range of training opportunities such as Trust induction, Jnr Docs induction, Student Nurse Training etc.”

• We circulate the comments widely and use examples in poster displays to raise awareness of the survey.”


Results

Is there a dedicated person within the trust responsible for analysis and reporting of

results?

47%

7%

7%

39%

No

Other(s), not surveycontact

Survey contact &other(s)

Survey contact only

“Other(s)” are mainly:

PALS/PPI

Senior management inc. clinical directors

Audit & information staff


Results

Ease of understanding & usefulness: Benchmark Reports & National Tables

58

29 25

4740

51

42

3 2

10

20

50

19 13DK/NS

Not at alleasy/useful

Not veryeasy/useful

Quiteeasy/useful

Veryeasy/useful

Benchmark Reports National tables

Understanding Usefulness Understanding Usefulness


Results

How are the Benchmark Reports used?

12

13

27

44

Comparisons over time

Feed back inreports/ publications/ presentations

To identify areas forimprovement/ action

Comparisons with other trusts/ atnational level

Asked of the 74 trusts saying they found the benchmark reports very or quite useful

97% of trusts say they find the Benchmark Reports very or quite useful. The number of trusts mentioning each “use” is as follows:


Results

How could the Benchmark Reports be improved?

3

4

6

7

Simpler explanations

Receive results sooner

Changes/ improvementsto presentation

More analysis - e.g.ethnic group, elderly

patients

Asked of the 74 trusts saying they found the benchmark reports very or quite useful

The number of trusts suggesting each type of improvement is as follows:


Results

How are the National Tables used?

2

5

5

24

Comparisons overtime

Use in internalreports/ feeback

To identify areasof interest/ forimprovement

Comparisons withother trusts/atnational level

Asked of the 51 trusts saying they found the national tables very or quite useful

67% of trusts say they find the National Tables very or quite useful. The number of trusts mentioning each “use” is as follows:


Results

How could the National Tables be improved?

1

5

4

7

Group by region

Make easier tomanipulate/ navigate

SUGGESTIONS:

Other outputs more useful(e.g. benchmarks/ contractor

reports)

Too much detail/ difficult tofind/ manipulate

CRITICISMS:

Criticisms asked of the 15 trusts saying not very useful. Suggestions asked of 66 trusts - 15 saying national tables not very useful + 51 saying they found them very or quite useful

The number of trusts making detailed criticisms, or suggesting improvements, is as follows:


Results

Who sees initial trust results from contractor/in-house team?

28%

60%

61%

70%

68%

70%

77%

90%

88%

92%

11%

33%

21%

13%

13%

13%

13%

17%

2%

5%

4%

62%

39%

19%

26%

17%

19%

17%

6%

8%

7%

4%

27%Others

HR

Team leaders

Other staff

Patient groups

PALS

Comms

Trust Board

Senior Managers

Clinical Gov.

PPI Lead

Yes No DK/NS


Results

Who sees the benchmark report?

24%

46%

53%

73%

71%

76%

85%

86%

84%

87%

11%

26%

21%

14%

2%

6%

7%

6%

4%

3%

4%

68%

50%

33%

33%

25%

23%

17%

9%

10%

13%

9%

21%Others

HR

Team leaders

Other staff

Patient groups

PALS

Comms

Trust Board

Senior Managers

Clinical Gov.

PPI Lead

Yes No DK/NS


Results

Who sees the national tables (spreadsheets)?

13%

21%

25%

24%

30%

42%

44%

43%

49%

59%

28%

34%

34%

25%

27%

29%

22%

29%

28%

19%

16%

63%

53%

45%

50%

49%

41%

36%

27%

29%

32%

25%

9%Others

HR

Team leaders

Other staff

Patient groups

PALS

Comms

Trust Board

Senior Managers

Clinical Gov.

PPI Lead

Yes No DK/NS


Results

Summary of presentations made to different audiences

(by contractor and/or in-house team)

32%

68%

71%

92%

88%

Other

PPI forums

Wider staff groups

Specific meeting e.g. clinicalgovernance or PPI

Board/senior management inc.clinical directors

The following two slides show the break down between presentations made by contractors and those made by in-house teams


Results

Presentations made by external contractor to different audiences

12%

21%

32%

33%

63%

72%

61%

58%

58%

26%

16%

18%

10%

9%

11%Others

PPI forums

Wider staff group(s)

Specific meetingse.g. clinical

governance/PPI

Board/seniormanagement, inc.clinical directors

Yes No DK/NS

Based on those receiving data/report from external contractor (unweighted base 57 trusts)


Results

Presentations made by internal staff/ trust survey lead to different audiences

66%

68%

88%

83%

17%

12%

13%

8%

6%

54%

22%

19%

4%

11%

29%Others

PPI forums

Wider staff group(s)

Specific meetingse.g. clinical

governance/PPI

Board/seniormanagement inc.clinical directors

Yes No DK/NS


Results

Other ways of disseminating results to staff

64%

74%

84%

10%

26%

21%

4%

58%

10%

5%

12%

32%Other

Newsletter

Intranet

Discussion at teammeetings

Yes No DK/NS


Results

Ways of sharing results with public/patients

26%

53%

64%

80%

16%

45%

22%

13%

3%

69%

29%

25%

23%

17%

15%Other

Posters/ leaflets intrust buildings

Trust website

Local media

PPI forums

Yes No DK/NS


Results

Other groups that results are shared with

26%

32%

45%

17%

31%

22%

20%

73%

43%

46%

35%

10%Other

Other local trusts

Overview &scrutiny

committees

Strategic HealthAuthority

Yes No DK/NS


Results

Other ways in which shared with different groups

(number of trusts mentioning each)

2

3

5

3

3

8

10

PPI forums/ committees

Presentations e.g. patient representatives,governors, local user/ voluntary groups

Written media e.g. posters, newsleters

PUBLIC/ PATIENTS:

Electronically

Presentations

Written media

Briefings/ group meetings

STAFF:

Based on those saying results disseminated to “other” groups


Results

Other ways in which trusts share the results

• “..used for focus group work..”

• “We have an annual Patient and Public awareness day and the results are always displayed then. There is always someone on hand to discuss any aspects of the survey that people wish to raise.”

• “ …large workshop events with key stakeholders from PCT, SHA, local patient forums and user groups including patient groups to discuss the results and prioritise areas for action.”

• “…Patients’ Panel newsletter that is circulated throughout local community groups, wards and departments in the hospital, all local GPs and members on the Panel.”

• “In-house magazine.”

• “Monthly communications bulletin to all staff.”


Results

In what ways do you use the information?

9%

69%

84%

91%

99%

99%

100%

Other

To publicise improvements topatients/public

Congratulate staff on improvements

In self-declaration to HCC

Input to action plans

To monitor progress

To identify areas for improvement


Results

Levels of action planning within the trust using the survey results

1%

2%

16%

49%

65%

77%

92%

Don't know

Other

Individual level

Site level

Team level

Departmentallevel

Strategic (trust-wide) level

Based on the 99% of trusts who use the results in action planning

In most trusts (91%) action plans are followed through with a dedicated person, or persons, leading, monitoring and checking progress.


Results

Ways in which improvements publicised to patients/the general public

8%

37%

51%

57%

72%

Other

Public meetings

Trust website

Local media

Written material,e.g. newsletter

Based on the 69% of trusts who publicise improvements identified through the results to patients/the general public


Results

Do you specifically mention that improvements have been identified through

the survey results?

8%0%

57%

31%

Always

Sometimes

Never

DK/NS

Based on the 69% of trusts who publicise improvements identified through the results to patients/the general public


Results

Preferred options for help in analysing/interpreting/reporting on findings

27%

29%

49%

68%

79%

84%

Help in understanding nationaltables

Help in understanding benchmarks

Help in understanding weighting

Help in understanding trends

Comparisons with other trusts

Case studies on how to use

(% mentioning each option)


Results

Preferred options for help in disseminating results

18%

30%

32%

39%

52%

52%

57%

65%

Press releasetemplate

Briefing notes to HRdept

Newsletter template

Briefing notes forComms

Presentation slidetemplate

Factsheet for staffabout patient surveys

Factsheet template fortrust's results

Factsheet for patientsabout patient surveys

(% mentioning each option)


Results

Usefulness of the survey section of the Healthcare Commission website

2%

9%

62%

24%

3%

Very useful

Quite useful

Not very/at alluseful

Not awareof/neveraccessed

Not stated


Results

Do you think the Healthcare Commission’s presentation of survey results on the site could

be improved?

2%

42%

17%

39%Yes

No

Don't know

Not stated

Based on those aware of this section of the site


Results

Agreements with suggested improvements to presentation of survey results on Healthcare

Commission website

51%

63%

67%

74%

More use ofgraphs inpresenting

survey findings

Make it easier tofind the results

More explanationof results

Include historicalcomparisons

Based on the 39% of respondents who felt the Commission’s website could be improved


Results

Appendices

1. Return of completed questionnaires

2. Who analyses and reports on the results


Results

Return of completed questionnaires

0

10

20

30

40

50

60

70

80

90

Num

ber

of

quest

ionnair

es

First mailing

Replacement contacts

First reminder

ACC e-bulletin

November 2006


Results

Who analyses and reports on the results?

9

9

13

18

33

13

16

22

8

24

Other staff in trust

Audit &information staff

Seniormanagement*

Clinicalgovernance

PALS/PPI

Main survey lead Others

*inc. clinical directors

the acute co-ordination centre for the nhs patient survey programme use of inpatient survey results...

Documents