���������

1

Tools for performance assessment of hospitals

National Conference on health indicators

March 2005, Ljubljana,

Slovenia

Oliver Gröne, Technical Officer, HospitalsProgramme

WHO Regional Office for Europe

���������

2

Introduction- Different strategies and tools to improve

effectiveness and efficiency:- Health technology assessment- Evidence-based medicine- Clinical practice guidelines and pathways- Quality standards and indicators

- Different foci of quality assessment:- clinical effectiveness- patient safisfaction- patient safety …

- Overview on indicator initiatives and presentation of WHO project

���������

3

Functions of a hospital “Hospital care is multidimensional. It is a service provided by a coordinated group of professional, technical, and other workers […] The quality of care […] is affected by the adequacy of the hospital facilities and theirmaintenance, by the administrative and professional organization of the hospital, bythe competence of the personnel, and theinterpersonal relations among the staff as well as between the staff and the patients.

Any consideration of evaluation, therefore, must recognize the large number of factors involved in patient care.“(Mindel Sheps 1953)

���������

4

What influences performance?

The Hospital is a human-service organization:

- “Raw-material” human-beings, require direct contact between professional and patient

- Uno-actu principle: production and consumption simultaneaously

- Long-term perspective of results (intervention, discharge, HRQoL)

- Patient coproducer, contributes through expectations, compliance, lifestyle to long-term results.

Limitations for planning, standardization, and evaluation.

���������

5

Performance assessment

Performance assessment can be defined as a comparison of what has been achieved with what the system should be able to accomplish – that is, the best that could be achieved with the same resources. Performance is achievement relative to resources.(World Health Report 2000)

���������

6

International experiencesGermany

www.bqs-qualitaetsreport.de

���������

7

International experiencesGermany

Postoperative complications after apendicitis

���������

8

International experiencesUnited Kingdom

South east London: Queen Mary’s Sidcup acute trust Key Performance Targets

Overall, this trust has received a performance rating of 1 star. This means your trust is showing come cause for concern regarding particular key targets, or across many balanced scorecard indicators.

The trust achieved on 7 out of 9 key targets (e.g.):

- All cancers: 2 weeks wait- Outpatient and elective booking- Total time in A&E: 4 hours or less […]

The trust underachieved in 1 out of 9 key targets: - hospital cleanliness

The trust significantly underachieved in 1 out of 9: - financial management

���������

9

International experiencesUnited Kingdom

South east London: Queen Mary’s Sidcup NHS acute trust - junior doctor’s working hours

���������

10

International experiencesQuality indicator project

���������

11

International experiencesJCAHO

National patient safety and quality development goals

���������

12

International experiences

- Denmark: Quality Indicator Project- France: COMPAQH Project- Canada: Hospital Report. Ontario Hospital

Association- Scotland: NHS Quality Improvement- Netherlands: National Performance Indicator

Framework- Australia: Austrialian Council on Health Care

Standards…

���������

13

International experiences- Differences with regard to:

- Quality dimensions assessed and indicators selected

- Regulations for participation (compulsory, voluntary)

- Frequency of assessment (biannually, quarterannually)

- Risk adjustment (age, sex, case-mix)- Use and application of results (provider feedback,

public disclosure)

- Hospital functions (Mindel Sheps): - Little attention to patient-participation, staff health

and community integration

���������

14

WHO Performance Assessment Tool for Quality Improvement in

Hospitals (PATH)

To support hospitals in: - assessing their performance- questioning their own results, and- translating them into actions for

improvementThrough:- providing tools for performance assessment- Enabling collegial support and networking

among hospitals

���������

15

Patient

Patient- -centeredness

centeredness

Safety

Safety

Clinical Clinical EffectivenessEffectiveness

EfficiencyEfficiency

Staff Staff OrientationOrientation

Responsive Responsive GovernanceGovernance

PA

TH conceptual m

odel

���������

16

PATH features

- Comprehensive framework- Six interrelated dimensions of

performance- Support the move from measurement to

quality improvement actions

- Descriptive sheets: Background information to motivate the use of indicators and support interpretation

- Dashboard: Key message: Do not interpret indicators in isolation

���������

17

Dashboard illustration

Alternatives: proportion working days, absence frequency rate, median duration of absence, proportion of short-term absenceBackground: staff ratio and distribution of qualificationsStratify by professional category, and for short-term absence, whether medically justified or not

Related measures

Work-related injuries by type Temporary or agency work Excessive working hours Increase/decrease in length of stayPatient satisfaction/experience Budget for health promotion activities

Related performance indicators

Cost-efficiency Organizational commitment, job involvement, morale, job satisfactionTechnical quality of carePatient experience Staff ill-health Health promotion

Related performance concepts

Short-term and long-term absenteeismIndicator

���������

18

Patient centerednessCancelled surgical proceduresScore on patient perception/satisfaction surveyScore on interpersonal apectsScore on client orientation: information and empowerment

Responsive governancePerceived continuity through patient surveyWomen breastfeeding at discharge

Staff orientationTraining expenditures on average number of FTE staffBudget dedicated to staff health promotion activitiesShort and long term absenteeismPercutaneous injuries on average number of FTE staffStaff excessive weekly working hours

Set of performance indicators

���������

19

Clinical effectivenessPrimary Caesarean section delivery rateAppropriateness of prophylactic antibiotic useRate of readmission for selected tracer conditionsRate of admission after day surgeryReturn to ICU for selected procedures/conditions

SafetyMortality rates for selected tracers and proceduresFormal procedure to report and analyze sentinel eventsWork-related injuries (percutaneous injuries)

EfficiencyAmbulatory surgery useMedian length of stay for specific proceduresAverage inventory in stock for pharmaceuticalsWastage of blood productsOperating rooms unused sessions

Set of performance indicators

���������

20

How can indicators contribute to quality improvement?

- Before collecting data- Transmit values, sensibilize

- While collecting data- Improve information systems

- After computing indicators- Identify areas for improvement- Share best practices

���������

21

Before collecting dataTransmit values, sensibilize

- Bring in cultural changes and innovations- Ask the patient what he thinks (patient

surveys)→Make aware of accountability to the

patient- Support use of guidelines for antibiotic

prophylaxis→Make aware of evidence-based-medicine

- Assess informed consent→Make aware of the central place of

patient in the process of care- Report events in medical record

→Make aware of adverse events / safety concerns

���������

22

… I never thought about it before …

… I do not see how it this information can help …

… I cannot do anything about it anyway …

… I already know that we are good on this aspect …

… It is too sensitive, I prefer that we do not talk too much about it, I could get into trouble if I revealed it …Often, main question is not resources but

organisation & sensibilization

While collecting dataWhy is it not recorded now?

���������

23

After computing indicators

Key messageMeasurement is just the starting point to trigger reflection on own practices; the really challenging work only begins afterwards...

���������

24

After computing indicators72% patients are satisfied or very satisfied

� Q1: Is it high or low?� Q2: What reference point should I use to answer

Q1?• Myself in the past: last year = 63%• Other hospitals in Slovenia = 76 % on average• Other hospitals in the catchment area = 88% on

average� Q3: How can I explain the differences?� Q4: What process could explain differences?

What did I change compared to last year?� Q5: What specific area should I try to improve?� Q6: What did I learn from the other hospitals with

better results that I could apply to my hospital? What resources would I need?

� Q7: What is my target for next year?

���������

25Data from Quality Indicator Project

Hip infection rate

• Sudden increase in infection rates (hips, knees and hysterectomies)

• coincided with change in cleaning contractors

• contract discontinued

• rate fell back 0

2

4

6

8

10

12

14

1 2 3 4 5 6 7

Hospital Project mean

%

Quarter

Monitoring over time

���������

26

Cancellation rates before and after intervention in General Hospital Day Case Unit

01234567

Apr

il

June

Aug Oct

Dec

Feb

Apr

il

June

Aug Oct

Dec

DNA Rate

% Cancellations

Month

Intervention

Monitoring over time

���������

27

Monitoring hospital performance

ADCB

���������

28

Performance assessment contributes to quality improvement through:

- Information- How do we perform?- Where can we improve?

- Inspiration- Eye-opener

- Monitoring- the implementation process

- Comparison- Over time - Benchmarking with other hospitals or

units

Conclusion

���������

29

THANK YOU!

���������

30

Contact

Oliver GröneTechnical Officer Hospital Programme

WORLD HEALTH ORGANIZATIONEuropean Office for

Integrated Health Care ServicesBarcelona, Spain

Phone +34 93 241 82 70Fax +34 93 241 82 71

Email Ogr@es.euro.who.int