Dept Social Medicine

Health System Performance Management

quality for better or for worse

• Niek Klazinga, April 27 2010• London LSE/NHS Confederation

Dept Social Medicine

Reasons for international comparisons on performance related to quality of care

• Accountability

• Strategic decision making

• Learning/improvement

Dept Social Medicine

Table 1.1 Conditions under which performance measurementTable 1.1 Conditions under which performance measurement is possible and problematicis possible and problematic

Performance measurement possible Performance measurement problematic

An organization has products An organization has obligations

and is highly value-oriented Products are simple Products are multiple An organization is product-oriented An organization is process-oriented Autonomous production Co-production: products are

generated together with others Products are isolated Products are interwoven Causalities are known Causalities are unknown Quality definable in Quality not definable in performance indicators performance indicators Uniform products Variety of products Environment is stable Environment is dynamic

Source: Managing performance in the public sector. De Bruijn H. (2002), p. 13

Dept Social Medicine

Measurement and Management

• A measure on quality of care does not exist independently• validation is dependent on the use/purpose• Validation is dependent on the boundaries of the universe it

is supposed to signal upon• Measures need to be integrated in management/decision

making mechanisms of government, financiers, managers, professionals and patients

• Apart from reliability and validity, relevance and usefullness are important criteria for selecting quality measures

• As a consequence the users should be involved in the development of the measures

Dept Social Medicine

Health systems performance management

• Health Systems (scope , components and boundaries)

• Performance (objectives on various dimensions such as health results, efficiency and equity – measurement challenges)

• Management (heterogeneous national governance models, integration of performance indicators in management mechanisms)

Dept Social Medicine

Related policies

• Health system sustainability

• Integrated care

• Prevention

• Patient Centered Care

• Equity

• Regulated market

• Incentive structures

7

Conceptual Framework for OECDHealth Care Quality Indicator(HCQI) Project.

(shaded area represents the current focus of the HCQI Project)

Source: Arah OA, et al. A conceptual framework for the OECD Health Care Quality Indicators Project. International Journal Quality Health Care. 2006; Sep 18; Suppl.1:5-13.

7

Dept Social Medicine

Combining various rationalitiesCombining various rationalities

• Public Health• Medicine• Management sciences• Economics• Societal / individual values

Dept Social Medicine

Performance indicators and benchmarking related to mortality data

- avoidable mortality (health system level)

- standardized mortality rates (hospital level)

- limitations of death statistics

Dutch hospital standardised mortality ratios 2001-3(HSMRs) vs hospital

(standardised for age, sex, urgency/readmission, LOS within 50 CCS groups leading to 80% all deaths,excluding small hospitals and those with poor data recording, using year 2000 standard)

0

20

40

60

80

100

120

140

96 35 68 14 83 81 51 25 89 50 103 3 52 44 85 5 78 36 12 100

72 94 13 104

65 33 34 95 101

39 93 82 79 23 61 47 37 20 87 97 45 31 107

19 98 54 102

Hospital number (assigned by BJ)

HSM

Rs (9

5%

CIs

) 2001-2

003

Dept Social Medicine

Performance indicators and benchmarking related to cancer care

• CONCORD study

• Eurocare

• Limitations of cancer registries and limited possibilities for linking with other (administrative) data-bases

1. Programme. 2. Survey.

5.7.1. Cervival cancer screening, percentage of women screened aged 20-69, 2000 to 2006 (or nearest year)

5.7.2 Cervical cancer five-year relative survival rate, 1997-2002 and 2002-2007 (or nearest period)

24.5

38.5

41.7

60.6

62.2

64.0

65.3

69.4

69.6

70.5

70.6

71.0

72.4

72.8

75.6

78.6

79.4

83.5

0 20 40 60 80 100

Japan 2

Hungary 1

Italy 1

Luxembourg 1

Australia 1

Ireland 1

OECD

Belgium 1

Denmark 2

Netherlands 1

Finland 1

New Zealand 1

Iceland 1

France 2

Canada 2

Norway 1

Sweden 1

United Kingdom 1

United States 1

Percentage

2006

2003

2000

27.5

65.5

62.0

54.1

62.9

67.8

66.4

67.3

63.0

63.3

66.0

70.6

61.9

74.1

50.1

57.6

61.3

61.6

63.3

65.6

65.8

65.9

67.0

67.7

69.0

69.0

71.0

71.9

76.5

0 20 40 60 80 100

Poland

United Kingdom

Denmark

Czech Republic

Ireland

OECD (14)

Sweden

Norway

United States

France

New Zealand

Netherlands

Finland

Japan

Iceland

Canada

Korea

Age-standardised rates (%)

2002-2007

1997-2002

Sources: OECD HCQI Data 2009. Survival rates are age standardised to the International Cancer Survival Standards population. OECD Health Data 2009 (cancer screening; mortality data extracted from the WHO Mortality Database and age standardised to the 1980 OECD population). The 95% confidence intervals are represented by H in the relevant charts.

5.7.3. Cervical cancer mortality, females, 1995 to 2005 (or nearest year)

11.4

6.9

6.0

5.7

4.7

4.3

3.7

3.4

3.0

2.9

2.9

2.4

2.4

2.4

2.2

2.1

2.1

2.1

2.0

1.9

1.9

1.7

1.6

1.5

1.3

1.2

1.1

0.7

0.6

0

2

4

6

8

10

12

14

16

181995 2000 2005Age-standardised rates per 100 000 females

1. Programme. 2. Survey.

5.8.1. Mammography screening, percentage of women aged 50- 69 screened, 2000 to 2006 (or nearest year)

5.8.2 Breast cancer five-year relative survival rate, 1997-2002 and 2002-2007 (or nearest period)

19.5

23.8

35.6

47.1

56.2

59.0

59.6

60.1

62.0

62.2

63.5

70.4

70.7

72.5

76.7

78.1

86.2

89.0

0 50 100

Slovak Republic 1

Japan 2

Czech Republic 1

France 1

Australia 1

Belgium 1

Italy 1

New Zealand 1

Hungary 1

Iceland 1

OECD

Luxembourg 1

Canada 2

United Kingdom 1

United States 2

Norway 1

Ireland 1

Finland 1

Netherlands 2

Percentage

2006

2003

2000

60.2

70.8

76.9

72.2

80.5

77.0

76.2

82.6

80.0

82.0

86.1

83.8

85.6

88.6

61.6

75.4

75.5

76.2

77.9

81.1

81.9

82.1

82.4

85.2

86.0

86.1

87.1

88.3

90.5

0 20 40 60 80 100

Poland

Czech Republic

Korea

Ireland

United Kingdom

OECD (14)

Norway

New Zealand

Denmark

France

Netherlands

Finland

Japan

Sweden

Canada

Iceland

United States

Age-standardised rates (%)

2002-2007

1997-2002

Sources: OECD HCQI Data 2009. S urvival rates are age standardised to the International Cancer Survival Standards population. OECD Health Data 2009 (cancer screening; mortality data extracted from the WHO Mortality Database and age standardised to the 1980 OECD population). The 95% confidence intervals are represented by H in the relevant charts.

5.8.3. Breast cancer mortality, females, 1995 to 2005 (or nearest available year)

1. Rates for Iceland and Luxembourg are based on a three-year average.

29.5

28.4

27.0

25.8

25.1

24.9

24.2

23.9

23.1

22.4

22.4

21.5

21.3

21.1

20.8

20.7

20.5

20.3

20.0

19.9

19.5

19.5

19.3

19.3

19.2

16.7

11.0

10.4

5.8

0

10

20

30

40

1995 2000 2005

Age-standardised rates per 100 000 females

France (1997-2002)

Finland (2002-2007)

New Zealand (2002-2007)

Ireland (2001-2006)

5.9.1. Colorectal cancer, five-year relative survival rate, total and male/female, latest period

Netherlands (2001-2006)

Japan (1999-2004)

Canada (2000-2005)

Sweden (2003-2008)

Denmark (2002-2007)

Korea (2001-2006)

Iceland (2003-2008)

United States (2000-2005)

OECD

Norway (2001-2006)

United Kingdom (2001-2006)

Czech Republic (2001-2006)

Poland (2002-2007)38.1

46.8

50.7

52.3

54.4

57.1

57.2

57.8

58.1

58.1

59.8

60.7

60.9

62.0

65.5

66.1

67.3

020406080100Age-standardised rates (%)

34.7

45.6

50.1

50.7

54.2

56.6

56.3

56.9

58.4

59.1

55.2

59.6

59.6

57.0

65.9

69.2

68.7

39.3

48.5

51.5

54.3

54.8

58.5

57.9

59.0

58.2

57.1

64.5

62.3

62.3

62.0

65.1

63.2

66.0

0 20 40 60 80 100Age-standardised rates (%)

Female Male

1. 2000-2005 rather than 2002-2007. 2. 1998-2003 rather than 1997-2002 3. 2001-2006 rather than 2002-2007.

5.9.3. Colorectal cancer mortality, 1995 to 2005 (or nearest year)

5.9.2. Colorectal cancer, five-year relative survival rate, 1997-2002 and 2002-2007

Sources: OECD HCQI Data 2009. Survival rates are age standardised to the International Cancer Survival Standards population. OECD Health Data 2009 (mortality data extracted from the WHO Mortality Database and age standardised to the 1980 OECD population). The 95% confidence intervals are represented by H in the relevant charts.

5.212.113.214.214.414.615.2

16.716.817.017.217.617.618.018.218.819.019.2

19.8

20.620.821.021.4

25.025.3

29.831.031.9

0 10 20 30 40

MexicoGreeceFinland

SwitzerlandUnited States

IcelandKorea

AustraliaSweden

ItalyFranceJapan

United KingdomCanada

LuxembourgAustria

OECD (27)Spain

PortugalGermanyBelgium

NetherlandsPolandIreland

NorwayDenmark

New ZealandSlovak RepublicCzech Republic

Hungary

Age-standardised rates per 100 000 population

2005

1995

20.2

19.8

41.1

48.9

50.2

55.0

54.6

56.9

52.3

57.4

59.6

57.0

60.0

62.5

46.8

52.3

54.4

57.8

57.9

58.1

58.1

60.1

60.7

60.9

62.0

65.5

0 20 40 60 80

Czech Republic 3

Ireland 3

Denmark

Norway 3

OECD (11)

Netherlands 3

Korea 3

Sweden

Canada 1

New Zealand

Finland 2

United States 1

Age-standardised rates (%)

2002-2007 1997-2002

Dept Social Medicine

Performance indicators and benchmarking on care delivered in hospitals

PATH, OECD, many national projects …………

Limitations (administrative) data-bases- Quality of coding practices- Lack of (internationally) standardized procedure codes- Lack of coding of secondary diagnoses- Lack of present at admission coding- Lack of linking via UPI’s- Limitations Electronic Health Records

5.4.1. In-hospital case-fatality rates within 30 days after admission for AMI, 2007

Poland

Austria (2006)

Canada

Italy (2006)

New Zealand

Denmark

Sweden

Iceland

United Kingdom

Norway

Spain

Czech Republic

Korea

Slovak Republic

Luxembourg (2006)

Netherlands (2005)

United States (2006)

Ireland

OECD

Finland

3.6

6.6

4.6

6.4

5.3

6.9

6.6

7.7

5.6

11.0

7.7

8.3

7.0

7.7

9.2

10.9

9.1

10.7

9.9

9.6

2.1

2.9

2.9

3.2

3.3

4.0

4.2

4.5

4.5

4.9

4.9

5.1

5.1

5.3

6.1

6.3

6.6

6.6

7.6

8.1

051015Rates per 100 patients

Age-sex standardised rates

Crude rates 3.3

3.0

2.7

3.0

3.3

3.7

4.0

4.5

4.5

5.6

4.9

5.2

4.9

5.1

5.6

5.8

6.0

8.7

7.0

7.2

0.9

2.9

3.1

3.4

3.2

4.3

4.4

4.5

4.6

4.2

4.9

5.0

5.4

5.5

6.5

6.7

7.1

4.8

8.1

8.9

0 5 10 15Age-standardised rates per 100 patients

Female Male

Source: OECD HCQI Data 2009. Rates have been age-sex standardised to the 2005 OECD population (45+). 95% confidence intervals are represented by H.

5.4.2. Reduction in in-hospital case-fatality rates within 30 days after admission for AMI, 2003-2007 (or nearest year)

8.8

8.3

8.1

7.7

6.5

5.8 6.3 6.9

6.0

5.3

4.8 4.9

4.7

3.7

8.5

3.8

6.6

6.6

6.2

5.2

5.2 5.7

5.2

5.2

3.7 4.2

3.9

3.4

8.1

6.6

6.1

5.1

4.9

4.7

4.5

4.5

4.2

3.3

3.2

2.9

2.9

0

2

4

6

8

10

12

2003 2005 2007

Age-sex standardised rates per 100 patients

5.5.1. In-hospital case-fatality rates within 30 days after admission for ischemic stroke , 2007

5.5.2. In-hospital case-fatality rates within 30 days after admission for hemorrhagic stroke , 2007

5.8

3.6

5.3

5.9

7.4

7.3

7.0

7.7

8.4

6.0

9.0

10.5

9.4

10.8

10.7

11.4

12.1

11.6

12.9

17.4

2.3

2.4

3.1

3.2

3.3

3.7

3.7

3.8

3.9

4.2

5.0

5.6

5.9

6.2

6.3

6.5

6.6

7.5

7.6

9.0

0 5 10 15 20

Iceland

Korea

Denmark

Finland

Norway

Italy (2006)

Austria (2006)

Germany

Sweden

United States (2006)

OECD

Luxembourg (2006)

Netherlands (2005)

Czech Republic

New Zealand

Spain

Ireland

Slovak Republic

Canada

United Kingdom

Rates per 100 patients

Age-sex standardised rates

Crude rates

11.1

13.1

11.3

17.2

19.9

19.7

21.3

20.8

22.5

29.2

23.5

27.3

26.8

27.3

28.2

31.0

26.0

32.1

29.5

32.6

9.5

10.8

11.0

12.8

13.7

14.5

16.7

17.2

19.4

19.8

19.8

23.2

23.8

24.0

24.2

25.2

25.5

26.3

29.3

30.3

0 10 20 30 40

Finland

Austria (2006)

Korea

Sweden

Norway

Germany

Denmark

Italy (2006)

Ireland

Iceland

OECD

Canada

New Zealand

Czech Republic

Spain

Netherlands (2005)

United States (2006)

United Kingdom

Slovak Republic

Luxembourg (2006)

Rates per 100 patients

Age-sex standardised rates

Crude rates

5.5.4. Reduction in in-hospital case-fatality within 30 days after admission for stroke, 2002-2007

5.5.3. In-hospital case-fatality rates within 30 days after admission for ischemic and hemorrhagic stroke, 2007

Source: OECD HCQI Data 2009. Rates are age-sex standardised to the 2005 OECD population (45+). 95% confidence intervals are represented by H in the relevant charts.

1. Based on change from 2002-2003 to 2006. 2. Based on a three-year period only.

AUT

CANCZE

DNK

FIN

DEU

ISL IRL

ITA

KOR

LUX

NLD

NZL

NOR

SVL

ESP

SWE

GBRUSA

R² = 0.54

0

5

10

15

20

25

30

35

0 2 4 6 8 10

Age-sex standardised case-fatality rates for hemorrhagic stroke (%)

Age-sex standardised case-fatality rates for ischemic stroke (%)

33.8

28.6

20.4

18.9

17.8

17.6

15.7

14.1

11.6

9.7

5.5

2.5

1.2

0.5

39.8

14.0

22.9

35.7

16.8

24.2

6.5

16.5

25.6

5.4

1.6

5.0

16.4

0.4

0 10 20 30 40 50

Norway

Austria

Korea 2

Netherlands 2

Germany 2

Finland

Sweden

OECD (13)

Ireland

Spain

Canada 2

Denmark

New Zealand

Luxembourg 1

% decline over period (standardised rates)

Ischemic stroke

Hemorrhagic stroke

Dept Social Medicine

Patient Safety Indicators

• Indicators based on administrative databases

• Adverse event reporting

• Safety culture

24

Indicators– Foreign body left in during procedure (PSI 5) – Catheter related bloodstream infections (PSI 7) – Postoperative pulmonary embolism or deep vein

thrombosis (PSI 12) – Postoperative sepsis (PSI 13) – Accidental puncture and laceration (PSI 15) – Obstetric trauma -- vaginal delivery with instrument

(PSI 18) – Obstetric trauma -- vaginal delivery without

instrument (PSI 19)

Dept Social Medicine

Performance indicators in primary care

• Avoidable hospital admissions

• Lack of comprehensive administrative data-sets

Avoidable hospital admission rates, 2007

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3Austria

Belgium

Canada

Denmark

Finland

Germany

Iceland

Ireland

Italy

KoreaNetherlands3

New Zealand

Norway

Poland2

Spain

Sweden

Switzerland

United Kingdom

United States1

Asthma COPD Diabetic acute complications CHF

Note: Data from Austria, Belgium, Italy, Poland, Switzerland and the United States refer to 2006. Data from the Netherlands refer to 2005. 1. Data does not fully exclude day cases. 2. Data includes transfers from other hospitals and/or other units within the same hospitals, which marginally elevate the rates. 3. Data for CHF includes admissions for additional diagnosis codes, which marginally elevate the rate. Source: OECD Health Care Quality Indicators Database, 2009

Dept Social Medicine

Patient experiences

• Service based surveys (CAHPS, Picker, CKZ)• Population based surveys (Eurobarometer, WHO,

CWF)

• Lack of standardization• Lack of research on validation• Lack of research on use

Limitations National Information Infrastructures

• Mortality Statistics • Registries• Administrative Data-Bases• - secondary diagnoses• - present-at-admission coding• - unique patient identifiers• Electronic Health Records• Household and Patient Surveys• Overall: privacy and data-protection

National Information Infrastructures

• Mortality statistics• Registries (cancer)• Administrative

Databases

• Electronic Health Records

• Surveys

• UPI’s/co-morbidity• UPI’s/coding-staging• UPI’s, present-at-

admission codes, secondary diagnoses

• Standardized secondary data-use, privacy concerns

• UPI’s

MA

RQ

uIS

-M

eth

ods

of

Ass

ess

ing R

esp

on

se

to

Qu

ali

t y I

mpro

vem

en

t S

trate

gie

s

Hospital Level

Q.I. Strategies

Ward Level

AIM Deliveries Appendicitis

QI Strategies QI Strategies QI Strategies

Outputs Outputs Outputs

1.2. Analysis of strategies inter-connection

Audit and

internal assessm

ent

Clinical guidelines Perform

a

nce

indicator

s

Org

aniz

atio

nal

qual

ity

Patie

nts’

sa

fety

Patients’ views

51%

Exploratory Factor Analysis

StrategiesLoading weights

- Patient Safety Systems- TQM- Performance Indicators- Systems for getting Patients Views- Clinical guidelines

.857

.822

.694

.581

.578

Dept Social Medicine

Health System Performance Management

• Whole system approach

• Sub-optimization

• Governance/stewardship

• Incentive structure

• Interconnection of strategies on performance indicators, guidelines, safety, TQM, patient experiences