systematic review of patient safety indicators (psi ... · systematic review of patient safety...
TRANSCRIPT
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
Dr Jean-Marie Januel, PhD, MPH, RN
Systematic review of Patient Safety
Indicators (PSI) Validation Studies
&
Focus on PSI 12
OECD Health Care Quality Indicators
Patient Safety Subgroup Meeting
Paris, France, May 9, 2012
Institute of social and Preventive Medicine (IUMSP),
Lausanne University Hospital, Switzerland.
Department of Medical Information, Health Evaluation and Clinical Research,
Hospices Civils of Lyon, France.
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
2
SUMMARY
I. Systematic review of PSI validation studies
o Methods of the review and meta-analysis
o Pooled estimate of PPV
o Impact of the “Present on Admission” code
o Conclusion
II. Validating and comparing PSI 12
o A Pilot Validation Study in France for Assessing the
Positive Predictive Value of PSI 12
o International Project of PSI 12 Comparison Against
an External Benchmark
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
3
Part I.
Systematic Review of
Patient Safety Indicators
(PSI) Validation Studies
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
4
Research strategy
2001 to 2011 (= 11 full years)
o PubMed
o EMBASE
Algorithm used
[“Patient Safety Indicator” OR “Patient safety Indicators”]
OR
[(“PSI” OR “PSIs”) AND (“AHRQ” OR “Agency for
Healthcare Research and Quality”)]
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
5
Inclusions / Analysis
Studies concerning PSI in adults
Principal objective to validate PSI (outcome
measured)
o POSITIVE PREDICITVE VALUE (PPV)
o Impact of Present on Admission (PoA) code using
pooled Rate Ratio (RR) of the proportion of PSI without
PoA divided by the proportion of PSI with PoA (as the
risk to “over-indentify” PSI without PoA code)
Statistical analysis
o Random effects across studies
o Pooled estimates and Test of variance using I2
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
6
Descriptive results
153 articles focused on PSI
Including
o 131 Original studies
o 13 Reviews
o 9 Commentaries
17 studies that estimated PPV
5 studies that showed impact of “Present
on Admission” (PoA) code
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
8
Pooled PPV for PSI in the literature
Pooled estimate of PPV Heterogeneity
PSI Label N-studies % (95% CI) I²(%) P-value
PSI 5 - Foreign body left during procedure 1 45% (35% - 56%) na na
PSI 6 - Iatrogenic pneumothorax 2 76% (72% - 81%) 0.0% 0.369
PSI 7 - Vascular catheter related bloodstream infection 2 46% (30% - 63%) 87.4% 0.005
PSI 9 - Postoperative hemorrhage or hematoma 1 75% (66% - 83%) na na
PSI 10 - Postoperative physiologic or metabolic derangement 2 60% (53% - 68%) 3.7% 0.308
PSI 11 - Postoperative respiratory failure 3 76% (67% - 84%) 88.9% <0.001
PSI 12 - Postoperative pulmonary embolism and deep vein thrombosis
9 50% (39% - 61%) 97.1% <0.001
PSI 13 - Postoperative sepsis 2 49% (41% - 57%) 21.0% 0.261
PSI 14 postoperative wound dehiscence 2 79% (63% - 94%) 92.1% <0.001
PSI 15 - Technical difficulty / Accidental punction or laceration during procedure
3 85% (77% - 93%) 76.5% 0.014
No significant heterogeneity Significant heterogeneity
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
9
Pooled PPV for PSI 12
NOTE: Weights are from random effects analysis
Overall (I2 = 97.1%, P = 0.000)
STUDY
6. Henderson et al., 2009
3. White et al., 2009
7. Henderson et al., 2009
9. Zhan et al., 2007
8. Zhan et al., 2007
2. Romano et al., 2009
5. White et al., 2009
4. White et al., 2009
1. Kaafarini et al., 2011
49.97 (38.65-61.29)
PPV (95% CI)
54.46 (44.78-63.90)
63.64 (54.40-72.19)
50.00 (40.40-59.60)
62.65 (58.55-66.63)
29.22 (24.20-34.65)
21.99 (16.93-27.76)
62.65 (58.55-66.63)
62.39 (57.74-66.87)
42.86 (33.55-52.55)
100.00
Weight %
10.73
10.84
10.72
11.45
11.34
11.32
11.45
11.40
10.74
0 0 25 50 75 100
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
10
Summary of Meta-Analyses for assessing the impact
of the Present on Admission (PoA) code
NOTE: Weights are from random effects analysis
.
.
.
.
.
.
.
PSI 1 - Complication of anesthesia
(I2 = 82.0%, P = 0.019)
PSI 8 - Postop. hip fracture
(I2 = 0.0%, P = 0.825)
PSI 9 - Postop. hemorrhage or hematoma
(I2 = 0.0%, P = 1.000)
PSI 10 - Postop. physiologic or metabolic derangement
(I2 = 58.6%, P = 0.089)
PSI 11 - Postop. respiratory failure
(I2 = 22.7%, P = 0.274)
PSI 12 - Postop. PE/DVT
(I2 = 90.7%, P < 0.001)
PSI 13 - Postop. sepsis
(I2 = 0.0%, P = 0.727)
PSI
4.04 (0.32-51.21)
4.87 (1.25-19.01)
1.15 (1.04-1.28)
1.59 (1.10-2.29)
1.22 (1.07-1.40)
1.80 (1.43-2.25)
1.23 (1.05-1.45)
RR (95% CI)
30 / 44'520
12 / 25'748
745 / 125'201
275 / 67'617
714 / 65'836
2'586 / 209'159
327 / 38'540
PSI / Sample
without PoA
11 / 44'497
2 / 25'193
646 / 125'015
195 / 67'436
578 / 65'606
1'528 / 207'126
265 / 38'444
PSI / Sample
with PoA
1 1 2 3 4 5 10 50
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
11
CONCLUSIONS – Part I
PSI validation is based on PPV in most
instances
Differences between studies were
important for PPV
Significant impact of PoA code that
should be taken into account in future
works and ICD development
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
12
Part II.
Focus on the PSI 12
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
13
IIa. A Pilot Validation Study in
France for Assessing Positive
Predictive Value of the PSI 12
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
14
Assessing PSI 12 PPV - Pilot Study
Positive Predictive Value of the PSI 12 (Postoperative Pulmonary Embolism and Deep Vein Thrombosis)
154 inpatients at the University Hospital of
Lyon, France with identified PSI numerator
cases from DRG database in 2008
Stratified analysis by subgroups = type of
surgery
Chart Review of Medical Record
PSI + PSI -
DR
G PSI + True Positive Cases False Positive Cases
PSI - False Positive Cases True Positive Cases
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
15
PSI 12 PPV and False Positive Cases
Patients
with PSI+
True Positive
Cases
False Positive Cases
Total < Admission
Confusion DVT / PE
Absence of
Event
N (%) N PPV (%) N (%) N (%)* N (%)* N (%)*
Total VTE 154 (100.0) 123 (79.9) 31 (20.1) 7 (22.6) 5 (16.1) 19 (61.3)
TYPE OF SURGERY
Hip Arthroplasty 43 (27.9) 38 (88.4) 5 (11.6) 1 (20.0) 1 (20.0) 3 (60.0)
Knee Arthroplasty 58 (37.7) 56 (96.6) 2 (3.4) 0 (0.0) 1 (50.0) 1 (50.0)
# Femur 19 (12.3) 15 (79.0) 4 (21.0) 2 (50.0) 2 (50.0) 0 (0.0)
Other Orthopaedic 13 (8.4) 8 (61.5) 5 (38.5) 3 (60.0) 1 (20.0) 1 (20.0)
Digestive Surgery 13 (8.4) 6 (46.2) 7 (53.8) 0 (0.0) 0 (0.0) 7 (100)
Thoracic Surgery 8 (5.2) 0 (0.0) 8 (100) 0 (0.0) 0 (0.0) 8 (100)
* Based on False Positive Cases
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
16
This pilot study allowed to…
Estimate PPV using refined PSI 12
(denominator focused on HIP and KNEE
inpatients only)
Analyse data from multiple hospital
centers in France (CLARTE Project)
o 80 hospitals (representative of all types of
hospitals, nationwide geographic distribution)
o 550 inpatients with positive PSI 12
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
17
IIb. International Project of
PSI 12 Comparison Against an
External Benchmark in Patients
undergoing Hip and Knee
Arthroplasty
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
18
What is good value for comparison?
Zero Risk ?
… But studies on healthcare adverse events show
that 45% of those are avoidable…
Average mean of providers are compared ?
… but it’s a theorical value that doesn’t take into
account possibilities to reduce the risk according to
improve healthcare toward “state of art” practice…
External value according “state of art”
healthcare practice (recommendations,
guidelines)
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
19
The Proposal / Framework conceptual
Enlargement of our previous project of PSI 12 using
Swiss, French and Canadian data (JM Januel presentations
at IMECCHI meetings in Calgary on May 12th-13th, 2008; and
in Vancouver on November 9th-10th, 2009)
To develop an external independent benchmark that takes
into account “state of the art” healthcare practice
(Januel JM et al. JAMA 2012; 307(3): 297-303.)
To compare adjusted occurrence rate of symptomatic
postoperative VTE (PE and DVT respectively) during
patient hospital stay (i.e., before discharge) from several
countries using PSI 12 and this external benchmark
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
20
Population setting
Data for years 2008 and 2009
(excepted in France, 2007 and 2008, because
change in PDx coding rule on April 1st, 2009)
HIP and KNEE arthroplasty inpatients
(surgical procedure codes using country specific
classifications)
Stratified analysis for HIP and KNEE
arthroplasty independently
o Total and Partial Prosthesis, Prosthesis replacement
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
21
ANALYSIS
Calculate PSI 12 (= outcome measurement)
Develop hierarchical models using data from
each country independently
(Two-level logistic regression models)
Assess differences in practices used to
diagnose DVT based on the proportion of use of
lower extremity ultrasound by providers in each
country (procedure codes)
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
22
Januel JM, et al. JAMA 2012;307(3): 294-303
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
23
Pooled Occurrence Rates of VTE
(Januel et al. JAMA 2012)
TPHA TPKA
% (95% CI) I² P % (95% CI) I² P
LMWH (Observ.) 0.83 (0.19-1.48) 67.3% 0.230 1.98 (0.69-3.27) 84.0% <0.001
LMWH (RCT) 0.51 (0.26-0.76) 45.4% 0.010 1.24 (0.87-1.60) 36.7% 0.060
Direct Directe IIa/Xa Inhibitors (RCT)
0.31 (0.03-0.59) 32.8% 0.070 0.81 (0.54-1.07) 42.5% 0.047
Indirect IIa/Xa Inhibitors (RCT)
0.68 (0.26-0.97) 0.0% 0.380 0.77 (0.00-1.65) - -
TOTAL 0.53 (0.35-0.70) 49.4% <0.001 1.09 (0.85-1.33) 55.7% <0.001
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
24
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
Ad
juste
d o
ccu
ren
ce r
ate
s,
% w
ith
95%
CI
(Weig
thed
on
pro
vid
er
pro
ced
ure
case v
olu
me)
Procedure case volume by provider annually
Upper limit = 0.70%
Lower limit = 0.35%
<100 [100-200[ [200-400[ ≥400 All Providers
Mean = 0.53%
France Switzerland
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
25
Patient’s factors associated to VTE (PSI 12)
Swiss national data (2008-2009) French national data (2007-2008)
INPATIENT LEVEL FACTORS OR (95% CI) P OR (95% CI) P
Sex (F) 1.03 (0.74 - 1.45) 0.854 1.41 (1.31 - 1.51) <0.001
Age (≥65 yrs) 0.86 (0.57 - 1.29) 0.460 1.53 (1.39 - 1.68) <0.001
LOS (Log) 2.18 (1.57 - 3.02) 0.000 2.28 (2.11 - 2.46) <0.001
Hypertension 0.70 (0.44 - 1.12) 0.134 1.02 (0.94 - 1.10) 0.651
Arrhythmias 0.87 (0.49 - 1.55) 0.635 1.03 (0.93 - 1.15) 0.539
Valve disorder 0.78 (0.37 - 1.65) 0.508 0.71 (0.58 - 0.85) <0.001
Diabetes (uncomplicated) 0.61 (0.27 - 1.41) 0.249 0.77 (0.68 - 0.88) <0.001
Diabetes (complicated) 0.18 (0.02 - 1.68) 0.132 0.87 (0.66 - 1.16) 0.351
Renal failure 0.30 (0.14 - 0.65) 0.002 0.74 (0.60 - 0.91) 0.005
Rheumatic disease 0.35 (0.04 - 3.04) 0.339 0.79 (0.53 - 1.17) 0.235
Obesity 0.60 (0.27 - 1.34) 0.213 1.21 (1.05 - 1.39) 0.007
Cancer 0.88 (0.38 - 2.02) 0.757 1.14 (0.95 - 1.37) 0.154
Dementia 1.05 (0.56 - 1.99) 0.878 0.67 (0.56 - 0.79) <0.001
Death 7.37 (4.05 - 13.41) <0.001 2.62 (2.18 - 3.14) <0.001
Mean number of SDx coded 1.25 (1.18 - 1.31) <0.001 1.09 (1.08 - 1.11) <0.001
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
26
Provider’s factors associated to VTE (PSI 12)
Swiss national data (2008-2009) French national data (2007-2008)
HOSPITAL LEVEL FACTORS OR (95% CI) P OR (95% CI) P
Ultrasound of lower extremity
None (reference) 1.00 1.00
<20% of inpatients 1.56 (0.95 - 2.54) 0.076 1.25 (1.05 - 1.49) 0.010
[20% - 80%[ NA NA NA 3.68 (2.92 - 4.64) <0.001
≥80% of inpatients NA NA NA 4.17 (3.25 - 5.36) <0.001
Procedure case volume
<100 annually (reference) 1.00 1.00
[100 - 200[ 0.68 (0.38 - 1.22) 0.193 1.07 (0.90 - 1.26) 0.438
[200 - 400[ 0.74 (0.41 - 1.34) 0.322 1.14 (0.94 - 1.38) 0.169
≥400 0.62 (0.30 - 1.27) 0.189 1.25 (0.95 - 1.64) 0.110
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
27
A new method to validate and compare data
from several providers / countries
Major study to assess international comparison
against evidence based benchmark
o Symptomatic event of VTE (including PE and DVT)
o Before hospital discharge (i.e., available data)
o Against benchmark that reflect best updated practice
(“state of the art”)
Specific exposed population (HIP / KNEE
arthroplasty) using PSI 12 outcome measurement
o Patients with high risk of postoperative VTE
o Subgroups with good PPV (>85% in pilot study in
France)
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
28
Using complex but performing models
Models take account of important parameters
related to potential biases (across hospitals in
each country, between countries)
o Heterogeneity of measuring events due to different
diagnostic methods available to assess deep vein
thrombosis (In particular the use of more or less
systematic examination of the lower limbs using
ultrasound, according the fact that both symptomatic
and asymptomatic DVT are identified in these cases)
o Proposal to include a questionnaire to collect
information on habits, coding rules and healthcare
practice in participating countries
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
29
Rationality and Pertinence of the Benchmark
Need to assess healthcare quality using a
patient safety outcome measurement
o Events occuring during hospitalization stays
(between surgery and discharge)
o Directly linked to adherence and observance of the
recommended antithrombotic prophylaxis
implemented for patients
o Symptomatic events = usual healthcare practice
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
30
Limitations
Missing outcome registration for events that
occur after discharge
o Near 50% of VTE taken into account
o Difficulty to attribute post discharge events to
healthcare defaults or organizational problems in
hospitals
(e.g., responsibility of the patient for the observance
and the adherence of the prophylactic treatment at
his home)
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
31
CONCLUSIONS – Part II
To refine the PSI 12 denominator according
to specific sub-groups with high risk of VTE
and potential good PSI 12 PPV is the good
way for the near future (patients undergoing
HIP and KNEE arthroplasty)
Refinements needed to separate DVT and PE
according to the potential differences related
to diagnostic assessment of events (use of
lower extremity ultrasound)
IUMSP Institut universitaire de médecine sociale et préventive, Lausanne
32
Contact: [email protected]