presentation_thesis defense _thi ha vo. 16.12.2015

Assessment of the potential impact of pharmacist interventions: development and validation of the CLEO multidimensional tool

Thi Ha VOPharmacist, PhD Student

Director: Assoc.Prof. Pierrick BedouchCodirecteur: Prof. Benoit AllenetLaboratory TIMC-IMAG UMR CNRS 5525

PhD Defense - 16 December 2015

1. Introduction

2. Development of a new tool

2.1. Review of models and tools

2.2. Development of the CLEO tool

3. Validation of the CLEO tool

3.1. Test in a general practice

3.2. Test in a specific clinical service

4. Discussion & Perspectives

5. Conclusion

Plan

Drug-related problems (DRP) 2-27%2 of hospital admission

50%1 DRPs are preventable

Solution = team working with clinical pharmacists

Introduction – What’s clinical pharmacists’ roles Introduction – What’s role of clinical pharmacists?

1. McKenney JM et al. Drug-related hospital admission. Am J Hosp Pharm. 1976;33(8):792-52. Kohn LT et al. To Error is Human. 1999

Collect patient information

Identify DRPs

Suggest Pharmacist interventions (PIs)

Medication Review

Pharmacist intervention (PI): «any action by a clinical pharmacist that directly results in a change in patient management or therapy»1. (Dooley at al. BJCP 2004)

Introduction – What’s pharmacist interventions ?Introduction – What’s a pharmacist intervention?

Introduction – How to evaluate impacts of PIs ?

Impacts of the

PI

When ?Potential or

actual impacts

Where ?Hospital,

community pharmacy

How ?Models, tools,

process of evaluation… What ?

Clinical, economic,

humanistic…

For whom ?Patient,

physician, nurse, pharmacist,

hospital, society

Evaluation of impacts of PIs is essential for demonstrating the added value of pharmacists, continuing quality improvement, research and education…

Introduction – How to evaluate PIs?

Reponses to these questions are important to develop a new tool for assessing impacts of PIs

Overhage and Lakes, USA,1999

5 distinct tools found in 10 studies from a review of articles from 1966 to 1997

Overhage and Lakes. Practical, reliable, comprehensive method for characterizing pharmacists’ clinical activities. Am J Health-Syst Pharm. 1999; 56:2444-50

Introduction – Tools to evaluate PIs?

Conduct an updated systematic review of tools for assessing

potential impacts of PIs

Review models of evaluations and develop an optimal model for

evaluation of PIs

Objectives

Develop and validate a multidimensional tool for assessing potential impact of PIs

1. Introduction








5. Conclusion

Plan

A systematic review

873 Abstracts scanned

Exclusion criteria:• Specific type of DRPs/PIs only• Actual impacts only• Economic impact only• Non-accessible and review

articles

82 tools in 133 studies: models & content, structure, validity

A systematic review – Method

Keywords: DRP* AND PI*Inclusion criteria :• Langue: English, French• Abstract available• Peer review journals• Pharmacists involved• Explicit description of tools

Databases- Pubmed (1986-2013)- PsycINFO (1999-2013)- PASCAL (1997-2013)- CINAHL (1993-2013)

Review of tools for assessing potential impacts of PIs

Hand searchs- References of (review) articles- Thesis of Quélennec

• Model of Donabedian

• Model of Kozma et al.

• Pharmacoeconomic model

• Risk assessment matrix

SP(ECH)O-P model

Models of evaluations

A systematic review – Results & Discussion

Indicators of content of tools

• Structure

• Process

• Outcome

- Clinical

- Humanistic

- Economic

• Probability

A. Models & content of tools


STRUCTURE

SettingsStaffs: trained pharmacistsEquipmentsInformation resourcesFinancal stability….NOT FOUND IN TOOLS

PROCESS

TechnicalConsistency to standardsInformational interventionAcceptance by HCPs…

InterpersonalSatisfaction of HCPsCollaboration between HCPsContinuation of care…

OUTCOME

Clinical Humanistic Economic…

1. Model of Donabedian 1966 Quality of a pharmacist intervention

Donabedian et al (1966). Evaluating the quality of medical care. Milbank Mem Fund Q.


STRUCTURE

SettingsStaffs: trained pharmacistsEquipmentsInformation resourcesFinancal stability

….

PROCESS

TechnicalN° DRPs/PIsConsistency to standardsAgreement between HCPsAcceptance

InterpersonalSatisfaction of HCPsCollaboration btw HCPs

OUTCOME

CLINICAL

• Severity of DRPs: harm, ADR…

• Significance of PIs: Health care resources avoided (emergency visit, hospitalization…)

ECONOMIC

• Direct costs: cost of implementation of the PI cost saving, cost avoidance

• Indirect costs: missing working

• Intangible costs: pain, suffering…

HUMANISTIC

• Patient’s satisfaction• Knowledge• Medication compliance• Quality of life…

2. Model of Kozma et al. 1993 Outcomes of a pharmacist intervention

Kozma et al (1993). Economic, clinical, and humanistic outcomes: a planning model for pharmacoeconomic research. Clin Ther.

STRUCTURE


….

PROCESS


InterpersonalSatisfaction of HCPsCollaboration between HCPs

CLINICAL

• Severity of DRPs: harm, ADR, therapeutic failure

• Significance of PIs: Health care resources avoided (eg., emergency visit, hospitalization…)

ECONOMIC




• HUMANISTIC

• Patient’s satisfaction• Knowledge• Medication compliance• Quality of life• …

3. Pharmacoeconomic model Value of a PI

Value = Differences between the scenarios without and with the PI


Schumock GT et al. (2000). Method to assess the economic outcomes of clinical pharmacy services. Pharmacotherapy.

15

STRUCTURE


….

PROCESS



CLINICAL


• Significance of PIs: Health care resources avoided(eg., emergency visit, hospitalization…)

ECONOMIC




HUMANISTIC


Severity X Probability = Risk Matrix

4. Risk assessment matrix


National patient safety agency (2008). A risk matrix for risk managers.

STRUCTURE


….

PROCESS



OUTCOME

Clinical Humanistic Economic…

CLINICAL


• Significance of PIs: Health care resources avoided (eg., emergency visit, hospitalization…)

ECONOMIC




HUMANISTIC


Value = Differences between the scenarios without and with the PI

Severity X Probability = Risk Matrix

An optimal model of evaluation of impacts of PIs: SP(ECH)O-P

Multi- or mono-dimensional

Independent/Integrated

Ordinal/Nominal

Numeric/Non-numeric

Explicit/Implicit

Opened/closed

B. Properties of structure of tools


Few tools have these optimal properties of structure


C. Properties of validation of tools


Inter-rater reliability

• Agreement between raters

• 48/133 studies

Intra-rater reliability

• Agreement of same raters between times

• 2/133 studies

Validity

• Agreement between raters’ and gold standard ratings

• 8/133 studies


B. Properties of structure of tools

Need of test inter-rater, intra-rater reliability and validity of a new tool

1. Introduction








5. Conclusion

Plan

20

Development of the CLEO toolPROCESS OF DEVELOPMENT

• GROUP: 7 pharmacists of the working

group of SFPC (French Society of Clinical

Pharmacy)

• WHY ? A simple, comprehensive, reliable

tool

• WHERE ? Hospital

• WHAT ? clinical, humanistic, economic,

process-related. No probability.

• WHEN? Potential impact

• FOR WHOM ? For the patient, hospital,

HCPs

• HOW ? 3 dimensions, 4-7 categories,

opened

• RESULTS: 6 direct meetings, 5 versions of

the CLEO tool

SCORE IMPACTS

CLINICAL (CL)

-1C Negative

0C Null

1C Positive – Humanistic

2C Favorable – Minor

3C Favorable – Major

4C Favorable – Vital

ND Non-determined

ECONOMIC (E) -1E Negative

0E Null

1E Positive

ND Non-determined

ORGANIZATIONAL (O) -1O Negative

0O Null

1O Positive

ND Non-determined

Score (CL, E, O)

Score Impact The clinical impact is evaluated according to the most likely case expected, not the worst/best case

-1C Nuisible The PI can lead to adverse outcomes on clinical status, knowledge, satisfaction, patient adherence and/or quality of life of the patient.

0C Null The PI can have no influence on the patient regarding the clinical status, knowledge, satisfaction, patient adherence and or quality of life of the patient.

1C Minor The PI can improve knowledge, satisfaction, medication adherence and/or quality of life of the patient OR the IP can prevent damage that does not require monitoring/treatment.

2C Moderate The PI can prevent harm that requires further monitoring/treatment, but does not lead or dose extend a hospital stay of the patient.

3C Major The PI can prevent harm which causes or lengthens a hospital stay OR causes permanent disability or handicap.

4C Lethal The PI can prevent an accident that causes a potentially intensive care or death of the patient.

ND Non-determined

The available information does not determine the clinical impact.

21

1. CLINICAL IMPACT6 levels of Hatoum’s tool

Severity categories from the NCC MERP Index

Humanistic indicators = low levels from the Williams et al.’s tool

Development of the CLEO tool

The « Clinical impact » focuses on clinical and humanistic impacts of the PI for the patient.

22

2. ECONOMIC IMPACT

Score Impact Definition

-1E Increase of cost The PI increases the cost of the drug treatment of the patient.

0E No change The PI does not change the cost of drug treatment of the patient.

1E Decrease of cost The PI saves the cost of drug treatment of the patient.

ND Non-determined The available information does not allow determining the economic impact.

The cost of drug therapy contains two main elements:• The cost of drugs• The cost of monitoring of drug therapy (e.g., clinical, kinetic, biological

monitoring ...). The cost of drug therapy is based on the financial cost of a hospital.


The « Economic impact » dimension focuses on cost savings of PIs based on the financial cost for a hospital

23

3. ORGANIZATIONAL IMPACT

Score Impact Definition

-1O Negative The PI reduces the quality of care process.

0O Null The PI does not change the quality of the care process.

1O Positive The PI increases the quality of the care process.

ND Non-determined The available information does not identify the organizational impact.

The organizational impact is coded regarding the overall impact on the quality of the care process from the perspective of health care providers (eg, time savings, improved security, knowledge, job satisfaction of nursing staff; facilitating professional tasks or teamwork, continuity of care, etc.)


The « Organizational impact » dimension focuses on benefits of PIs on process of care for HCPs

24

Development of the CLEO toolA CASE STUDY

• Description: Woman 85 years old was

treated by AUGMENTIN (amoxicillin +

clavulanic acid) IV for sinusitis.

• PM: the patient was known to be

allergic to beta-lactams

(angioedema).

• IP: change to PYOSTACINE

(pristinamycin) 500mg tablet.

SCORE IMPACTS

CLINICAL (CL)

-1C Negative

0C Null

1C Positive – Humanistic

2C Favorable – Minor

3C Favorable – Major

4C Favorable – Vital

ND Non-determined

ECONOMIC (E) -1E Negative – Increase of cost

0E Null – No change

1E Positive – Decrease of costs

ND Non-determined

ORGANIZATIONAL (O) -1O Negative

0O Null

1O Positive

ND Non-determined

Score (3C, -1E, 1O)

1. Introduction







4. Discussions & Perspectives

5. Conclusions

Plan

Test in a general practice - Method

1st rating

2nd rating

3rd rating

50 PIs from the Act-IP®database

30 PIs from the 6 pharmacists’ hospital practice

10 PIs from the same 30 PIs

The CLEO v1

The CLEO v2

The CLEO v2

Raters TestScenarios Tool

InTER-rater reliability

InTER-rater reliability

InTRA-rater reliability

7 pharmacists of the group of SFPC

The SP(ECH)O-P model The CLEO

Test in a specific practice - Method

Kappa value – Agreement< 0 : poor0.00-0.20: slight0.21-0.40: fair0.41-0.60: moderate0.61-0.80: substantial 0.81-1.00: almost perfect

Agreement: %

Validity/Reliability: Weighted kappa (kw)

Statistical test

Gisev et al (2013). Interrater agreement and interrater reliability: key concepts, approaches, and applications. Res Social Adm Pharm.

Expected weighted kappa: kw ≥ 0.41 (moderate agreement)

Interpretation of kappa value by Landais & Koch:

Test in a general practice - Results

1st rating

2nd rating

3rd rating

The CLEO v1

The CLEO v2

The CLEO v2

Raters TestTool

InTER-rater reliabilityCL: 36%, kw = 0.34 XE: 65%, kw = 0.53 O: 57%, kw = 0.26 X

InTER-rater reliabilityCL: 39%, kw = 0.41 E: 90%, kw = 0.93 O: 62%, kw = 0.39 !

InTRA-rater reliabilityCL: 33%, kw = 0.38 ! E: 80%, kw = 0.70

The CLEO v2 CL nearly validatedE validatedO nearly validated


1. Introduction








5. Conclusions

Plan


Objective: To assess validity and reliabilty of the CLEO tool used in daily practice in a Centralized Preparation Unit of Anticancer Drugs (CPU) at the Grenoble University Hospital

Type of study: mono-center, prospective

Services included : • Complete Hospitalisation in hematology and oncology• One-day Hospital in hematology, oncology, pneumology, hepato-

gastroenterology and radiotherapy

Period of study : 10 weeks (July 2014 to September 2014)

Sample: 214 PIs related to 167 patients.

1st rating

2nd rating

3rd rating

Ward-based pharmacist (P2)

Hematology43 PI

Oncology radiotherapy

146 PI

Pneumology33 PI

Hepatogastroenterology

15 PI

1 panel for each speciality 4 panels

Pharmacist at the CPU (P1)

Expert panel (EP)

Composition of each panel : 4 members (a specialist – physician, a clinical pharmacist, a pharmacist at the CPU and a pharmacist at a pharmacovigilance center)

The CLEO v2

The CLEO v2

The CLEO v2

InTER-rater reliability (P1-P2)

Validity (P1-EP)

Validity (P2-EP)

Raters TestTool

Test in a specific practice - MethodProcess of ratings


1st rating

2nd rating

3rd rating

Ward-bassed pharmacist (P2)

Pharmacist at the CPU (P1)

Expert panel (EP)

The CLEO v2

The CLEO v2

The CLEO v2

Inter-rater reliability (P1-P2)CL: 51%, kw = 0.48 E: 71%, kw = 0.61 O: 60%, 0.27 X

Validity (P1-EP)CL: 41%, kw = 0.32 ! E: 68%, kw = 0.53 O: 57%, kw = 0.17 X

Validity (P2-EP)CL: 54%, kw = 0.56 E: 81%, kw = 0.75 O: 49%, kw = 0.11 X

Raters TestThe tool

The CLEO v2 CL nearly validatedE validatedO NOT validated


1. Introduction








5. Conclusion

Plan

A updated systematic review of tools of assessing

potential impacts of PIs was conducted with:

• 82 distinct tools found in 133 studies: rich resources for

development of new tools

• 6 models of evaluations

• 12 recommendations of methods of assessment of

potential impacts of PIs: theoretical, psychometric and

pragmatic properties

Discussion & Perspectives

1. Principal findings

The SP(ECH)O-P model:• The first specific model in literature for evaluation of PIs

• A comprehensive model including 6 main types of

indicators

• Relationships between types of indicators


1. Principal findings

▷ Content & Structure: humanistic indicators included, health care resouces included, structure inspired the Hatoum’s tool, content inspired from NCC MERP, 7 categories

▷ Validation properties33-54%, Kw = 0.32-0.56 VALIDATED

▷ ▷ Content & Structure: cost savings

(drug, monitoring), 4 categories▷ Validation properties68-90% , Kw = 0.53-0.93 VALIDATED

▷ Content & Structure: integration of many organizational indicators and persepctives of HCPs, 4 categories

▷ Validation properties49-63% , Kw = 0.11-0.39 NOT VALIDATED

Economic impact

Test of the CLEO in 2 studies - Results

Clinical impact

Organizational impact

Content, Structure & Validity of the CLEO tool1. Principal findings


Improvement of validation properties: training, examples Establisment relationship between potential and actual clinical

impact

Development of separate tools for measuring humanistic impacts

Specific study for improvement of organizational impact: eg., qualitative studies on perceptions of HCPs of organizational impacts

Estimation of direct costs (cost savings, cost avoidance and cost of implementation of a PI)

Adding estimation of probability

2. Validity of findings – Quality of the CLEO tool

Develop of assessment matrix (a global score)

Discussion – The CLEO tool2. Validity of findings – Validity of research method

Internal validity - Confounding: perceptions

of raters

- Maturation of raters:

familiarity with the tool

- Testing bias: awareness of

ratings

- Selection bias of PIs- Statistical test: %, kw

External validity- Raters: 7 raters involved to

develop another tool 10

years ago

- Ratings: requirement of

confirmations of ratings in

some cases

- Settings: a specific

practice (CPU)

- Tool: the French-written

CLEO tool for hospital

Database: Adding only the

« Clinical impact » and « Economic

impact » into the Act-IP® database.

French hospitals: test for daily use

in other services/hospitals (Lyon,

Annecy, Epernay…)

Community pharmacies:

modifications of the CLEO

Other langues/countries: Vietnam

276,851 documented PIs 1723 pharmacists

Senior 45% Resident 50% Student 6%

666 hospitals University 34% General 62% Psychiatric 4%

Act-IP© http://www.actip.sfpc.eu

Perspectives

http://sfpc.adiph.asso.fr/admin/

1. Introduction








5. Conclusion

Plan

The updated systematic review of tools for assessing of potential

impacts of PIs synthesized models, content, structure and

validation properties of tools.

The specific and comprehensive model for evaluations of PIs, named SP(ECH)O-P was developed.

The CLEO tool of assessing potential impacts of PIs (including 3

dimensions: Clinical Impact, Economic Impact, and

Organizational impact) was constructed and tested in 2 studies.

However, only « Clinical impact » and « Economic impact »

dimensions were validated.

Conclusion

42

Discussion, conclusion

Working group “Standardizing and demonstrating the value of clinical pharmacy activities” of the SFPC (the French Society of Clinical Pharmacy)

Group of clinical practitioners (pharmacists, physicians) at the Grenoble University Hospital

Acknowledgements

43

It is not hard to make decisions once you know what your values are. Roy E. Disney

presentation_thesis defense _thi ha vo. 16.12.2015

Health & Medicine