rft 2003-02 evaluation of clinical interventions in community pharmacy final report this project was...

RFT 2003-02 Evaluation of clinical interventions in community pharmacy

Final Report

This project was funded by the Australian Government Department of Health and Ageing as part of the Third Community Pharmacy Agreement Research and Development Program, which is managed by The

Pharmacy Guild of Australia

2

The Research Team

• Gregory Peterson• Peter Tenni • Helen Kruup• Omar Hasan• Brita Pekarsky• James Reeve • Michael Roberts • Roger Rumble • Julie Stokes

3

RFT 2003-02: Evaluation of clinical interventions in community pharmacy

• Clinical Intervention– Where a pharmacist identifies, or is presented

with, an actual or potential drug related problem and he or she recommends an action to be taken to resolve or prevent the problem

4

Outline of Today’s Presentation

• Methods– Recruitment, training– Evaluation of value

• Results– Frequency, Types, Drugs involved– Economic Analysis

• Conclusions

• Where to from here?

Method

s

Conclusion

sR

esultsR

esults

5

Overview of Methods

Method

s

6

Pharmacy Recruitment and Enrolment

• 250 WiniFRED® pharmacies invited to participate

• 75 enrolled for the project, only 52 possible due to software, hardware or location issues

• Arms– Remuneration– Intervention

Prompt– Observation

Method

s

7

Remuneration Randomisation

Method

s

• Crossover design

8

PROMISe Observers

• Seven observers, each visiting 3 pharmacies 9 times in 3 weeks– 21 pharmacies “observed”

• Assist with documentation– Identify opportunities for documentation

– Aid classification/documentation process

• Time some events– Investigation of problems, phone calls

to doctors, discussions with patients

Method

s

9

Automatic Intervention Prompt

• Related to antiplatelet prophylaxis for vascular events in diabetic patients

• Activated when oral antidiabetic agents were dispensed

• 31 of 52 pharmacies randomised to receive prompt

Method

s

10

WiniFRED Interface Training

• Three training evenings plus initiation visits to each site

Method

s

11

Online Classification Training

• All pharmacists who indicated that they would participate were required to complete an on-line training package– 20 scenarios to be classified– 2 case-based clinical skills assessments

• Pharmacist demographics questionnaire completed at this point

• 20 scenarios re-classified after 3 weeks of use of the system

Method

s

12

PROMISe Data Collection

• Pharmacy Demographics– Daily workload and staffing– Entrepreneurial orientation– Prescriptions dispensed

• Pharmacist Demographics– Clinical skills– Job satisfaction

• Clinical Intervention Parameters– Patient demographics – Drug involved and other

drugs taken by patient– Type of problem– Action taken,

Recommendation made– Acceptance of

recommendation– Reactive or proactive– Time taken– Documenting pharmacist’s

rating of clinical significance

Method

s

13

Data Collection

• Initially planned for 3 weeks

• Extended to 4, then 8 weeks to obtain sufficient numbers of interventions

• Loss of interest from many pharmacies once observation phase was over and project team left Melbourne.

Method

s

14

Assessment of Value:Probability and Severity Considered

• Considerations: • How Addressed in Method

1. The nature and severity of the potential consequence(s) had the intervention not occurred

2. The probability that the consequence(s) will occur before the intervention

3. The probability that the consequence(s) will occur despite (after) the intervention

4. The degree to which the intervention can be attributed to the pharmacist

1. Consequences table•Economic and non-economic parameters for each level of severity, •validated by experts, •multiple consequences (positive and negative) possible

2,3. Panel members considered probability and severity for each consequence selected before and after the intervention

4. Panel members provided a value for attribution

Method

s

15

Method

s

Consequences Table

16

Assessment of Value

The Value of Clinical Interventions (VOCI)Economic Assessment System

AttributabilityAttributable Probability Reduction

Parameters of Consequence(s)

Value of Intervention Described as:

•Days of Loss of Poor Health•Cost of Admissions Avoided•Number of Admissions Avoided•Number of Days in Hospital Avoided•Number and Cost of GP Consults Avoided•Number and Cost of Specialist Consultations Avoided•Cost of Investigations Avoided

Probability of Consequence

BeforeIntervention

Probability of Consequence

AfterIntervention

- X =

X

Attributable Probability Reduction

=

AttributabilityAttributable Probability Reduction

Parameters of Consequence(s)

Value of Intervention Described as:

•Days of Loss of Poor Health•Cost of Admissions Avoided•Number of Admissions Avoided•Number of Days in Hospital Avoided•Number and Cost of GP Consults Avoided•Number and Cost of Specialist Consultations Avoided•Cost of Investigations Avoided

Probability of Consequence

BeforeIntervention

Probability of Consequence

AfterIntervention

- X =

X

Attributable Probability Reduction

=

PROMISe Economic Simulation

and Extrapolation

Model

Method

s

17

UTAS Server

I NTERNETINTERNET INTERNETINTERNET

INTERNETINTERNET

INTERNETINTERNET

Data Base and Analysis

Interface

Data Base and Analysis

Interface

Assessment of Value

• 16 Clinical Assessors in 4 panels– 2 physicians, 6 GPs, 8 pharmacists

• Secure internet access to intervention details• Each panel assessed the same set of 51

common interventions and a “panel specific” set of 60 randomly selected interventions

– 51 common interventions and 240 randomly selected interventions were assessed

Method

s

18

Clinical Panel Intervention Display

Method

s

19

Clinical Panel Selection of Consequences

Method

s

20

Economic Analysis: Derivation of Main Value Indicators

Method

s

21

Sources of Information

Results

5. PROMISe SQL Database (~13,000 interventions and ~430,000

prescriptions)

3. PROMISe Pharmacist Demographics (125)

6. Clinical Panel Assessments(16 members, ~290 interventions)

7. PROMISe Pharmacists’ Feedback

(~80)

1. PROMISe Pharmacy Demographics (52)

2. Non- PROMISe WiniFRED Pharmacy Demographics (~40)

8. Non-PROMISe Pharmacists’ Opinions (~400 phone interviews)

4. Direct Observation Visits

(63 visits)

22

Method

sR

esults

23

PROMISe Pharmacy Demographics (n = 52)

• Entrepreneurial Orientation– ~15% more innovative (self determined from responses to 2

questions)

• Location and Size– No different to Non-PROMISe and non-WiniFRED pharmacies

• Date of QCPP Accreditation– More Innovators (accredited before December 1999)

• 5/48;10.4% cf 2.5% in Victoria

• IT Facilities and resources– Used to determine attitudes to skills in IT area

• Daily Staffing levels– Used for workload analysis and simulations

Results

24

PROMISe Pharmacist Demographics (n= 125)

• Gender, Age, Registration year– Younger age group (80/125; 64% <40yo)

• Practice Profile– 17/122; 14% accredited for medication reviews (cf ~5%)

• Scenario Classification Score (Before and after study)– Improved from 76% to 83% post trial

• Scored well for– Job Satisfaction (83%), – Professional Integrity (77%), – Change Readiness (73%)

• Clinical Skills– Good range of scores

Results

25

PROMISe Database: Non Clinical Interventions

• 11,493 Non-Clinical (Brand Substitution interventions) from 305,519 scripts (average rate of 3.7%)

• Under-utilised by pharmacists in study, still extrapolates to ~$15M pa

Results

26

Clinical Interventions

Results

• 2396 interventions from 435,520 scripts

0.55 interventions per 100 scripts

0.00

0.20

0.40

0.60

0.80

1.00

1.20

21/04/2005

28/04/2005

5/05/2005

12/05/2005

19/05/2005

26/05/2005

2/06/2005

9/06/2005

16/06/2005Clinical Interventions per 100 Prescriptions

Cumulative Clinical Intervention Rate (per 100prescriptions)Poly. (Clinical Interventions per 100 Prescriptions)

27

Clinical Interventions

Results

Observers Present

Project Team PresentRemuneration

• Decline in recording of interventions

28

Clinical Interventions: CategoriesR

esults

29

Clinical Interventions: Actions

Results

• Investigation and discussion with patient common (71%)

• 18% contact with prescriber

30

Clinical Interventions: Recommendations Results

31

Clinical Interventions: Acceptance of Recommendations

Results

• Dose, Drug or Education category interventions highly accepted

32

Clinical Interventions: Proactive vs Reactive

• Drug selection, Dose problems more likely to be proactive

• Education and Toxicity less likely to be proactive (direct patient requests)

Results

33

Clinical Interventions: Clinical Significance (Pharmacist reported)

• More likely to be drug selection or toxicity problems and result in referral to GP

Results

34

Clinical Interventions: Drugs Involved- Numbers

Results

35

Clinical Interventions: Drugs Involved- Rates

• Skewed by intervention prompt

Results

36

Clinical Interventions for particular groups of drugs : Antidiabetic Agents

• Skewed by intervention prompt

Results

37

Clinical Interventions: Drugs Involved- Rates

Results

38

Results of Randomisation

Results

52 Pharmacies Enrolled

22 Ever Observed 30 Unobserved

12 Paid in Phase 1,3

18 Paid in Phase 2,3

11 Paid inPhase 1,3

11 Paid inPhase 2,3

9 Aspirin Prompt

5 No Aspirin Prompt

6* No Aspirin Prompt

2 No Aspirin Prompt

5 Aspirin Prompt

9 Aspirin Prompt

7 Aspirin Prompt

9 Aspirin Prompt

*1 pharmacy converted to prompt after 1 week

39

Clinical Interventions: Effect of Remuneration

Results

• Effect of remuneration in first two weeks of study (univariate)

• Small impact when payment instituted (ameliorated reduction cf 20% reduction from Phase 1 to 2)

1.09

0.61

0.71

0.81

0

0.2

0.4

0.6

0.8

1

1.2

Phase 1 Phase 2

Clin

ica

l In

terv

en

tio

n R

ate

Paid

Unpaid

25% reduction

14%

40

Clinical Interventions: Effect of Observation

• Observation significant (unadjusted)– 1.02 vs 0.46 for all

three phases (ever observed)

– 2.02 vs 0.8 for Phase 1 (observed days)

Results

41

Aspirin Intervention Prompt Effectiveness

Total194/7895 = 2.46

Aspirin prompt 193/4174 = 4.63 No prompt

1/ 3721 = 0.03

Ever observed0.03

Never observed0

Ever observed157/2128 = 7.34

Never observed37/2046 = 1.81

• Aspirin interventions per 100 diabetic patients• Only 7 in phase 3 (when aspirin prompt switched off)

Results

42

Aspirin Interventions: Observer Effect

0

20

40

60

80

100

120

140

160

180

Nu

mb

er o

f A

spir

in In

terv

enti

on

s

Never ObservedEver Observed

Phase 1: Observers Present Phase 2: Observers Absent

12.6

1.84

2.3

1.3

Phase 3: Prompt off

Results

43

Clinical Interventions: Effect of Intervention Prompt

• Significant effect in first half of study on other interventions as well

Results

44

Aspirin Intervention Prompt

• No aspirin interventions without prompt

• Observation had a marked effect on increasing rate

• Increased rate of other interventions as well

• No residual effect of prompt – 7 after prompt turned off

• ?Fatigue to prompt after 3-4 weeks

Results

45

Clinical Interventions: Multivariate Analysis of Effects Within Phases

Results

2,384 Pharmacy Days where more than 20 prescriptions dispensedMean Intervention Rate 0.74 (Standard Deviation 1.78)

Phase 1

1.01 (1.79)

Phase 20.88 (1.73)

Phase 30.52 (1.76)

Observed2.02 (2.26)

Observed2.40 (2.87)

Unobserved0.78 (1.59)

Unobserved0.52 (1.76)

Unobserved0.80 (1.61)

Paid2.29 (2.43)

Unpaid1.68 (2.00)

Paid0.98 (1.94)

Unpaid0.65 (1.27)

Paid2.05 (2.29)

Unpaid2.67 (3.29)

Paid0.66 (1.37)

Unpaid0.91 (1.80)

Paid0.52

(1.76)

Aspirin 2.45

(2.59)

No Aspirin

1.56 (1.27)

Aspirin 1.95

(2.17)

No Aspirin

1.29 (1.69)

Aspirin1.31

(2.29)

No Aspirin

0.48 (1.05)

Aspirin0.66

(1.32)

No Aspirin

0.63 (1.18)

Aspirin 2.90

(2.51)

No Aspirin

1.20(1.81)

Aspirin 3.07

(3.58)

No Aspirin

1.10 (0.70)

Aspirin0.74

(1.40)

No Aspirin

0.56 (1.33)

Aspirin1.27

(2.12)

No Aspirin

0.26 (0.52)

Aspirin0.81

(2.98)

No Aspirin

0.42 (1.10)

Phase, Prompt, Observation independently significant, payment not

46

Clinical Interventions: Multivariate Analysis of Effects Within Phases

Results

• No relationship effects when analysed by phase• Phase One

– Prompt significant (F=8.87; p = 0.003)– Observation significant (F=26.4; p <0.001)– Payment not significant (F=2.41; p = 0.121)

• Phase Two– Prompt significant (F=14.6; p < 0.001)– Observation significant (F=18.4; p <0.001)– Payment not significant (F=0.05; p = 0.813)

• Phase Three– Prompt significant ( F= 10.1; p = 0.001)

47

Economic Analysis Overview

Results

• Involves complex simulation of outcomes based on variable assumptions regarding:– The rate of interventions in observed vs non-observed

days– The rates of interventions on busy and less busy days– The rates of interventions with and without the

intervention prompt– The proportion of interventions performed that were

actually documented (recording rate)

• Study design allows for estimate of opportunity for intervention

48

Economic Analysis - Current Value

Results

All Australian pharmacies

232M scripts pa

Average Value of Interventions in

PROMISe data• 0.22 days in hospital• 1.23 consultations• $290 in total costs• 44 days of poor health

Value of interventions in all Australian pharmacies

• 262,424 days in hospital• 1.48M consultations• $349M in total costs• 53M days of poor health

PROMISe Sample

52 Pharmacies for 8 weeks

2396 Interventions435,000 prescriptions

PROMISe intervention data

2373 Interventions420,152 scripts

PROMISe Assessed Sample

291 Interventions1779 Assessments

16 AssessorsPROMISe

Assessed Sample

291 Interventions

Clinical Assessment

Process

1.6M interventions

49

Economic Analysis: Assumptions in Current Value Simulation

Results

• Recording rate on observed days 90%– Higher rate reduces final value

• Recording rate on Unobserved Days 50%– Higher rate reduced final value

• Attribution Rate 75%– Lower rate reduces final value

• Intervention Rate 0.69 per 100 scripts

50

Economic Analysis: Varying Assumptions in Current Value Simulation

Results

51

Effect of Activity (Workload)

13.1

7.8

4.8 4.8

3.4

0

2

4

6

8

10

12

14

1 (6) 2 (8.7) 3 (11.1) 4 (14.1) 5 (18.8)

Quintile of Activity (Scripts per hour)

Inte

rve

nti

on

ra

te p

er

10

00

pre

sc

rip

tio

ns

Increased workload decreased the intervention rate by 75% for all interventions (even in the same pharmacies)

34/52; 65%

Results

Representation of pharmacies across each quintile

45/52; 87% 46/52; 88% 47/52; 90% 42/52; 81%

52

Effect of Activity and Intervention Prompt

0.611.11

0.840.750.97

12.5

7.3

4.2 4.2

3.2

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

1 2 3 4 5

Quintile of Activity

Inte

rve

nti

on

ra

te p

er

10

00

pre

sc

rip

tio

ns

Aspirin Rate

Other Rate

Uptake of educational prompt was relatively resistant to increased workload

Results

53

Economic Analysis: Assessing the Increased Opportunity for Intervention by Improving Staffing levels

Results

• Based on intervention rate from second lowest quintile (45 of 52 pharmacies represented ; 87%)

• 28 hours of extra pharmacist time per 1000 prescriptions (~Cost $1300; ~ savings $910)

54

Economic Analysis: Assessing the Increased Opportunity for

Intervention by an Aspirin Prompt (or similar)

Results

• Based on intervention rate achieved in Aspirin pop-up arm with observation– Observation

mimics an education/ incentive program

• Additional $319M

55

Economic Analysis: Assessing the Increased Opportunity for

Intervention by Optimal Identification

Results

• Additional $606M

• Based on 2.08 ints/100 scripts

• Achievable with suitable motivation…

56

Motivators and Barriers for clinical interventions

• Key motivators were identified – Work environment– Clinical knowledge/continuing education– Professional satisfaction– Information continuity – Remuneration

• Barriers to documentation– Lack of time– Forgetfulness– Workflow restrictions– Software concerns

Results

57

Potential to increase interventions

• In the feedback on project, participants indicated that they would like to be able to carry out more interventions– Adequate staffing and staff mix– Continuing education– Identification of recordable incidents could be

further optimised

Results

58

Conclusions

• Current value of clinical interventions is high

• Considerable scope for increasing intervention rate (and value) with educational techniques– Up to threefold

Conclusion

s

59

Conclusions

Conclusion

s

60

Potential Roll-out Strategies for PROMISe

• Integrated “whole solution” approach– Repository model, – Feedback based on information received

(education and quantified), – Pharmacist access to individual results and

examples– “Push” information to pharmacies

• Incentive payment structure associated with targets for interventions

Conclusion

s

61

Recommendations

• Explore additional educational alerts• Expand economic analysis to other datasets• More detailed economic analysis to evaluate

different types of interventions• Identify factors associated with increased

intervention rates – Workload

• Obtain more representative information – Larger sample for longer period

• Simplify and modify classification system for more widespread use

Conclusion

s