evaluation of inhaler technique improvement project · evaluation of inhaler technique improvement...

42
Evaluation of Inhaler Technique Improvement Project August 2012 The Cambridge Consortium

Upload: hoanglien

Post on 28-Jul-2018

224 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

Evaluation of Inhaler Technique Improvement Project

August 2012

The Cambridge Consortium

Page 2: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

2 | P a g e

Contents

Executive summary Section 1 Introduction Section 2 Background Section 3 Methodology Section 4 Findings Section 5 Strategy Acknowledgments References Appendices

Page 3: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

3 | P a g e

Executive summary

This report comprises an evaluation of an Inhaler Technique Improvement Project delivered in South Central Region between April 2011 and June 2012. Inefficient inhaler technique is a common problem resulting in poor drug delivery, decreased disease control and increased inhaler use. The costs to the nation and to patients are significant (Section 1.2).

The main findings of the evaluation are as follows:

The Inhaler Technique Improvement Project was a success on a number of levels: - at the level of the individual patient (improved outcomes and quality of life), - at a health systems level (improvements in emergency admissions), - in innovation (use of IT system and inhaler devices), - in education terms (enhanced skills applied in respiratory and other health related

areas); and - in a more generic sense of the HIECs bringing together and integrating all component

areas into a cohesive and effective entity (Section 4.1.1).

From the start of the project in April 2011, over four thousand Asthma Control Test (Medicine Use Reviews) MURs were conducted and almost 600 2nd (“follow up”) Asthma Control Test MURs were completed; 828 COPD Assessment Test pre- and post MURs were conducted (Section 4.1.1).

The creative use of well designed ‘second intervention’ MURs enabled participants, both patients and professionals, to see the impact of the intervention very quickly. In this way motivation was maintained and the credibility of the project was enhanced (Section 4.1.1).

A high number of Community Pharmacies engaged with the project – 206 up to and including August 2012 (Section 4).

The project delivered substantial improvements in the management of both asthma and COPD (Section 4.1.2).

Data on emergency admissions suggests a positive association between the introduction of the inhaler technique improvement project and changes in hospital emergency admissions. A more detailed analysis would be required to look at any more meaningful (i.e. statistically) significant correlation (Section 4.1.3).

The training delivered by the project was well received by patients and, following the training, pharmacists were able to deal effectively with patients’ concerns (Section 4.1.3).

Other skills benefits which were reported included the view that patients: increased their knowledge of respiratory conditions and how to control them better, developed greater confidence in controlling these conditions and were able to use “simple tools” to aid management of conditions (Section 4.1.4).

A number of innovative approaches were successfully applied to the project (Section 4.2). These included

o Innovative approaches to project management and support by the HIECs o The use of ESMAQ (Enhanced Services Monitoring and Quality System) o The use of inhaler training devices: ‘2-tone inhalers’ and ‘In-Check’ dials o The systematic use of asthma and COPD assessment tests o The innovative use of second intervention MURs o Innovative approaches to training and development

Page 4: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

4 | P a g e

Overwhelmingly, PCT project leads spoke positively of the power of the “Enhanced Services Monitoring and Quality” system (ESMAQ) to monitor, inform and motivate. They also reported that ESMAQ also helped pharmacists and other professionals to plan and deliver MURs. In addition, patients were reported as finding access to ESMAQ motivating, especially when it displayed evidence of improvement in the management of the condition between successive MURs (Section 4.2.2).

Participants found the inhaler training devices useful, although supply problems with the 2-Tone inhaler meant that it was not possible to assess fully its contribution.

Respondents valued both the outcome measures used, the Asthma Control Test and the COPD Assessment Test – especially when results were recorded onto ESMAQ and used to monitor patient scores between the first and second MURs (Section 4.2.3).

The project stimulated an innovative use of MURs incorporating elements of training in the use of inhalers and structured monitoring and feedback. Community Pharmacy respondents and PCT leads identified that the incorporation of ESMAQ into the MUR process and the systematic use of CAT and ACT tests gave the MURs a structure which was both helpful and motivating for patients and practitioners (Section 4.2.4).

The delivery of effective, high quality education and training in the use of tools and techniques to address the problem of poor inhaler use was of critical importance to the success of the project. The most successful models of training involved approaches linked to intended project outcomes. While such systematic approaches are well established and well documented elsewhere in the educational literature, they were perceived as innovative by both the project leads and the recipients of the training (Section 4.2.5).

The project was successful in enhancing the skills of participants with all but one of the PCT project leads and 98% of respondents to the on-line Pharmacy survey reporting strong evidence of skills enhancement as a result of the project (Section 4.3).

Although a detailed rigorous cost-benefit analysis was beyond the scope of this study, there is evidence that was a relatively low cost-high impact intervention (section 4.4).

In terms of sustainability, there is evidence that since the conclusion of the project the number of MURs related to inhaler technique has continued to increase. This is very positive considering that a number of key players in many areas across the region have changed role. In addition, several PCT leads reported that the approaches developed during the delivery of the project have been successfully applied to other initiatives, including, for example, smoking cessation (Section 4.5).

The findings from this evaluation have been drawn together to inform an outline strategic model for organisations wishing to implement the Inhaler Technique Improvement approach (Section 5). This model has been devised by a systematic critical appraisal of key features and characteristics emerging from the NHS South Central experience; the strategy is referenced to the NHS Change Model. Such a strategy and information, in the right hands, will help to secure the cultural transition in the practice of Community Pharmacy and its on-going contribution to an integrated care pathway for all patients. The implementation of this strategy will deliver real benefits to patients, to enhancing the skills of pharmacists and to the cost-effectiveness of NHS provision.

Page 5: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

5 | P a g e

Section 1 Introduction

Thames Valley and Wessex Health Innovation and Education Clusters (HIECs) commissioned the Cambridge Consortium to evaluate the outcomes of an Inhaler Technique Improvement Project that was undertaken across the South Central region in the period 2011-2012.

1.1 Overview of the project

Thames Valley and Wessex HIECs explored ways of improving inhaler technique across the region within both the NHS and independent sector.

The project was delivered in the following PCT areas: Berkshire East, Berkshire West, Buckinghamshire, Hampshire, the Isle of Wight, Milton Keynes, Oxfordshire, Portsmouth City and Southampton City.

The original project had five main objectives: - roll out of access to inhaler training for inhaler users in all Primary Care Trusts across

South Central region; - training of healthcare professionals; - development of quality standards and an exemplar service specification; - ensure measurement of success / deliverables; and - build in sustainability strategies beyond the project phase.

Within each PCT area, lead managers were established who were responsible for training pharmacists and other health professionals in the effective use of inhaler devices. Community Pharmacists then incorporated this training into Medicine Use Reviews. These MURs enabled data to be collected regarding the control of asthma and COPD symptoms. This data was collected and recorded by Community Pharmacists on the “Enhanced Services Monitoring and Quality” (ESMAQ) system. This enabled comparisons to be made between initial and subsequent MURs. This project was supported by a number of innovative features such as low-technology training devices and an innovative approach to pharmacists’ education which will form part of this evaluation.

1.2 Purpose of this evaluation

Initial indications from a review of both patient and pharmacy data suggested that the Inhaler Improvement Technique Project initiative had delivered real and tangible benefits to both patients and the health system. This evaluation seeks to examine these preliminary findings in the context of regional data and to draw conclusions and recommendations for further action.

The evaluation report:

- evaluates the impact of the initiative that can be drawn from both ESMAQ and Emergency Admissions data; and

- highlights components of success from PCTs that could be utilised when advising others of the most effective approach.

Page 6: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

6 | P a g e

Section 2 Background

2.1 Rationale for the Inhaler Technique Improvement Project

Inefficient inhaler technique is a common problem resulting in poor drug delivery, decreased disease control and increased inhaler use1. A small scale prospective study found that 92% of users have a poor inhaler technique; moreover, 94% of healthcare staff teach a poor technique2. These statistics have wide ranging implications including poor patient experience, increased prescribing costs and increased emergency admissions.

The main cause of poor inhaler technique is too high an inspiration rate leading to most of the inhaled medication being swallowed instead of inhaled. The medicine is largely wasted, thus reducing the efficacy of the medication. Low efficacy leads to poorer disease control and higher rates of exacerbation.

The clinical context within which the Inhaler Technique Improvement Project was delivered was framed by the Outcomes Strategy for COPD and Asthma, published by the Department of Health in 20123 which identified that:

For COPD:

one person dies from COPD every 20 minutes in England - around 23,000 deaths a year. If the whole NHS were to deliver services in line with the best around 7,500 lives could be saved;

death rates from COPD in the UK are almost double the EU average. 15% of those admitted to hospital with COPD die within three months and around 25% die within a year of admission;

COPD is the second most common cause of emergency admissions to hospital and one of the most costly inpatient conditions to be treated by the NHS.

In terms of Asthma:

there are around 1,000 deaths from asthma a year in the UK, the majority of which are preventable;

the UK has the highest prevalence of asthma in the world, at around 9-10% of adults;

just under £1billion is spent on respiratory inhalers with £170 million spent on one product alone (the Seretide 250 evohaler).

Regionally, it was recognised that earlier work conducted by the Isle of Wight PCT had made significant progress in improving inhaler technique through specific training for health care practitioners who then delivered training to inhaler users. As part of this, they utilised a locally developed innovative 2-Tone training inhaler. It was anticipated that the Inhaler Technique Improvement Project offered potential to improve the health of inhaler users across the region by enabling health care practitioners to assess, teach and reinforce patient’s inhaler technique.

1 Brocklebank D, Ram F, Wright J, Barry P, Cates C, Davies L, Douglas G, Muers M, Smith D, White J. Comparison of the

effectiveness of inhaler devices in asthma and chronic obstructive airways disease: a systematic review of the literature Health Technol Assess. 2001; 5(26):1-149. 2 Personal correspondence G Warner 6 August 2012

3 An Outcomes Strategy for COPD and Asthma: NHS Companion Document Department of Health London 2012

Page 7: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

7 | P a g e

Management of the project The initiative was managed through the Thames Valley and Wessex HIECs. NHS South Central provided funding to enable PCTs to develop the ideas generated on the Isle of Wight into bespoke projects, building upon work that had already been done to a greater or lesser extent within their areas. Funding was allocated to PCTs based upon individual project needs and was used to pay for training, equipment, additional MURs, publicity materials and project management support.

In parallel, funding was also provided to enable PCTs to purchase the Enhanced Services Monitoring and Quality (ESMAQ) system to enable pharmacies to record respiratory Medicine Use Reviews (MURs), including patient outcomes, and enable PCTs and the SHA to access real time data.

Within the project, two innovative devices to improve patient inhaler techniques were tested. These were the ‘2-Tone inhaler’ and the ‘In- Check’ device – see section 4.2.2 below).

2.4 The HIEC approach At the outset, it was decided that the funding would be used to build upon, and reinforce local PCT initiatives rather than replace them. This, it was felt, would help secure “buy in “ to the project and build on “bottom-up” initiatives thereby enabling an assessment of the effectiveness of different approaches.

The project design also stimulated sharing of information between PCTs and between PCTs and external agencies.

It had been identified prior to the commencement of the project that GPs posed a risk to realising the full benefits of increased inhaler technique training, due to a poor perception of the concept and practice of MURs. The project thus offered an opportunity to develop a whole systems approach to promote the concept of GPs and Community Pharmacies working together, and to change the perceptions of GPs.

Page 8: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

8 | P a g e

Section 3 Evaluation Methodology

The approach adopted for this study was designed to appraise the impact of the Inhaler Technique Improvement Project and draw valid conclusions and recommendations as to future large scale programmes to address the management of respiratory disease through improvements in inhaler technique. The approaches taken are summarised below. Interviews with HIECs and PCT managers Telephone interviews were conducted between May and June 2012 with the project leads of all participating PCTs (see Appendix 1a and 1b for interview schedules). In addition, two separate interviews were conducted with the responsible project managers of the Thames Valley and Wessex HIECs. Finally, feedback was obtained from one Chief Executive Officer of a local Pharmaceutical Committee. Pharmacy Perspective An online survey was designed and piloted for the specific use of pharmacists in the evaluation of the inhaler programme (see Appendix 2). Provision was made also to enable participants to respond to the survey via a postal method. Dissemination of the survey was carried out through the lead PCT managers in the first instance; a reminder was issued as the initial response rate, although attracting responses from all nine PCTs, was low. The online survey received fifty eight responses; data from the postal survey (N=8) was transcribed by hand and entered into the main data set. ESMAQ data CiREM was given access to all raw data and reports from PCTs contained on the ESMAQ system covering the period July 2010 to July 2012. Data is entered onto the ESMAQ system at source i.e. by individual participating pharmacists though a structured online pro forma. Emergency Admissions data CiREM was given access to both asthma and COPD emergency admissions data from the Information Services department of NHS South Central. Data for the comparator sites (Dorset and Wiltshire) was also received. The data is part of the standard NHS reporting system and is therefore validated at source. Statistical analysis Statistical analysis of the ESMAQ data involved the Chi-Square test; this is applicable for data organised in groups or categories.

This was a retrospective study and analyses of previously collated data sets. Moreover, the data related to an opportunistic sample of patients and therefore due caution should be exercised when making any generalisation of the findings. The Institute analysed the provision of revised data sets from ESMAQ.

Page 9: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

9 | P a g e

Section 4 Findings

This section provides a summary arising from an analysis of i) outcomes data (i.e. ESMAQ generated data through MURs) ii) impact data (i.e. emergency admissions data from NHS South Central) iii) interviews with PCT project managers, iv) interviews with the HIEC project managers and finally, v) an online survey of participating pharmacists.

The findings in this study point to a highly successful programme on a number of levels: - at the level of the individual patient (improved outcomes and quality of life), - at a health systems level (improvements in emergency admissions), - in innovation (use of IT system and inhaler devices), - in education terms (enhanced skills applied in respiratory and other health related areas); - in a more generic sense of the HIECs bringing together and integrating all component areas into a cohesive and effective entity.

It is noted that the success of such a programme (good evidence for an effective, efficient and deliverable solution to long term conditions) comes at a time of considerable change and increasing demands on individuals and organisations within the NHS.

The findings have been grouped into the following headings:

4.1 Patient engagement and benefit

4.2 Innovation

4.3 Improvements in skills

4.4 Value for money

4.5 Sustainability 4.1 Patient engagement and benefit This project was successful in gaining the active involvement of the Community Pharmacists, increasing the numbers of MURs conducted, and in helping to control better patient’s asthma and COPD scores.

The report provides analyses of the following data: - 4600 Asthma Control Test pre and post MURs; and - 448 COPD Assessment Test pre- and post MURs.

The use of ‘second intervention’ MURs was a key success factor. These enabled participants, both patients and professionals, to see the impact of the intervention very quickly. In this way, motivation was maintained and the credibility of the project was enhanced.

The number of engaged pharmacies was another key success. Again, this varied between PCTs. As of August 2012, there were 206 Active Pharmacies, which had delivered 5168 first intervention MURs and 886 second intervention MURs.

Further details, including the delivery of MURs by PCT area are given in Appendix 3.

Page 10: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

10 | P a g e

4.1.2 Outcome data (ESMAQ) Outcome data consists of changes in symptom control by patients collected through initial and follow-up MURs. This showed significant improvements both in the management of asthma and COPD. The Asthma Control Test (ACT) was used to measure outcomes for patients with asthma and the COPD Assessment Test (CAT) for patients with COPD.

In relative terms, 40% of asthma patients showed better asthma control over the time period; 55% of COPD patients showed an improvement in symptom management.

Asthma Control Test (ACT) The ACT asks five questions related to a patient's asthma control over the previous 4 weeks; each question enables a score from one to five in terms of control. The total score therefore has a range from 5 to 25 with higher scores indicating better control.

If a patient scores 19 or less, the asthma is considered not well controlled, on the basis of validation studies4. In general, the project should, if it is successful, evidence higher scores for patients returning for second intervention MURs. 4004 patients were seen at the 1st intervention with ages as shown in Table one; the average age was 49. The ACT scores for the first and second MUR are provided graphically in figure three.

Figure 3 ACT Scores at 1st and 2nd MUR

Note: Raw data for these scores is given in Appendix 4.

The ACT scores show evidence of improved asthma control between the 1st and 2nd MURs. There was a 40% relative increase in range 20-25 score over the above time period - the increase in the higher score on the ACT test shows better asthma control. The improvement in scores is significant at p<0.0015.

Note: Details of the statistical analyses can be found in Appendix 5.

4 For example, see http://www.medscape.org/viewarticle/558605

5 A significant result at the 95% probability level tells us that our data are good enough to support a conclusion

with 95% confidence (but there is a 1 in 20 chance of being wrong). In bio-medicalwork we accept this level of significance as being meaningful.

Page 11: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

11 | P a g e

COPD Assessment Test (CAT) The COPD Assessment Test provides a numerical score, which is a measure of the overall impact of COPD on a patient’s life. A higher CAT score indicates a more severe impact of COPD on a patient’s life. The aim of the intervention was thus to achieve a lowering of the CAT score.

The CAT is made up of 8 items, each scored on a numeric scale of 0 (no impact) to 5 (very severe impact). Each item is weighted equally for the final score, giving a range of CAT scores from 0–40. In general the project should, if it is successful, evidence lower scores for patients returning for second MURs. 808 patients were seen at the first MUR as shown in Table 2; the average age of patients seen was 67.

The CAT scores for the first and second MUR are provided in table four and graphically displayed in figure four.

Figure 4 CAT score by 1st and 2nd MUR

Note: Raw data for these scores is given in Appendix 4.

The CAT scores show evidence of improved COPD management following the intervention.

When the data for scores in range 1-10 is compared between 1st and 2nd MUR against the amalgamated scores (i.e. 11-40), there are more patients in a lower score banding indicating improvement in COPD management; the improvement in scores is significant at p<0.05.

Note: Details of the statistical analyses can be found in Appendix 5.

Page 12: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

12 | P a g e

4.1.3 Impact data Impact data was derived from standard reporting functions undertaken by the Information Services Department at NHS South Central. As such, they constitute validated data sets for emergency admission to hospital for both asthma and COPD. A more detailed presentation of this data is given in Appendix 6.

The impact data on emergency admissions suggests that there is a positive association between the introduction of the inhaler technique programme and changes in emergency admissions – though no firmer conclusions beyond an association can be drawn at the present time owing to a number of contextual factors. Such factors include for example weighting for smoking prevalence, weighting for co-morbidity, age ranges and medians, the presence (or absence) of unique and specific inhaler improvement programmes in intervention and comparator regions. A more detailed analysis would be required to determine any more meaningful (i.e. statistically significant) correlation.

4.1.4 Patient interface The project helped ensure that the advice given by pharmacists was well received by patients and enabled pharmacists to deal with patients’ concerns. In addition, it raised pharmacists’ understanding of integrated care pathways.

Figure 1 Patient interface

Page 13: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

13 | P a g e

Patient knowledge and skills

There was strong support from pharmacists for the view that the advice given as a result of the training was well received by patients (93% agreed or strongly agreed) and that, following the training, pharmacists were able to deal effectively with patients’ concerns (88% agreed or strongly agreed). The statement on better understanding of integrated care pathways evoked a more varied response and this area is perhaps worthy of further examination (see figure 2).

In addition to patient interaction, the PCT leads reported on other skills benefits for patients. These included the fact that patients:

increased their knowledge of respiratory conditions and how to control them better;

developed greater confidence in controlling these conditions; and

were able to use “simple tools” to aid management of conditions.

4.2 Innovation

The following innovative approaches were applied to the project:

Innovative approaches to project management and support by the HIECS

The use of ESMAQ (Enhanced Services Monitoring and Quality System)

The use of inhaler training devices including 2-tone inhalers and In-Check dials

The systematic use of asthma and COPD assessment tests

The innovative use of second intervention MURs

Innovative approaches to training and development Each of these is now discussed. 4.2.1 Innovative approaches to project management by the HIECS

The HIECs played an important strategic role in the Inhaler Technique Improvement Programme. Critically, the approach taken was to support and reinforce local PCT initiatives rather than replace them. This helped secure “buy in” to the project and build on “bottom-up” initiatives. It also enabled an assessment of the effectiveness of different approaches between PCTs.

The project design also stimulated sharing of information between PCTs and between PCTs and external agencies, and enabled approaches to be refined and further improved as the project progressed.

The HIECs’ role in supporting, enabling and promoting existing agencies and partnerships in the region was widely appreciated and it is evident that the HIECs added to the development of a community of practice

A number of PCT project leads stressed the importance of: - having a collaborative approach to planning future projects which involved listening

to people on the ground and adapting the approach to the local context;

Page 14: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

14 | P a g e

- planning for sustainability of the intervention; and - accurately costing the direct and indirect time commitments necessary to make the

intervention successful.

Across the board, there was a general consensus from the PCT leads for a more formal approach to building a project lead network at an early stage in any future project.

4.2.2 ESMAQ ESMAQ is an innovative IT system developed in the Isle of Wight and owned by Pinnacle Health Partnership. It enabled pharmacies to record respiratory MURs, including patient outcomes. ESMAQ also enabled the collation of base line data and facilitated progress monitoring. It helped pharmacists see what was possible from data generated in other PCT areas. In addition, ESMAQ enabled PCTs and the SHA to access real time outcome data.

The online survey of pharmacists supported the evidence provided by the PCT leads, with 73% indicating that the data collection was straightforward and 63% indicating that the performance measures were appropriate ( 33% offered a neutral response and only 3% indicated disagreement).

Overwhelmingly, PCT project leads spoke positively of the power of the “Enhanced Services Monitoring and Quality” system (ESMAQ) to monitor, inform and motivate. They reported that ESMAQ also helped pharmacists and other professionals to plan and deliver MURs. This was supported by data from the pharmacy respondents. In addition, patients reported that they found access to ESMAQ motivating, especially when it displayed evidence of improvement in the management of the condition between successive MURs.

Regarding its ease of use, respondents generally found “ESMAQ easy to record and interpret” (see figure 8 below).

Figure 8 Review and Monitoring

Page 15: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

15 | P a g e

4.2.2 The use of innovative devices to improve inhaler technique The main cause of poor inhaler technique is too high an inspiration rate leading to most of the inhaled medication being swallowed instead of inhaled. The medicine is largely wasted, thus reducing the efficacy of the medication. Low efficacy leads to poorer disease control and higher rates of exacerbation.

Within the project, two innovative devices to improve patient inhaler techniques were used. These were the 2-Tone Inhaler and the In-Check Dial.

The 2-Tone inhaler is used to teach health care professionals and patients the correct inhaler technique –i.e. to breathe in slowly and deeply when they inhale. Too rapid inhalation results in the device emitting two sound tones; if the rate of inhalation is too slow, the inhaler emits no tone and if the speed of inhalation is correct, a single tone is emitted.

The In-Check Dial is another device which helps to identify whether a patient is using their inhaler correctly. The In- Check dial has the added advantage in that it can be set to simulate inhalation through different types of inhalers. It measures the speed at which air is inhaled when the patient blows through the mouthpiece. By comparing the patient's results with the optimum results, patients can be trained to use their inhaler correctly. The device comes with an optimum inspiratory flow card, which informs users of the optimum inspiratory flow rates for different types of inhalers.

Participants found the inhaler training devices useful. However, supply problems with the 2-Tone inhaler meant that it was not possible to assess fully its contribution beyond reporting that participants found it easy to use and anecdotal evidence suggested that it has considerable potential utility.

4.2.3 The systematic use of asthma and COPD assessment tests

Respondents valued both the outcome measures used, the Asthma Control Test and the COPD Assessment Test – especially when recorded onto ESMAQ and used to monitor patient scores between the first and second MURs. They also reported that patients found these scores useful and they assisted in building motivation as patients could see evidence of progress between successive MURs.

4.2.4 The innovative use of second intervention MURs

The project stimulated an innovative use of MURs incorporating elements of training in the use of inhalers and structured feedback. Community pharmacy respondents and PCT leads identified that the use of ESMAQ into the MUR process and the systematic use of CAT and ACT tests gave the MURs a structure which was both helpful and motivating for patients and practitioners.

Page 16: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

16 | P a g e

4.2.5 Innovative approaches to training and development

The delivery of effective, high quality education and training in the use of tools and techniques to address the problem of poor inhaler use was of critical importance to the success of the project. The most successful models of training involved systematic approaches linked to intended project outcomes. While such approaches are well established and well documented elsewhere, they were perceived as innovative by the project leads and the recipients of the training.

These approaches embraced:

Sessions designed to deliver observable changes in practice – integrating theory and practice. This involved clear linkages between the education and training and the intended outcomes of the project

Hands-on use of devices and tools

A “see one, do one teach one” approach to delivery

Multiple sessions, organised at times and venues which were accessible to the target audiences

Clear references to, and development in, the use of ESMAQ and its role in measuring impact both of the project as a whole and of attendees.

Maintaining support and communication – post the education and training events in order to maintain motivation and commitment

The use of external expert presenters

Establishing clearly defined expectations regarding the next steps: “(PCT Project lead) I kept in touch with them all regularly. When I saw a problem emerging I chased up the pharmacy to try and find out what was happening and to offer help.”

The DVD produced by the HIECs was well received, but for some PCTs its delivery was too late for incorporation into many of the education and training sessions. 4.3 Improvements in skills

All but one of the PCT project leads and 98% of respondents to the on-line Pharmacy survey reported strong evidence of skills enhancement as a result of the project.

The PCT Project leads reported that pharmacists:

- increased their clinical knowledge, were better able to assess their skill base and monitor their impact;

- improved the quality of their interaction with patients; and - improved their ability to deliver MURs.

“We critically helped pharmacists understand why each step in the process was important both for COPD and asthma. This aided practice”. (PCT project manager)

Page 17: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

17 | P a g e

The pharmacists who undertook the training supported this assertion, as is evidenced from their on-line survey responses. Ninety three per cent agreed or strongly agreed that the training had enabled them to implement the service confidently (see figure 9).

Figure 9 Training and development

4.4 Value for money

The costs of poor inhaler technique are highly significant. In terms of asthma, just under £1 billion is spent on respiratory inhalers with £170 million spent on one product alone (the Seretide 250 evohaler).

COPD is the second most common cause of emergency admissions to hospital and one of the most costly inpatient conditions to be treated by the NHS. Asthma is responsible for large numbers of attendances to Emergency Departments and admissions, the majority of which are emergency admissions, 70% of which may have been preventable with appropriate early interventions6.

A detailed rigorous cost benefit analysis was beyond the scope of this study. However, it is clear that this was a relatively low cost - high impact intervention.

It may be useful to summarise the main elements of cost and an outline of the major benefits in relation to the inhaler improvement programme. The harm brought about by non –adherence and poor adherence to appropriately prescribed medicines is a major public health issue in many, if not all, chronic illness. Previous studies7 though have pointed to the difficulty in preparing accurate cost data. Increased morbidity has a cost to the patient, to the care provider and in more general terms to society.

6 'Unsustainable' rise in Emergency admissions is avoidable and no longer affordable Nuffield Trust Press Release 5 July

2012 7 See for instance Elliot RA, Barber N, Horne R Cost effectiveness of adherence enhancing interventions- a

qualitative assessment of the evidence Ann Pharmacotherapy 2005;39:508-15

Page 18: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

18 | P a g e

Cost elements The cost parameters obviously relate to patient based resource use and costs. The direct cost elements in relation to the inhaler programme include:

Direct project funding to PCTs. An average of £8,000 was allocated to each PCT and this was utilised for education and training (venues and speakers), technical devices (the 2-Tone Inhaler and ‘In-Check Dial), publicity (information leaflets) and additional MURs.

ESMAQ. This was funded by the SHA at a cost of £2,650 for one year. In addition, most PCTs paid an additional £1,300 for increased functionality which enabled them to use ESMAQ to monitor pharmacy activity beyond respiratory MURs.

MURs. An MUR ‘quota’ was already funded by PCTs; however, most PCTs, through the project, focused these on respiratory MURs. In some cases, PCTs utilised an element of their project funding for additional MURs over and above the quota as mentioned above.

The indirect costs include: - Time of pharmacists: travel to venues - Opportunity costs : other activities that might been carried out by pharmacists - Health benefits foregone

Programme benefits The direct programme benefits of the inhaler programme include:

- Secondary care reduction in emergency admissions - Reductions in medicines waste - Reduced prescribing due to better symptom control - Patient knowledge, understanding and beliefs around medication - Patient quality of life - Patient outcomes (i.e. symptom control) - Skill of pharmacists in MURs - Enhanced contribution of pharmacists to a model of integrated care.

The indirect benefits of the inhaler programme include

- The application of the model of MURs and associated education being applied to other relevant settings and patient behaviours in community care of patients.

- Tacit knowledge (“know-how”) and the further development of a community of practice.

4.5 Sustainability

There is evidence that, since the conclusion of the project, the number of MURs related to inhaler technique has continued to increase. This is a positive indication of sustainability, considering that a number of key players in many areas across the region have changed role and that specific project funding has now ended. PCT leads and Pharmacists have suggested that, in order to maintain momentum, resource needs to be found to maintain and update the initial education and training programmes developed by the PCT leads and also to establish who will take overall responsibility for the initiative in the future.

Page 19: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

19 | P a g e

It is significant that several PCT leads reported that the approaches developed during the delivery of the project have been successfully applied to other initiatives, including, for example, smoking cessation and emergency contraceptive training. In addition, several PCT project leads commented that a number of Pharmacies identified potential for marketing enhanced services.

Finally, respondents also reported that the project helped raise awareness amongst GPs of the potential role of Community Pharmacists conducting MURs: “........ the emerging ESMAQ data helped change GP attitudes.”

Page 20: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

20 | P a g e

Section 5 Strategy for Promotion, Implementation and Evaluation

The findings from this evaluation have been drawn together to inform an outline strategic model for organisations wishing to implement the Inhaler Technique Improvement approach. This model derives from a systematic critical appraisal of key features and characteristics emerging from the NHS South Central experience. This proposed strategy is referenced to the NHS Change model which is shown in Figure 10 below. Figure 10 The NHS Change model

Each of these Strategic features is now described:

5. 1 Shared purpose The approach taken by the two HIECs within the regions clearly embraced the NHS Change model and their approach is commended to organisations wishing to replicate the success of the project. The HIECS took great care to connect people to the change in a very open and inclusive way. Shared purpose was developed at different levels in the change process and connected back to the overall improvements which were sought.

Excellent examples of this are referred to throughout this report and are also described in paragraphs 5.2-5.7 below.

Page 21: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

21 | P a g e

5.2 Leadership for change There needs to be a credible and respected individual who leads and takes overall responsibility for the programme. There is a range of important management functions associated with the effective implementation of this programme. The manager must be a highly motivated and respective individual who appreciates the day to day activities of pharmacists. Leadership will be required in the early stages of implementation, particularly when there is either inertia or stronger forms of resistance to engagement to the programme.

In terms of formative evaluation, it is vital that information is gathered on a number of levels including the effectiveness of the training (learner satisfaction and intention to change), the outcomes of the intervention (changes in patient behaviour and quality of life) and in the overall impact of the intervention to the health care system (emergency admissions and medicines management).

It should be noted that the experience in South Central indicated significant differences in performance between PCT areas according to the energy and enthusiasm of the designated lead person. It is thus vital at the outset to recruit to this leadership role individuals who are able, enthusiastic and who have the energy to initiate and maintain project momentum.

5.3 Spread of innovation As described in section 4 above, there were a number of innovative components which were delivered in support of the successful implementation of the project. These included the approaches to project management, the use of ESMAQ, the utilisation of inhaler training devices, the systematic use of asthma and COPD assessment tests, the innovative use of second intervention MURs and innovative approaches to training and development.

To an extent, all of these innovations were interrelated but those, which respondents singled out most frequently, included the introduction of ESMAQ and the way in which its use facilitated the development of a highly effective model of delivering MURs.

Networking between participating organisations was a key feature of the rapid uptake and achievement of early impacts within the project. Respondents suggest that any future developments play particular emphasis to this issue – especially during the early stages where “recent adopters” have significant potential to learn quickly from colleagues who are further along in the implementation cycle.

In addition, respondents strongly supported the idea of bringing in external expertise to assist in delivering the training and also suggested that Community Pharmacists who have successfully engaged with this intervention could be a valuable training resource.

It was noticeable that certain aspects of the project which were seen by participants as innovative, were seen by experts simply as “good practice”. An example of this was seen in terms of the approach to education and training. The lesson from this is that what really mattered was not so much that the approach was innovative per se, but that it was new to the participants and it was effective.

Page 22: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

22 | P a g e

5.4 Improvement methodology

The Inhaler Technique Improvement Programme has provided an evidence-based improvement methodology which will help ensure that organisations wishing to replicate similar changes will be assisted to deliver them in a planned, proven way following an established methodology.

The ESMAQ IT software programme provides a strong evidence base for the analysis of impact. It elicited positive critical feedback from both pharmacists and PCT managers and is strongly advocated for organisations wishing to implement similar improvement projects. It is suggested that further enhancement of this software could include regular reporting and e-alerts (see Nudge section below).

The need for high quality education and training from respected individuals is a fundamental part of the infrastructure for implementation. Good NHS educational materials and information packs have been produced and are available; medical devices such as the “2-tone inhaler” trainer and the In-Check Dial are readily available.

Education and training should address a noted skills gap in pharmacy training, namely that of the why, how and when to undertake follow-up in the instance of undertaking a Medicines Use Review (MUR). 5.5 Rigorous delivery Where the project was particularly successful, the approach to delivery involved a systematic approach of designing carefully for impact and then monitoring the impact of each component of project delivery.

One such function is to design, facilitate and co-ordinate educational workshops, resources, information packs, and stocks of medical devices; this requires considerable forward planning and project management skills. Another related function is to promote the training to ensure effective take-up, sign up to using the data entry and responding to feedback and review.

The importance of the HIECs’ management role in supporting, enabling and promoting existing agencies and partnerships in delivering a successful intervention was widely acknowledged and a number of PCT project leads voiced the opinion that the project would have not been delivered had it not been for the management lead given by the HIECs.

Leadership and management come together to digest and critically appraise the data and intelligence from the implementation of the programme. Reports from ESMAQ can help inform where targeted support would be best suited; in this way timely interventions can be delivered. It also includes the ability to take decisive action at the early stage of implementation to respond positively to feedback on basic infrastructure (training and materials) and informal soundings.

The lead project managers in the most successful PCT areas stressed to project participants, the intention to deliver practical skills and identified that the impact of the new skills would be evaluated. Participants thus knew that the training was designed specifically to generate improvements in performance not simply to improve participants’ knowledge.

Page 23: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

23 | P a g e

A balance has to be struck between the need for flexibility of delivery (e.g. e-learning, DVD and podcasts) and the opportunity to have a “hands-on” training session which enables informal networking and group discussion.

The human dimension of monitoring project delivery was significant, with the most effective PCT leads closely monitoring ESMAQ data and responding rapidly and proactively when service delivery data deviated from that which was expected.

5.6 Transparent measurement Particular strengths of the project included the use of systematic tests to measure COPD and Asthma control and the recording of these data in a manner which was easily accessible to Community Pharmacists, patients, PCT leads, the HIECS and the SHA. These tools are thus recommended to organisations wishing to replicate the work in South Central.

5.7 System drivers The timing and positioning of the programme should be placed within an existing contextual framework of working e.g. Health Living Pharmacy Initiative8 from a health profession perspective and to QIPP from a wider health systems perspective9. In this way, pharmacists view the required changes of behaviour as part of a coherent and consistent strategy for improving the service to patients, carers and users.

Integrated care pathways can also be interpreted as the continuing evolution of a vision for Community Pharmacies to manage patient, carer and user expectation on the one hand and an ever increasing demand from patients with long term conditions and co-morbidity on the other. This evolving vision of the role of Community Pharmacy may be challenged (or indeed blocked) by senior management within the organisation – hence the need for strong executive leadership in the programme.

The current contractual arrangement (e.g. ‘Advanced Service’) for community pharmacy10 has strengthened the position of accredited pharmacists to make a more targeted intervention with patients who present with long term conditions.

5.8 Engagement to mobilise A range of approaches was used to mobilise interest in the project, and these varied between PCTs. The most common approaches included:

Meetings , emails and telephone discussions with key stakeholders (including medical and regional committees, hospital Trust boards and GPs);

Establishing local project groups;

Establishing educational and training sessions for Community Pharmacists – outside of working hours in order to facilitate access; and

Using the local media to raise awareness amongst patients and the general public.

8 See http://www.nhsalliance.org/fileadmin/files/pdf/Pharma%2001.pdf

9 QIPP – Quality, Innovation, Productivity and Prevention – is a large scale transformational programme for the NHS,

involving all NHS staff, clinicians, patients and the voluntary sector. It will improve the quality of care the NHS delivers while making up to £20billion of efficiency savings by 2014-15, which will be reinvested in frontline care. 10

http://www.psnc.org.uk/pages/mur.html

Page 24: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

24 | P a g e

Nudge: maintaining change The nudge principle is based on the idea that change and maintenance in behaviour benefits the patient, the role of the pharmacist and the wider health care system. The ‘nudge’ includes placing visual reminders at the point of interaction, making the ‘default’ i.e. preferred action easier and simpler, and changing the way we actually carry out actions e.g. change the system of patient review.

There thus needs to be some way of keeping the programme live in the mind of pharmacists. The nudge to change and maintain behaviour in provision of ITIP education could be undertaken through the IT software (e.g. an automated e-alert) or through a more personal approach via a telephone call from the inhaler programme manager. Furthermore, the manager in question would have a good idea which format would best suit which pharmacy practice.

5.9 Summary This strategy and information, in the right hands, will help to secure the cultural transition in the practice of Community Pharmacy and its on-going contribution to an integrated care pathway for all patients. The implementation of this strategy will deliver real benefits to patients, to enhancing the skills of pharmacists and to the cost-effectiveness of NHS provision.

Page 25: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

25 | P a g e

Acknowledgements The Cambridge Consortium would like to acknowledge the following individuals and respective organisations for the time and assistance contributed to this evaluation report. We also acknowledge the contribution of Isle of Wight PCT in piloting the online survey for pharmacists.

Name Organisation

Carol Alstrom Isle of White PCT

Karen Ashton NHS South of England

Janet Bowhill NHS Portsmouth City

Kevin Brooks Wessex HIEC

Jane Butterworth NHS Buckinghamshire

Katherine Cheema NHS South of England

Julie Dandridge NHS Oxfordshire

Richard Freeman Thames Valley HIEC

Ben Hannah NHS Information Centre

Nikki Hughes Milton Keynes PCT

Kate Holburn NHS Oxfordshire

Claire Howard Department of Health

Paul Jerram Isle of White PCT

Sue Lawton NHS Southampton City

Gail Naylor NHS Information Centre

Kevin Noble Isle of Wight PCT

Dr Catherine O’Sullivan Thames Valley HIEC

Jason Peett NHS Hampshire

Elaine Sharpe NHS Buckinghamshire

Linda Sharpe NHS South of England

Lorette Sanders NHS Berkshire East

Carol Trower NHS Berkshire East

Gary Warner Regent Pharmacy (ESMAQ -Health Partnership LLP)

Maha Yassaie NHS Berkshire West

Page 26: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

26 | P a g e

References An Outcomes Strategy for COPD and Asthma NHS Companion Document Department of Heath London May 2012 Asthma Control Test see http://www.asthma.com/resources/asthma-control-test.html Community Pharmacy Osteoporosis and Falls Medicines Use Review and Service Overview Hampshire and Isle of Wight LPC with NHS Southampton Community Health Care August 2009 COPD Assessment Test see http://www.catestonline.co.uk/hcpanswers.htm Directed Medicine Use Review (MUR) for Asthma and COPD Patients – improving Inhaler Technique NHS Buckinghamshire March 2012 Elliott RA, Barber N, Horne R. Cost-effectiveness of adherence enhancing interventions: a quality assessment of the evidence. Ann Pharmacother 2005;39:508–15 Elliott RA , Barber N , Horne CR, Hartley E. The cost effectiveness of a telephone-based pharmacy advisory service to improve adherence to newly prescribed medicines Pharm World Sci (2008) 30:17–23 Guidelines on the use of “In-Check” as part of a review of inhaler technique NHS Sheffield Clinical Executive, April 2011 (http://www.sheffield.nhs.uk/respiratory/resources/incheckguidelines.pdf) Howard RL, Avery AJ, Howard PD, Partridge M. Investigation into the reasons for preventable drug related admissions to a medical admissions unit: observational study. Qual Saf Health Care 2003;12:280–5.

Inhaler Devices for the management of asthma and COPD Effective Health Care Vol 8 No1 2003 The Royal Society of Medicine Jerram P, Medicines Use Reviews reduce asthma admissions Pharmacy Management Vol25 Issue 3 2009 Lewis A. Noncompliance: A $100 billion problem. Remington Rep 1997;5:14–5. Zuger A. The ‘other’ drug problem: forgetting to take them. The New York Times 1998 Jun 2.

MUR - the Medicines Use Review and Prescription Intervention Service http://www.psnc.org.uk/pages/mur.html accessed 28 June 2012

Simpson SH, Eurich DT, Majumdar SR, Padwal RS, Tsuyuki RT, Varney J et al. A meta-analysis of the association between adherence to drug therapy and mortality. Br Med J 2006;333 (7557):15–21

Page 27: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

27 | P a g e

Appendices

Page 28: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

28 | P a g e

Appendix 1a Evaluation of South Central Inhaler Improvement Programme PCT Interview pro-forma

PCT Contact Name

Email

Tel No

Role in the programme

Date

Population size

Asthma burden

COPD burden

Approaches Training

Equipment

MURs No Asthma……. No COPD Low Med High Other comments

Publicity Timely, effective…?

Staffing Project management support HIECs role?

Symptom Control

Who is the most appropriate person at the PCT to talk to about the data ( + ESMAQ)

Winter Admissions

Who is the most appropriate person at the PCT to talk to about the data (+ ESMAQ)

Page 29: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

29 | P a g e

High MUR PCTs

Enabling factors (could they rank them ?)

Barriers

How did you surmount these barriers?

Skills Tangible/maintain them/

What did success mean in your terms

What single action would you do or change to improve the programme?

General comments Are there any other comments you wish to make on the programme?

Exit Is there anyone else you feel we should talk to about the programme?

Thank you

Page 30: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

30 | P a g e

Appendix 1b Evaluation of South Central Inhaler Improvement Programme HIEC Interview pro-forma

HIEC Contact Name

Email

Date

Tel No

HIEC Role Overview Strengths Development points for future/ Things which could have been improved

Comments on approach taken to training

Strengths of particular approaches Development points for future /Areas for improvement

Equipment

Strengths of particular equipment used to support the project Development points for future /Areas for improvement

Page 31: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

31 | P a g e

Publicity

Strengths of particular approaches Development points for future /Areas for improvement

Staffing

Please comment on your views on how the project was staffed and managed at the PCT level Development points for future /Areas for improvement

Any comments on data capture, collation

High MUR PCTs

Enabling factors of the more active/engaged PCTs (could you rank them ?)

What prevented some PCTs in becoming more engaged?

Page 32: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

32 | P a g e

Skills Tangible/maintain them/

What did success mean in HIEC terms?

What single action would you do or change to improve the programme?

General comments Were there unintended consequences Were there any indirect costs? Are there any other comments you wish to make on the programme?

Exit Is there anyone else you feel we should talk to about the programme?

Thank you

Page 33: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

33 | P a g e

Appendix 2

South Central NHS Inhaler Technique Improvement Programme

Survey of Community Pharmacy Perspective The Cambridge Consortium has been commissioned on behalf of the Thames Valley & Wessex HIECs to undertake an independent evaluation of the Inhaler Technique Improvement programme. Please return ‘hard copy’ back to your respective PCT Lead.

Note: There are five sections in the survey; place a cross (x) in one box

In order to analyse the data, could you please let us know in which area your pharmacy is located?:

PCT X

Berkshire East

Berkshire West

Buckinghamshire

Hampshire

Isle of Wight

Milton Keynes

Oxfordshire

Portsmouth City

Southampton City

Section 1 Contextual and cultural aspects

Strongly agree

Agree Uncertain Disagree Strongly disagree

Q1 There is ease of access to appropriate patients

○ ○ ○ ○ ○

Q2 Patients are willing to discuss inhaler technique

○ ○ ○ ○ ○

Q3 The pharmacy is able to see ‘hard to reach‘ patients

○ ○ ○ ○ ○

Section 2 Training

Strongly agree

Agree Uncertain Disagree Strongly disagree

Q4 The training provided enabled me to implement this service confidently

○ ○ ○ ○ ○

Q5 I received up to date information in order to be able to advise the patient

○ ○ ○ ○ ○

Q6 Support for the programme was well directed

○ ○ ○ ○ ○

Page 34: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

34 | P a g e

Section 3 Education

Strongly agree

Agree Uncertain Disagree Strongly disagree

Q7 The advice was well received by patients

○ ○ ○ ○ ○

Q8 I could deal with all patients’ concerns

○ ○ ○ ○ ○

Q9 The programme has enhanced my understanding of integrated care pathways

○ ○ ○ ○ ○

Section 4 Review and monitoring

Strongly agree

Agree Uncertain Disagree Strongly disagree

Q10 Data collation was straightforward

○ ○ ○ ○ ○

Q11 Key performance indicators were appropriate

○ ○ ○ ○ ○

Q12 Feedback on the programme has been timely

○ ○ ○ ○ ○

Section 5 Other general comments

Yes No

Q13 Were there any unintended consequences e.g. influence on risk factors/understanding of medication?

○ ○

If yes, please state

Yes No

Q14 Were there any other indirect costs associated with the programme?

○ ○

If yes, please state

Q15 What do you think the main success factors have been?

Q16 Any other general comments?

Page 35: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

35 | P a g e

Appendix 3 MUR data retrieved from ESMAQ Table 1: Delivery of MURs by PCT area

PCT No of MURs

Milton Keynes 1078

Portsmouth City 757

Berkshire East 704

Oxfordshire 687

Southampton City 617

Buckinghamshire 600

Isle of Wight 438

Berkshire West 163

Hampshire 98

Table two provides data on the number of MURs undertaken from June 2010 until August2012. Table 2: Respiratory MURs

Descriptor No

Active Pharmacies 206

Patients with First Intervention 5168

Patients with Second Intervention 886

Pharmacies delivering at least 5 Respiratory MUR 151

Pharmacies delivering 30 or more Respiratory MURs 58

Page 36: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

36 | P a g e

Appendix 4 ACT and CAT Scores after 1st and 2nd MURs

Table 1 ACT scores at 1st and 2nd MUR*

1st MUR 2nd MUR % change between 1st and 2nd MURs

ACT score by range 5-9 319 (8%) 15 (2.5%) -320%

10-14 831 (21%) 89 (15%) -40%

15-19 1162 (29%) 150 (25%) -16%

20-25 1612 (40%) 339 (56%) + 40%

NA 80 (2%) 3 (0.5%)

Total No of pts 4004 596

* figures have been rounded off

Table 2 CAT scores by 1st and 2nd MUR11

1st MUR 2nd MUR % change between 1st and 2nd MURs

CAT score by range 1-10 41 (18%) 62(28%) +55%

11-20 85(38%) 83(37%) -2.5%

21-30 78(35%) 68(30%) -13%

31-40 20 (9%) 11 (5%) -45%

NA 2(0%) - (0%)

Total No of pts 224 224

* figures have been rounded off

11

Figures are rounded off

Page 37: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

37 | P a g e

Appendix 5 Chi Square Analyses ACT Scores

Range Observed with (E) value Observed with (E) value Totals

5-9 319 (290) 15 (44) 334

10-14 831 (799) 89 (121) 920

15-19 1162 (1140) 150 (172) 1312

20-25 1612 (1695) 339 (256) 1951

Totals 3924 593 4517

Note (E) – Expected value X2= ∑[(O-E)2/E] X2 = 292/290 + 292/44 + 322/799 + 322/121 + 222/1140 + 222/172 + 832/1695 + 832/256 = 2.9 + 19.1+ 1.28 + 8.46 + 0.42 + 2.81 + 4.06 + 26.9 = 65.93 Degrees of freedom = 3 x 1 = 3 For the test to be significant at 0.001 level, given 3 degrees of freedom, the value of X2 has to be at least 16.266 CAT Scores - I

Range Observed with (E) value Observed with (E) value Totals

1- 10 41(51.5) 62 (51.5) 103

11- 20 85(84) 83 (84) 168

21- 30 78(73) 68(73) 146

31-40 20(15.5) 11(15.5) 31

Totals 224 224 448

Note (E) – Expected value X2= ∑[(O-E)2/E] X2 = 10.52/51.5 + 10.52/51.5 + 12/84 + 12/84 + 52/73 + 52/73 + 4.52/15.5 + 4.52/15.5 = 2.14 + 2.14+ 0.01 + 0.01 + 0.34 + 0.34 + 1.30 + 1.30 = 7.58 Degrees of freedom = 3 x 1 = 3 For the test to be significant at 0.05 level, given 3 degrees of freedom, the value of X2 has to be at least 7.815. Any effect in the range 1-10 is masked by lack of effect in the other ranges.

Page 38: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

38 | P a g e

CAT Scores – II Amalgamated ranges Amalgamating ranges 11-20, 21-30 and 31-40 to one range i.e. 11-40. Reducing the number of cells in this version of the test reduces the number of degrees of freedom and hence the size of X2 value required (3.841: 7.815) to achieve significance. In other words, there are fewer opportunities to pick up discrepancies between observed and expected frequencies and so the chances of deviations arising fortuitously are correspondingly diminished.

Range Observed with (E) value Observed with (E) value Totals

1-10 41(51.5) 62(51.5) 103

11-40 183(172.5) 162(172.5) 345

Totals 224 224 448

Note (E) – Expected value X2= ∑[(O-E)2/E] X2 = 10.52/51.5 + 10.52/51.5 + 10.52/172.5 + 10.52/172.5

= 2.14 + 2.14+ 0.64 + 0.64 = 5.91 Degrees of freedom = 1x1 = 1 For the test to be significant at 0.05 level, given 1 degree of freedom, the value of X2 has to be at least 3.841 Correction Factor For a 2x2 table and a smaller sample size, a correction factor is required namely the Yate’s Correction: X2 = 102/51.5 + 102/51.5 + 102/172.5 + 102/172.5 = 1.94 + 1.94 + 0.58 + 0.58 = 5.04 i.e. still larger than 3.841; significant at p <0.05

Page 39: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

39 | P a g e

Appendix 6 Trend data for Asthma and COPD Emergency Admissions

Trend data was examined for all ages and for the time period April 2009 till March 2012. this data is presented fully in Appendix 5. In order to look for specific additionality due to the inhaler programme, two comparator PCT sites were chosen for analysis12.

Asthma Figure 1 (NHS South Central) and figure 2 (anonymised South Central PCT) depicts the rolling average for asthma admissions (all ages) has decreased. However, comparator data for asthma is shown to be increasing over the same time period - as shown in figures 5 and 6.

Figure 1 Asthma emergency admissions (all ages) – NHS South Central

Figure 2 Asthma emergency admissions (all ages) – anonymised South Central PCT

12

Wiltshire and Dorset PCTs

Page 40: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

40 | P a g e

COPD Figure 3 (NHS South Central) and Figure 4 (anonymised South Central PCT) depicts the rolling average for COPD admissions (all ages) to have increased. However, comparator data for COPD is shown to be increasing as shown in figures 7.1 and 7.2.

Figure 3 COPD Emergency Admissions NHS South Central (all ages)

Figure 4 COPD Emergency Admissions Anonymised South Central PCT (all ages)

Page 41: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

41 | P a g e

Figure 5 Dorset Comparator

Figure 6 Wiltshire Comparator

Page 42: Evaluation of Inhaler Technique Improvement Project · Evaluation of Inhaler Technique Improvement Project ... decreased disease control and ... The Inhaler Technique Improvement

42 | P a g e

Contact

On behalf of The Cambridge Consortium:

Dr Chris Loughlan

Director

Cambridge Inst. for Research Education and Management (CiREM)

CiREM House

20 Teversham Rd

Cambridge CB21 5EB

www.cirem.co.uk

[email protected]