european drug prevention quality standards equs conference, brussels, 15-17 june 2011
TRANSCRIPT
EUROPEAN DRUG PREVENTION QUALITY STANDARDS
EQUS conference, Brussels, 15-17 June 2011
Synergy between two projects
Prevention part of the EQUS standards was developed through an earlier, separate EC co-funded project
Project started 2008, completed in 2010
It was recognised that it was more efficient to adapt the existing prevention standards than to develop a new set of prevention standards synergy between both projects
However: different structure of standards, different methodologies etc.
Outline of the presentation
1. Introduction to the prevention standards project
2. Methodology of standards development
3. Structure/content of the prevention standards (brief!)
4. Adaptation of the standards to the EQUS project
5. Structure/content of the EQUS prevention standards
If you have any questions, pleasefeel free to ask at any time
Introduction to the Standards project
Project beneficiaries
Project partners: Liverpool John Moores University (LJMU), United Kingdom
(Project lead) Azienda Sanitaria Locale della Città di Milano (ASL), Italy Consejeria de Sanidad - Servicio Gallego de Salud (Xunta de
Galicia) (CS-SERGAS), Spain Azienda Sanitaria Locale n. 2 - Savonese (ASL2), Italy Institute for Social Policy and Labour (SZMI-NDI), Hungary National Anti-Drug Agency (NAA), Romania National Bureau for Drug Prevention (NBDP), Poland European Monitoring Centre for Drugs and Drug Addiction
(EMCDDA)
Adaptation of standards for EQUS project carried out by LJMU
Background & Aims
At the time of starting the project: No EU-level guidance on evidence-based drug prevention National or regional guidance available in some countries –
applicable to wider EU? USA standards of evidence – applicable to European
context? Lack of guidance for policy makers and practitioners
Aims: To bridge the gaps between science, policy and practice To produce a set of evidence-based drug prevention
standards for use in the EU To provide a checklist for policy makers and practitioners
Potential Impact of Standards
Policy and practice: Improve drug prevention practice (e.g. increase sustainability of
interventions) Improve efficiency of funding (better outcomes)
Target populations: Reduce likelihood of implementation of ineffective or iatrogenic interventions Increase relevance and acceptability of interventions for target populations
Research and evidence base: More evidence-based and scientifically sound interventions Improve European evidence base for prevention by promoting research
methodology Interested? European Society for Prevention Research (EUSPR)
(http://euspr.org)
Methodology of developing the Standards
Collecting information about available drug prevention guidance
12 EU countries with drug prevention standards or guidance(9 already on EMCDDA website, 3 new)
9 EU countries: no standards (yet) 6 EU countries: no information received
Available guidance – EU countries(spring 2009)
Drug prevention standards and/or guidelines available
No standards No information received
Czech RepublicDenmarkFinlandGermanyIrelandItaly (regionally)LithuaniaPolandPortugalRomaniaSpain (Galicia)United Kingdom
AustriaCyprus (in progress)*FranceGreeceHungary (in progress)LatviaNetherlands (in progress)SloveniaSweden
BelgiumBulgariaEstoniaLuxembourgMaltaSlovakia
* Cyprus: standards completed in late 2009
Methodology
Method Aims Implementation Timeline
Collation and review of existing guidance
To produce a long list of standards; to identify a common structure that will synthesise existing standards
77 documents retrieved, 19 documents selected
March-September 2009
First draft of standards
Methodology
Field testingUsability and feasibility of standards
72 professionals took part
August-September 2010
Method Aims Implementation Timeline
Delphi surveyPerceived priority of standards
423 professionals completed both rounds
January-February 2010
Focus groups(Cultural) relevance of standards
14 focus groups held March-April 2010
First draft of standards
Second draft of standards
Final standards
Participants in consultations
In six countries: Galicia (Spain), Hungary, Italy, Poland, Romania, UK
Sampling frame covered ten professional backgrounds: Regional drug teams or networks Education Health Mental Health Social services/ Children, young people, families Criminal Justice Voluntary/ Community sector Government representatives Prevention consultants Media
Online Delphi survey
Aim: to validate draft structure and to identify priority items Two rounds, with 2-3 weeks per round (January and February 2010)
First round: Participants rated the priority of each component from “High Priority” to
“Not a priority at all” Additionally, possibility to choose whether standards should be made
“mandatory” or not Text comments possible
Second round: Same as above, but – Also information on how all participants had rated the components in the
first round, and examples of text comments from the first round Only results from Round 2 are shown in this presentation
Response rates
EU ES HU IT PL RO UK
Recruitment: Persons contacted
987 56 114 210 319 82 206
Round 1: complete and valid responses
487 53 91 118 95 53 77
% of contacted 49% 95% 80% 56% 30% 65% 37%
Round 2: complete valid responses
423 53 83 94 84 46 63
% of Round 1 responses 87% 100% 91% 80% 88% 87% 82%
Delphi survey findings (excerpt)
Priority
Consensus
High priority
Mandatory yes
Priority Rank
1. Situation Analysis 1.1. Assessing drug use and community needs
2.73 74.6 % 87.7 % 12
1.2. Defining the problem - Justifying the intervention
2.63 66.4 % 87.3 % 16
1.3. Understanding the target population 2.88 88.7 % 92.9 % 12. Resource Assessment 2.1. Knowing policies and legislation 2.36 42.3 % 80.9 % 362.2. Identifying funding opportunities 2.35 < 50 % 43.4 % 72.1 % 372.3. Mapping existing stakeholders and services
2.48 53.7 % 83.4 % 30
2.4. Assessing target population and community resources
2.74 76.1 % 84.9 % 10
2.5. Assessing internal capacities 2.54 55.5 % 81.3 % 243. Programme Formulation 3.1. Defining the target population 2.88 88.8 % 94.5 % 23.2. Defining goals and objectives 2.83 83.5 % 90.0 % 43.3. Defining the setting 2.32 40.8 % 78.4 % 383.4. Summarising the evidence base 2.10 26.9 % 57.5 % 443.5. Illustrating the theoretical framework 1.97 20.1 % 60.0 % 463.6. Developing or selecting an effective intervention
2.75 78.6 % 89.5 % 8
3.7. Determining the timeline 2.28 < 50 % 40.4 % 76.3 % 423.8. Ensuring an ethical intervention 2.61 69.0 % 85.5 % 20
Note: Some UK results not included under "Mandatory", as missing values > 50%.
First draft standardsFirst draft standards
Delphi findings - summary
Across all partner countries, 23 out of 46 components obtained consensus and a higher priority rating
Components with highest / lowest priority ratings:
Components PriorityConsensu
sHigh
priority Mandatory
yesPriority Rank
1.3. Understanding the target population 2.88 88.7 % 92.9 % 1
3.1. Defining the target population 2.88 88.8 % 94.5 % 2
4.2. Adapting the intervention to the target population
2.86 87.9 % 89.7 % 3
3.4. Summarising the evidence base 2.10 26.9 % 57.5 % 44
6.3. Conducting a pilot intervention 2.05 < 50 % 30.1 % 42.4 % 45
3.5. Illustrating the theoretical framework 1.97 20.1 % 60.0 % 46
Low ratings – standards not necessarily for exclusion, but for discussion and modification
Focus groups: Relevance, usefulness,feasibility
Relevance: What is the current practice of professionals working in the area, and how do the draft standards relate to it? Are the draft standards appropriate for these professional groups and for their everyday work/projects?
Usefulness: How would the standards improve current practice? What are potential unintended consequences of the draft standards (e.g. economic, ethical)? Are the standards clear to understand and without ambiguity? Are the draft standards likely to be accepted by intended users?
Feasibility: How can the standards be integrated with current practice? Which standards are already common practice in most projects; which standards are “new” and yet to be achieved? Overlap with existing statutory frameworks? What further resources, workforce development/ training or support might be needed to implement the standards? What are the organisational, professional and other barriers to / opportunities for implementation?
Field testing (further focus groups)
Aims:
To discuss second draft standards Content of introduction and appendix Layout
To discuss possibilities for implementation Practical use of the standards Implementation tools
Consultations: Emerging challenges
Issues identified included: Knowledge on how to use standards, and what for - Promote the use of
quality standards Lack of content guidance (i.e. what to do) - Develop content guidelines Funding (e.g. for outcome evaluations, staff development), knowledge
on how to attract funding - Attract funding Prevention work is given less priority than treatment - Prioritise and
support prevention Lack or weakness of governmental prevention agencies - Establish and
strengthen central prevention agencies Diversity of prevention work - Consider differences in prevention practice Duplication of work - Create synergies Scepticism among professional groups - Develop professional attitudes
and skills
- Will be discussed by Dr Rachele Donini in tomorrow’s session
Using the Standards
Applicability of the standards
The standards are applicable to a wide range of drug prevention activities, for example:universal, selective, indicated or environmental prevention;preventing initiation of drug use, reducing the frequency of use and/or reducing drug-related harms;targeting legal and/or illegal drugs;structured manualised programmes or ongoing participant- and needs-led services;short- or long-term projects;different methods, target populations, settings …
The standards give advice on how to plan, implement, and evaluate interventions, and they can be used to reflect on new, ongoing, or completed activities.
Although the standards refer to programmes, they can be used to reflect on prevention work at several levels of delivery, including:
People: individual staff members or teams. Activities: singular interventions or wider programmes comprising several interventions. Organisations: organisations involved in drug prevention, such as service providers or schools. Strategies: priorities, action plans and tenders set out by government or funding bodies.
Applicability of the standards
Who contributes to drug prevention?
Policy- and decision-making: those who work on a strategic level, for example government representatives, commissioners, regional planning teams, funders.
Service management: those who are in charge of managing drug prevention activities at an organisational level within prevention providers.
Front-line work / work in direct contact with the target population: those who conduct, or contribute to, drug prevention in (direct) contact with the target population, such as psychologists, youth workers, social workers, outreach workers, teachers, pharmacists.
Who contributes to drug prevention?
Training: those who provide training in drug prevention. Training may be directed at practitioners (e.g. social workers), or it may take place in a higher education setting (i.e. university students).
Supervision: those who supervise and support drug prevention professionals (particularly practitioners).
Programme development: those who design and develop drug prevention interventions.
Consultancy / evaluation / academic research: those who provide consultancy on prevention issues, for example with regard to effective drug prevention, prevention policy development, adaptation of programmes, and/or evaluation.
Recommended uses of Standards
Purpose Recommended
Information, education and guidance
Developing or updating quality criteria
Self-reflection checklist
Discussion in group settings
Performance appraisals
Dissemination and evaluation plan
Purpose Recommended
Formal self-assessment Not yet
Funding decisions Not yet
External accreditation Not yet
Follow-on projects: Produce user-friendly implementation tools/manuals Trial standards with real programmes Develop accreditation system for model programmes Provide training/education to relevant target audiences International Standards initiative with CCSA, CICAD,
EMCDDA, UNODC (in early planning stage)
Importance of additional materials
Example scenarios of how the standards could be used General and country-specific conditions for (and potential barriers
to) implementation Implementation considerations for each of the 35 standard
components References to additional literature, materials, websites Bespoke glossary developed for use with the standards (over 100
terms) Self-reflection checklist allowing users to reflect upon their current
position in relation to the standards
The Prevention Standards
The drug prevention project cycle- a model to be adopted and adapted
Content of Standards
Cross-cutting Considerations
A: Sustainability and funding
B: Communication and stakeholder involvement
C: Staff development
D: Ethical drug prevention1 Needs Assessment
1.1 Knowing drug-related policy and legislation
1.2 Assessing drug use and community needs
1.3 Describing the need – Justifying the intervention
1.4 Understanding the target population
2 Resource Assessment
2.1 Assessing target population and community resources
2.2 Assessing internal capacities
Content of Standards
3 Programme Formulation
3.1 Defining the target population
3.2 Using a theoretical model
3.3 Defining aims, goals, and objectives
3.4 Defining the setting
3.5 Referring to evidence of effectiveness
3.6 Determining the timeline
4 Intervention Design
4.1 Designing for quality and effectiveness
4.2 If selecting an existing intervention
4.3 Tailoring the intervention to the target population
4.4 If planning final evaluations
Content of Standards
5 Management and Mobilisation of Resources
5.1 Planning the programme - Illustrating the project plan
5.2 Planning financial requirements
5.3 Setting up the team
5.4 Recruiting and retaining participants
5.5 Preparing programme materials
5.6 Providing a programme description
6 Delivery and Monitoring
6.1 If conducting a pilot intervention
6.2 Implementing the intervention
6.3 Monitoring the implementation
6.4 Adjusting the implementation
Content of Standards
7 Final Evaluations
7.1 If conducting an outcome evaluation
7.2 If conducting a process evaluation
8 Dissemination and Improvement
8.1 Determining whether the programme should be sustained
8.2 Disseminating information about the programme
8.3 If producing a final report
Example layout
Level 2: Component title
Implementationconsiderations
Glossary terms
Level 3: Attributes (basic)
Examples to clarifymeaning
Level 3: Attributes (expert)
Three levels of detail
1. Project stages: major areas of prevention practice covered by the standards
2. Components: actions to be taken at that point in the project
3. Attributes: show how achievement of the standards can be evidenced
Two quality levels
Basic standards: applicable to all drug prevention work, regardless of its particular circumstances.
Additional basic standards (“basic standard if”): reflect the basic level under particular circumstances.
Expert standards: in addition to the basic standards, representing a higher level of quality; desirable but may not always be applicable or feasible.
Adaptation to EQUS project
Adaptation of the standards
1) Description and re-assessment of original standards documents in line with template used for treatment and harm reduction areas
Development, applicability, legal status etc. of original standards
All original standards deemed relevant for EQUS project
2) Coding of individual ‘basic’ standards and components in line with standards structure developed by EQUS project coordinators
i.e. structure/ process/ outcome standards + sub-categories No need for new data extraction
Adaptation of the standards
3) Developing prevention standards for EQUS project
Adaptation at component rather than attribute level Reduction of 35 components to 30 EQUS prevention
standards Phrasing changed in order to correspond to treatment and
harm reduction standards, but meaning has been preserved Proposed list of minimal quality standards reflects a
summary of the basic standards submitted for EMCDDA publication
Adaptation of the standards
4) Revisiting results from Delphi survey to determine level of consensus for standards in line with methods used in treatment and harm reduction areas
Cut-off levels (> 80%, >50-80%, 50% or less) ‘Acceptability’ defined as percentage of those who thought
that a standard would represent a ‘mandatory’ part of good quality drug prevention work (based on first draft standards)
Remember that Delphi survey conducted with first draft standards – likely that final version would receive higher levels of acceptability
EQUS prevention standards
The standards can consequently be distinguished as follows: standards with high acceptability (rated as mandatory by > 80%
of responding participants) standards with medium acceptability (rated as mandatory by 50-
80% of responding participants consensus) – for discussion standards with low acceptability (rated as mandatory by less
than 50% of responding participants) (already excluded/modified through previous work).
Lists were established for structural standards at service level (5 + 3) outcome standards at system level (4 + 0) process standards at service/intervention level (11 + 7)
High acceptability – for information
Discussion:•Are the proposed lists of minimum quality standards (high consensus in surveys) acceptable?
•For which types of services / interventions are they accepted?
EQUS prevention standards
Structural Standards of Services
1. Ethical drug prevention: A code of ethics is defined. Participants’ rights are protected through informed consent. The programme has clear benefits for participants, and will not cause them any harm. Participant data is treated confidentially. The physical safety of participants and staff members is protected.
2. Staff composition: The staff required for successful implementation is defined and (likely to be) available (e.g. type of roles, number of staff). The set-up of the team is appropriate for the programme. Staff selection and management procedures are defined.
3. Policy and legislation: The knowledge of drug-related policy and legislation is sufficient for the implementation of the programme. The programme supports the objectives of local, regional, national, and/or international priorities, strategies, and policies.
Structural Standards of Services
4. Staff composition: Internal resources and capacities are assessed (e.g. human, technological, financial resources). The assessment takes into account their current availability as well as their likely future availability for the programme.
5. Financial requirements: A clear and realistic cost estimate for the programme is given. The available budget is specified and adequate for the programme. Costs and available budget are linked. Financial management corresponds to legal requirements.
Outcome Standards at the System Level
6. Goal: It is clear what is being ‘prevented’ (e.g. what types of drug use?). The programme’s aims, goals, and objectives are clear, logically linked, and informed by the identified needs. They are ethical and ‘useful’ for the target population. Goals and objectives are specific and realistic.
7. Outcome evaluation: Evaluation is seen as an integral and important element to ensuring programme quality. It is determined what kind of evaluation is most appropriate for the intervention, and a feasible and useful evaluation is planned. Relevant evaluation indicators are specified, and the data collection process is described.
Outcome Standards at the System Level
8. Monitoring the implementation: Monitoring is seen as an integral part of the implementation phase. Outcome and process data are collected during implementation and reviewed systematically. The project plan, resources, etc. are also reviewed. The purpose of monitoring is to determine if the programme will be successful and to identify any necessary adjustments.
9. Outcome evaluation: The sample size on which the outcome evaluation is based is given, and it is appropriate for the data analysis. An appropriate data analysis is conducted, including all participants. All findings are reported in measurable terms. Possible sources of bias and alternative explanations for findings are considered. The success of the programme is assessed.
Process Standards of Interventions
10.Assessment procedures: The needs of the community (or environment in which the programme will be delivered) are assessed. Detailed and diverse information on drug use is gathered. The study utilises existing epidemiological knowledge as possible, and adheres to principles of ethical research.
11.Assessment procedures: Sources of opposition to, and support of, the programme are considered, as well as ways of increasing the level of support. The ability of the target population and other relevant stakeholders to participate in the programme is assessed.
12.Assessment procedures: Justifying the intervention: The need for an intervention is justified. The main needs are described based on the needs assessment, and the potential future development of the situation without an intervention is indicated. Gaps in current service provision are identified.
Process Standards of Interventions
13.Assessment procedures: The target population is chosen in line with the needs assessment. The chosen target population(s) can be reached. The needs assessment considers the target population’s culture and its perspectives on drug use.
14.Intervention design: The programme is adequate for the specific circumstances of the programme (e.g. target population characteristics), and tailored to those if required. Elements to tailor include: language; activities; messages; timing; number of participants
15.Intervention design: The programme builds on positive relationships with participants by acknowledging their experiences and respecting diversity. Programme completion is defined.
Process Standards of Interventions
16.Intervention design: If selecting an existing intervention, benefits and disadvantages are considered, as well as the balance between adaptation, fidelity, and feasibility. The interventions’ fit to the needs assessment and other local circumstances are also examined. The chosen intervention is adapted carefully, and changes are made explicit.
17.Implementation: The programme is implemented according to the written project plan. The implementation is adequately documented, including details on failures and deviations from the original plan.
18.Process evaluation: The implementation of the programme is documented and explained. The following aspects are evaluated: target population involvement; activities; programme delivery; use of financial, human, and material resources.
Process Standards of Interventions
19.Staff development: It is ensured prior to the implementation that staff members have the competencies which are required for a successful programme implementation. If necessary, high quality training based on a training needs analysis is provided. During implementation, staff members are supported in their work as appropriate.
20.Dissemination: The final report documents all major elements of programme planning, implementation, and (where possible) evaluation in a clear, logical, and easy-to-read way. Details on implementation experiences and unintended outcomes are included. Legal aspects of reporting on the programme are included (e.g. copyright).
Moderate acceptability – for discussion
Discussion:•Which standards from the presented additional lists of quality standards (medium consensus in surveys) should be included in the definite lists of minimum standards?
•For which types of services / interventions?
EQUS prevention standards
Structural Standards of Services
1. Sustainability: The programme promotes a long-term view on drug prevention and is not a fragmented short-term initiative. The programme is coherent in its logic and practical approach.
2. Planning the programme: A systematic programme plan is constructed. A written project plan outlines the main programme elements and procedures. Contingency plans and risk management strategies are developed.
3. Sustainability: A programme is continued on the basis of evidence provided by monitoring and/or final evaluations. If it is to be continued, opportunities for continuation are outlined. The lessons learnt from the implementation are used to inform future activities.
Outcome Standards at the System Level
(none – all high acceptability)
Process Standards of Interventions
4. Communication and stakeholder involvement: The multi-service nature of drug prevention is considered. All stakeholders relevant to the programme (e.g. target population, other agencies) are identified, and they are involved as required for a successful programme implementation. The organisation cooperates with other agencies and institutions.
5. Physical environment: The setting(s) for the activities is (are) described. It matches the aims, goals, and objectives, available resources, and is likely to produce the desired change. Necessary collaborations for implementation of the programme in this setting are identified.
6. Recruiting and retention: It is clear how participants are drawn from the target population, and what mechanisms are used for recruitment. Specific measures are taken to maximise recruitment and retention of participants.
Process Standards of Interventions
7. Intervention design: The programme is based on an evidence-based theoretical model that allows an understanding of the specific drug-related needs and shows how the behaviour of the target population can be changed. Scientific literature reviews and/or essential publications on the issues relating to the programme are consulted. The reviewed information is of high quality and relevant to the programme. The main findings are used to inform the programme.
8. Intervention design: Materials necessary for implementation of the programme are specified. If intervention materials (e.g. manuals) are used, the information provided therein is factual and of high quality.
9. Intervention design: A written, clear programme description exists and is (at least partly) accessible by relevant groups (e.g. participants). It outlines major elements of the programme, particularly its possible impact on participants.
Process Standards of Interventions
10.Implementation: Flexibility is possible if required for a successful implementation. The implementation is adjusted in line with the monitoring findings, where possible. Issues and problems are dealt with in a manner that is appropriate for the programme. Adjustments are well-justified, and reasons for adjustments are documented. (Not rated in Delphi survey. Included as a result of target group consultation)
Next steps
For European drug prevention quality standards project Publication of standards by EMCDDA
For EQUS project To determine which of the adapted EQUS prevention
standards are acceptable/feasible as EU minimum quality standards – AT THIS MEETING
Suggestions for better adaptation and integration into the EQUS project? – AT THIS MEETING
LJMU will submit revised list of standards to project coordinator following meeting
EMCDDA Manual
Full list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).Full list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).
Contacts
Angelina BrotherhoodPublic Health [email protected]
Dr Harry SumnallReader in Substance [email protected]
Centre for Public HealthLiverpool John Moores University, UK