High Fidelity: Translating the evidence-base into real world settings

Dr Jo Holliday, Research Fellow, Cardiff University

Sally Good, Chief Operating Officer, DECIPHer Impact

Potential conflict of interest: Sally Good is the Chief Operating Officer at DECIPHer Impact Ltd, a not-for-profit company wholly owned by the Universities of Bristol and Cardiff set up to licence and support the implementation of effective health promotion interventions.

Background

Health promotion interventions inherently complex

Background

Health promotion interventions inherently complex

Presents challenges for standardization of delivery evaluation

Pragmatic trials incorporating process evaluation can provide insight into where variation occurs why it occurs and implications of this for wider implementation

Aims of study Discuss issues of fidelity of intervention delivery

encountered within A Stop Smoking in Schools Trial (ASSIST)

Describe the systems established to ensure translation of positive trial findings into real-world settings.

The ASSIST Programme

A school-based, peer-led smoking prevention programme Encourages new norms of smoking behaviour by training

influential Year 8 students to work as ‘peer supporters’ Peer supporters identified as influential by peer group Peer supporters trained during school time but off school site Peer supporters have informal conversations with other

students Support sessions held in school

Evaluation of the ASSIST Programme Evaluated in a pragmatic RCT with integral process evaluation Involved 10,720 students from 59 schools at baseline

Demonstrated a 22 % reduction in the odds of being a regular smoker in intervention schools compared with control schools (Odds ratio 0.78 CI: 0.64-0.96) using follow- up data collected at three time points over two years

MethodsStage Source Method and numberPeer nomination Researchers and sessional staff

involved in administration of peer nomination questionnaire in 30 intervention schools

Self-complete questionnaires (n= 319)

Peer supporter recruitment, training and follow-up sessions

Two researchers in four schools selected for in-depth process evaluation

Trainers who conducted the recruitment meeting, training and follow-up sessions in 30 intervention schools

Non-participant observationRecruitment (n= 3)Training (n= 4)Follow-up session 1 (n= 4)Follow-up session 2 (n= 3)Follow-up session 3 (n= 4)Follow-up session 4 (n= 3)Self-complete questionnaires (n= 583)Post-intervention semi-structured interviews (n= 11)

Post-intervention School staff in 30 intervention schools who were involved with the intervention

Self-complete questionnaires (n= 24)

ResultsVariations observed in terms of: Peer nomination and recruitment Venues Length of sessions Intervention timetable

Trainer to student ratios School staff involvement Training approach

Early implementation of ASSIST Learning from trial incorporated into detailed documentation

‘Training the Trainers’ guide ASSIST programme manual

Monitoring process of early roll out enabled refinement of programme Wales A London Borough A Primary Care Trust in the

South West

Quality Assurance Scores

10

What we have done to enable ASSIST to be implemented

Not-for-profit company - DECIPHer Impact formed in March 2010 Wholly owned by Cardiff University and the University of

Bristol Five board directors (two from each University and an

independent Chair) Chief Operating Officer and new Chief Executive Officer Offices near Bristol

What we have done to enable ASSIST to be implemented

Maintaining fidelity through DECIPHer Impact Ltd

Provision of training to ensure a consistently high standard Provision of a comprehensive programme Manual and regular

updates to good quality materials Support and monitoring of implementation A comprehensive Quality Assurance Framework including:

observation of delivery student feedback school feedback self-assessment

Evidence-based but not prescriptive

Recognition that every customer is different Provision of a framework and guidance to work within

based on our experience of implementing the programme Flexibility is built in and includes:

Traffic light system Suggestions to extend or shorten activities Examples of different ways to achieve objectives

Broad parameters to encompass different group sizes and backgrounds

Ongoing Customer Support through DECIPHer Impact Ltd

Academic guidance available to Company via Board of Directors

Working group provides customers with a direct conduit for feedback

Helpdesk ensures that customers can access support quickly and easily

Teleconferencing provides opportunities throughout the year for sharing best practice

Regular seminars and an annual conference enable sharing of best practice

Using ASSIST Sold under licence for 3 year periods Population based model Banded licence fees based on the number of 10-14 year olds

in a geographical area Option for separate areas to cluster together to benefit from

economies of scale An indication of costs is £42 per student, including the licence

fee, based on 6,000 students taking part in the programme over 3 years

UK rollout to date Over 20,000 young people participated in 2011-2012 Our 3 early adopters have continued to use ASSIST for

over 5 years 1 region of 13 Local Authorities 12 individual Local Authorities 2 Channel Islands 1 country

Contact and Acknowledgements

Jo Holliday, Cardiff University: hollidayj1@cardiff.ac.uk

Sally Good, DECIPHer Impact: sally.good@decipher-impact.com

ASSIST: Prof R Campbell, Prof L Moore, Dr J Holliday, Dr S Audrey

DECIPHer Impact: Directors: Prof R Campbell, Prof L Moore; Dr G Pierce-Jones,Dr D Sheader, Dr M Hughes, CEO: M Day, COO: S Good

A Stop Smoking in Schools Trial was made possible by funding from the Medical Research Council (grant number G9900538).

The writing of this paper was supported by DECIPHer, a UKCRC Public Health Research: Centre of Excellence. Funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council (RES-590-28-0005), Medical Research Council, the Welsh Assembly Government and the Wellcome Trust (WT087640MA), under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged.