The use of ‘colloquial evidence’ in HTA:

the experience of NICE

Health Technology Assessment international (HTAi) Annual Meeting26 June 2012: 17:30-19:00

(Abstract ID: 381)

Presenter: Thomas Wilkinson

Authors: Tarang Sharma, Bindweep Kaur, Moni Choudhury, Bhash Naidoo, Sarah Garner, Lucy Connor (NICE)

Evidence for developing guidance

• NICE produces guidance on best available evidence.

• Evidence:– “findings from research

and other knowledge that may serve as a useful basis for decision making in public health and health care.” (Health evidence network, WHO in Lomas et al, 2005)

Colloquial evidence

• Definition:

“anything that establishes a fact or gives reason for believing something.” (Oxford Dictionary, 1980)

• Provides context and complements scientific evidence

• Increasingly used in HTA

• ControversialAdapted from Lomas et al (2005)

Aims and objectives

• Aim:• To explore the use of colloquial evidence in the

process of developing a guidance across all guidance-producing centres at NICE.

• Objectives:– To understand the context of colloquial evidence

from the available literature.– To identify consistencies and variations in the use of

colloquial evidence across all centres at NICE.– To map the use of colloquial evidence at NICE to the

conceptual framework outlined by Lomas et al 2005.

Methods

Results: Mapping colloquial evidence at NICE

• All guidance-producing centres utilise colloquial evidence with scientific evidence, as per Lomas et al 2005.

Results: Colloquial evidence is utilised at all stages of guidance production

Expert/ Patient/Carer

input

Grey Literature

Stakeholder views

E.g. CG105 Motor Neurone Disease - Information and support needs of patients with motor neurone disease

and their families and carers • Systematic review searches

identified 11 eligible studies and a set of interviews from health talk online.

• All were critically appraised via qualitative studies checklist and analysed through a ‘Thematic-Conceptual Meta Matrix’ (Miles and Rosenblum 1987).

• Example of one of the evidence profiles looking at one outcome (timing, level of information and ways of communication).

NICE clinical guideline 105 – Non-invasive ventilation for motor neurone disease:http://publications.nice.org.uk/motor-neurone-disease-cg105/other-versions-of-this-guideline#quick-reference-guide

Discussion and Conclusion

• Colloquial Evidence is utilised alongside scientific evidence across NICE.

• Methodology for integrating all kinds of evidence:– Deliberative process: “an integration of technical

analysis and stakeholder and lay public deliberation, contrasting with the traditional ‘top-down’ or ‘bureaucratic rationalistic’ policy orientation” (Culyer et al, 2006)

– Lack of methodology?– MCDA approach?

Next Steps

• No formal tools for critically appraising colloquial evidence utilised across NICE:– Proposing SART Model

• Source• Accuracy• Relevance• Timeliness

• Following changes may impact the way Colloquial Evidence is utilised currently at NICE:– Methods and Process Manuals Update– Changing NICE’s remit – introduction of social care

guidance– Value based pricing

• Developing further methods of identifying and capturing colloquial evidence.

References

• Culyer AJ, Lomas J (2006) ‘Deliberative processes and evidence informed decision making in healthcare: do they work and how might we know?’ Evidence and Policy Vol. 2(3) pp 357-71.

• Lomas J, Culyer T, McCutcheon C et al. (2005) Conceptualizing and combining evidence for health system guidance: final report. Ottawa: Canadian Health Services Research Foundation

• Reay CA, Colechin ES, Bousfield DR et al (2011) ‘Review of published literature relating to methods for identifying, synthesis and integration of colloquial evidence’. Newcastle upon Tyne Hospitals NHS foundation trust

• 20 National Institute for Health and Clinical Excellence (2009). Centre for Clinical Practice. The guidelines manual 2009 Available at: http://www.nice.org.uk/aboutnice/howwework/developingniceclinicalguidelines/clinicalguidelinedevelopmentmethods/GuidelinesManual2009.jsp?domedia=1&mid=5F238D80-19B9-E0B5-D4CB1191544B5D45

• National Institute for Health and Clinical Excellence (2009) Centre for Public Health Excellence. Methods for development of NICE public health guidance 2009 Available at: http://www.nice.org.uk/aboutnice/howwework/developingnicepublichealthguidance/publichealthguidanceprocessandmethodguides/public_health_guidance_process_and_method_guides.jsp?domedia=1&mid=F6A97CF4-19B9-E0B5-D42B4018AE84DD51

• National Institute for Health and Clinical Excellence (2008) Implementation support process manual Available at: http://www.nice.org.uk/usingguidance/niceimplementationprogramme/nice_implementation_programme.jsp?domedia=1&mid=656BA97E-19B9-E0B5-D437A317C324122F

• National Institute for Health and Clinical Excellence (2011) Citizen’s Council http://www.nice.org.uk/aboutnice/howwework/citizenscouncil/citizens_council.jsp

Links/Contact detailsLinks:

Research and Development, NICE:

http://www.nice.org.uk/aboutnice/howwework/researchanddevelopment/about.jsp

Authors/Contributors: • Ms Tarang Sharma, Senior Analyst• Ms Bindweep Kaur, Analyst• Miss Moni Choudhury, Analyst• Dr Bhash Naidoo, Associate Director• Dr Sarah Garner, Associate Director• Ms Lucy Connor, Programme Manager

Contact:• Sarah Garner Tel: 44 (0)207 045 2097; • Email: Sarah.garner@nice.org.uk