WHO Guidelines for Guidelines 2008

Update and overview for Guideline Group Members

Revised process from 2008

New WHO Guideline review committee Revised WHO guidelines for guidelines Minimum standards for:

• Reporting • Processes• Use of evidence

Different types of guidance documents recognised to fit different purposes:

• Emergency• Standard• Full • 'Books '• Joint guidelines?

Minimum standards for reporting in WHO guidelines:

Who was involved and their declaration of interests

How the guideline was developed, including how the evidence was identified how the recommendations were made

Use by date (review by date)

Practicalities

For principle and/or controversial recommendations: Synthesis of ALL available evidence Evidence summaries for group meetings using standard

template Formal assessment of quality of evidence Consideration of resource use and costs Link evidence to recommendations, explaining reasons

for judgements

What type of outcomes should WHO consider

Important outcomes (e.g. mortality, morbidity, quality of life) should be preferred over surrogate, indirect outcomes, (e.g. CD4, cholesterol levels, lung function) that may or may not correlate with patient important outcomes.

Ethical considerations should be part of the evaluation of important outcomes (e.g. impacts on autonomy).

Desirable (benefits, less burden and savings) and undesirable effects should be considered in all guidelines.

Undesirable effects include harms (including the possibility of unanticipated adverse effects), greater burden (e.g. having to go to the doctor) & costs (including opportunity costs).

Standards for evidence

Judging the quality of evidence requires considering the context

In the context of a systematic review

The quality of evidence reflects the extent to which we are confident that an estimate of effect is correct.

In the context of making recommendations

The quality of evidence reflects the extent to which our confidence in an estimate of the effect is adequate to support a particular recommendation.

What types of evidence should be used to address different types of questions?

Evidence of the effects of the interventions or actions that are considered in a recommendation are essential, but not sufficient

Other types of required evidence are largely context specific. Study designs to be included in reviews should be dictated by the

interventions/outcomes being considered. There is uncertainty regarding what study designs to include for

some specific types of questions, particularly for questions regarding population interventions, harmful effects and interventions where there is only limited human evidence.

Decisions about the range of study designs to include should be made explicitly.

Great caution should be taken to avoid confusing a lack of evidence with evidence of no effect, and to acknowledge uncertainty.

Expert opinion is not a type of study design and should not be used as evidence. The evidence (experience or observations) that is the basis of expert opinions should be identified and appraised in a systematic and transparent way.

Quality of evidence – GRADE approach

Quality of evidence

(summary score)

Study design Lower if * Higher if *

High (4) Randomized trial Study quality:-1 Serious limitations-2 Very serious limitations-1 Important inconsistency Directness:-1 Some uncertainty-2 Major uncertainty-1 Sparse or imprecise

data-1 High probability of reporting bias

Strong association:+1 Strong, no plausible confounders, consistent and direct evidence+2 Very strong, no major threats to validity and direct evidence+1 Evidence of a Dose response gradient

Moderate (3)

Low (2) Observational study

Very low (1)

Developing Recommendations

Recommendations are judgements- guided by

Quality of evidence Trade off between benefits and harms Costs Values and preferences Feasibility of implementation

Developing recommendations

All recommendations should be classified into:

Strong

Weak

Conditional (research , time or group)

Strength of recommendations Strong Weak ConditionalImplications: the desirable effects of adherence to the recommendation outweigh the undesirable effects.

Implications the desirable effects of adherence to a recommendation probably outweigh the undesirable effects, but not confident that it is always indicated.

Implications: for specific groups or in specific situations the desirable effects of adherence to a recommendation outweigh the undesirable effects, or that the intervention should be used an only in the context of research

Patients: Most people would want the recommended course of action and only a small proportion would not

Patients: Most people would want the recommended course of action, but more than a handful would not

Patients:Specific recognized sub groups of people in the specific situation would benefit from the recommended course of action

Clinicians: Most patients should receive the recommended course of action

Clinicians: Patients will need assistance to make a decision as to if they want the recommended course of action that is consistent with their own values and context

Clinicians: Specific patient subsets or in specific situations the recommended course of action should be followed

Policy makers: The recommendation can be adapted as a policy in most situations

Policy makers: need for substantial debate & involvement of a range of stakeholders in adopting and or implementing the recommended course of action

Policy makers: Policy directives need to reflect that in specific situations this recommendation should be followed and outline those specific situations.

Recommendation to use an intervention only in the context of research - there is currently insufficient evidence to suggest using or not using an intervention and further research is likely to have a large potential for reducing uncertainty about the effects of the intervention, and for doing so at a reasonable cost.

Judgments about the strength of a recommendation –criteria to consider for WHO

Factors Comments

Quality of the evidence Higher the quality of the evidence the more likely a

strong recommendation can be made

Balance between desirable and undesirable effects

Larger the gap or gradient between these then more likely a strong recommendation will be made

Values and preferences If there is a great deal of variability or strong reasons that the recommended course of action is unlikely to be accepted then it is more likely a weak recommendation will be made.

Costs/financial implications (resource use) Higher the cost both financial and in terms of infrastructure, equipment or requirements, and more resource intensive requirements, then less likely to make a strong recommendation

Feasibility Where intervention is possible and practical in the settings where greatest impact is likely to be attained or is being sought, strong recommendation is more likely

Considering cost

Resource implications, including health system changes, for each recommendation in a WHO guideline should be explored. At the minimum, a qualitative description that can serve as a gross indicator of the amount of resources needed, relative to current practice, should be provided.

A scenario approach can be used, and will also need to include health system implications of the recommendations, from training, changes in supervision, monitoring and evaluation, advocacy, etc.

Ideally models should be made available and designed to allow for analysts to make changes in key parameters and reapply results in their own country.

Users of the guidelines need to work out the cost implications for their own service

Expected Functions of the guideline group

Review scope and questions for guideline

Advise & Identify outcomes critical for decision making

To advise on the interpretation of the evidence with explicit consideration of the overall balance of risks and benefits 

Provide end user input

Formulate recommendations taking into account diverse values and preferences.

Review drafts of guideline document

Review and approve final recommendations  

Tasks for this meeting

Review evidence summaries

Appraise risk/benefit, cost feasibility of recommendations

Agree on final recommendations