Critical Appraisal of Clinical Practice Guidelines

November 6, 2012Mary H. Palmer, PhD, RN,C

Helen W. & Thomas L. Umphlet Professor in Aging

University of North Carolina at Chapel Hill

Outline

• Quiz• Focused discussion of revised PICOT questions• Powerpoint review of critical appraisal of

clinical practice guidelines• Brief report of homework critical appraisals,

reporting relevance and noteworthiness• Small group activity– Critical appraisal and recommendations re. CPGs

Need for Clinical Practice Guidelines

• Increased volume of research-based evidence is a barrier to better use of knowledge.

• Lack of time to search, appraise and apply knowledge in clinical decision-making.

• Some erroneously equate lack of evidence for treatment effectiveness as the treatment being ineffective.

• Information should be reliable, relevant, and readable.

Source: Straus & Haynes, 2009 CMAJ 180(9), 942.

Developmental Approach to Clinical Practice Guideline

• Source: Morris et al. Orthopaedic Nursing, 2010 29(5):290-316.

CPGs as Tools

Definition: “systematically developed statements based on the best available evidence, including syntheses, make recommendations for specific clinical conditions across a broad array of clinical diagnoses and situations.” (Tricoci, et al, 2009).

Reduce unnecessary variations in clinical practice

designed with flexibility for individual patients who fall outside the scope of the guideline.

Clinical Practice Guidelines

• Outcomes meaningful to patients• Inclusive in considering all reasonable options• CPGs are a snapshot of evidence at given

point in time – need updating• Rating scheme– Quality and strength of studies– Level of evidence– Class or grade of recommendations

Accessing Guidelines Googling “practice guideline” will yield numerous

sources, including National Guideline Clearinghouse Centre for Health Evidence Canadian Medical Association Registered Nurses’ Association of Ontario (RNAO)

The term “practice guideline” can be used as a limit to define a publication type in PubMed

Guidelines International Network (G-I-N) has the world’s largest guideline library

Accessing Guidelines Googling “practice guideline” will yield numerous

sources, including National Guideline Clearinghouse Centre for Health Evidence Canadian Medical Association Registered Nurses’ Association of Ontario (RNAO)

The term “practice guideline” can be used as a limit to define a publication type in PubMed

Guidelines International Network (G-I-N) has the world’s largest guideline library

National Guideline Clearinghouse (NGC)

Developed in partnership with the American Medical Association and the American Association of Health Plans

Of all the guidelines sources, the NGC contains the most descriptive information about guidelines

National Library of Medicine (NLM) Gateway

Allows users to put in a search term that is then sent out to eight different NLM databases

Health Services/Health Technology Assessment Text (HSTAT)

Takes large guidelines, systematic reviews, and technology assessments and enables their texts to be searchable on the Internet

Question

Which of the following online databases is the most likely source of a CPG to guide a nurse’s care?

a. CINAHL

b. EBSCO

c. PubMed

d. PsycInfo

Answer

c. PubMed Rationale: PubMed can be searched for CPGs by setting

publication limits on the query to “practice guideline.”

Finding the Right Guideline Locating and reviewing current guidelines on a particular

subject is often overwhelming CPG should specify information such as:

Who developed and funded it Who was on the panel How the guideline was developed What dates the literature review covered

Finding the Right Guideline (cont’d)

Need to keep in mind that “one size does not fit all” Assess their application to the right person at the right

time and in the right way Ask

What are the guideline recommendations? Are the guideline recommendations valid? How useful are the recommendations?

Reading Guidelines Recommendations should be as unambiguous as possible Consider the developers’ values Should specify the process used to systematically search

and review the evidence that underlies the guideline Evidence should be graded using a recognized format

Recommendations themselves should be graded Updates are important Consider whether a particular guideline will help your

patients

Evaluating Guidelines Consider

Validity Reliability and reproducibility Clinical applicability Clinical flexibility Clarity Documentation Development by a multidisciplinary process Plans for review

Evaluating Guidelines (cont’d)

Rapid critical appraisal (RCA) checklist – see Box 8.4 AGREE instrument NGC summaries

Developing Guidelines There is a need for a CPG that addresses a topic when:

The topic is clinically important The topic is complex and requires clarity There is evidence of a gap between actual and

optimal care There are no existing valid or relevant guidelines

available There is evidence available to support guideline

development The topic is central to healthy public policy

Developing Guidelines (cont’d)

Processes and panels Both must be identified

Review questions must be created Conducting the literature search and review Drafting recommendations Peer review and dissemination

Implementing CPGs

Requires multifaceted and sustained interventions Practitioners’ commitment and organizational leadership

are keys Use of best practice champions EBP mentors Guideline implementation must be sustained over time Context must be considered

Question

Which of the following factors has the greatest bearing on the success or failure of CPG implementation?

a. The strength of the evidence that underlies the guideline

b. The validity of the process that was used to develop the guideline

c. The education level of the nurses who will implement the guideline

d. The commitment of the caregivers who will put the guideline into practice

Answer

d. The commitment of the caregivers who will put the guideline into practice

Rationale: While CPGs must be based on strong evidence and must be developed in a valid manner, these factors do not determine the success of their clinical implementation. Successful implementation is dependent on the commitment of practitioners to the process. Education level is not a key determinant of success.

Use of innovative technology to promote EBP

• Clinical decision support system– Access data, interventions and evidence-based

recommendations focused on changing the system and/or patient policies

– Integrate research– Evaluate interventions– Provide feedback to providersSource: Williamson et al. (2011). Worldviews on Evidence-

based Nursing, fourth quarter

Discussion of Homework Critical Appraisal

As you report on each article you critically appraised for your homework assignment, rate each article for Relevance and Noteworthiness as part of your evidence summary

McMaster Online Rating of Evidence (MORE)Source: Straus & Haynes (2009), CMAJ:180(9)

Relevance of individual studies7- directly and highly relevant6 - definitely relevant5 - probably relevant4 -possibly relevant; likely of indirect of

peripheral relevance at best3- possibly not relevant2 - probably not relevant; content only remotely related1 - definitely not relevant; completely unrelated content area

McMaster Online Rating of Evidence (MORE)Source: Straus & Haynes (2009), CMAJ:180(9)

Newsworthiness of individual studies7 – useful information; most practitioners in my specialty definitely don’t know this6 – useful information; most practitioners in my specialty probably don’t know this5 – useful information; most practitioners in my specialty possibly don’t know this4 – useful information; most practitioners in my specialty possibly already know this3- useful information; most practitioners in my specialty probably already know this2 – it probably doesn’t matter whether they know this or not1 – not of direct clinical interest

Critical Appraisal

• Rapid critical appraisal checklist• AGREE instrument– Scope and purpose– Stakeholder involvement (bias)– Rigor of development– Clarity and presentation– Application– Editorial independence

Class Exercise

• Randomly assign to two groups– Group 1: Schuster et al paper– Group 2: Moyer et al paper

Use rapid critical appraisal checklist and then use AGREE instrumentReport your findings and respond:

Were there any differences in your findings and conclusion about the CPG?