case 1 - ucsf cme grade definition grade definition suggestion for practicesuggestion for practice...

1

Agency for Healthcare Research and QualityAgency for Healthcare Research and QualityAdvancing Excellence in Health CareAdvancing Excellence in Health Care • • www.ahrq.govwww.ahrq.gov

Preventive Medicine 2009: Understanding the US Preventive Services Task Force Guidelines

George F. Sawaya, MDAssociate Professor

Department of Obstetrics, Gynecology and Reproducti ve SciencesDepartment of Epidemiology and Biostatistics

University of California, San FranciscoDirector, Cervical Dysplasia Clinic, San Francisco General Hospital

Some slides courtesy of AHRQ/USPSTF

Advancing Advancing Excellence in Excellence in Health CareHealth Care Case 1Case 1

�� Your 74Your 74--yearyear--old father is found to have a slightly elevated old father is found to have a slightly elevated serum PSA drawn as part of a routine exam. He asks you serum PSA drawn as part of a routine exam. He asks you what this means. He says his doctor recommends that since what this means. He says his doctor recommends that since he smokes, he be screened for an abdominal aortic he smokes, he be screened for an abdominal aortic aneurysm, COPD and lung cancer.aneurysm, COPD and lung cancer.

He was also told that he has a carotid bruit.He was also told that he has a carotid bruit.

Advancing Advancing Excellence in Excellence in Health CareHealth Care Who is the USPSTF?Who is the USPSTF?

■■ Experts in primary care, prevention, research methodsExperts in primary care, prevention, research methods

�� Government supported by AHRQ but independentGovernment supported by AHRQ but independent

�� Family medicine, internal medicine, pediatrics, Family medicine, internal medicine, pediatrics, obstetrics/gynecology, nursing, behavioral healthobstetrics/gynecology, nursing, behavioral health

�� Scientific support from an EvidenceScientific support from an Evidence--Based Practice Based Practice Center (EPC)Center (EPC)

�� NonNon--member liaisons from primary care clinician member liaisons from primary care clinician associations, Federal agenciesassociations, Federal agencies

Advancing Advancing Excellence in Excellence in Health CareHealth Care

Current MembersCurrent Membersof the USPSTF of the USPSTF

�� Thomas G. DeWitt, MDThomas G. DeWitt, MD�� Allen Dietrich, MD, MPHAllen Dietrich, MD, MPH�� Kimberly D. Gregory, MD, Kimberly D. Gregory, MD,

MPHMPH�� David Grossman, MD, MPHDavid Grossman, MD, MPH�� George Isham, MD, M.P.H.George Isham, MD, M.P.H.�� Michael LeFevre, MD, MSPHMichael LeFevre, MD, MSPH�� Rosanne Leipzig, MD, PhDRosanne Leipzig, MD, PhD

�� Bernadette Melnyk, PhD, RN, Bernadette Melnyk, PhD, RN, CPNP/NPPCPNP/NPP

�� Lucy N. Marion, PhD, RNLucy N. Marion, PhD, RN�� Virginia A. Moyer, MD, MPH Virginia A. Moyer, MD, MPH �� *Judith K. Ockene, PhD, *Judith K. Ockene, PhD,

MSEdMSEd�� *George F. Sawaya, MD*George F. Sawaya, MD�� J. Sanford Schwartz, MD, ABJ. Sanford Schwartz, MD, AB�� Timothy Wilt, MD, MPHTimothy Wilt, MD, MPH

Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair)Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair)Diana B. Petitti, M.D., M.P.H. (Vice Chair)Diana B. Petitti, M.D., M.P.H. (Vice Chair)

2

Advancing Advancing Excellence in Excellence in Health CareHealth Care Task Force ActivitiesTask Force Activities

�� Provide Provide evidenceevidence--basedbased scientific reviews of preventive scientific reviews of preventive health services for use in health services for use in primary healthcare delivery primary healthcare delivery settingssettings for patients without recognized signs or for patients without recognized signs or symptoms of target conditionsymptoms of target condition

�� AgeAge-- and riskand risk--factor specific recommendations for factor specific recommendations for routineroutine practicepractice

�� Recommendations include:Recommendations include:

–– Screening testsScreening tests

–– CounselingCounseling

–– Preventive medicationsPreventive medications

Advancing Advancing Excellence in Excellence in Health CareHealth Care Why EvidenceWhy Evidence--Based?Based?

�� Need transparent systematic process to obtain Need transparent systematic process to obtain and distill best available (or best feasible) and distill best available (or best feasible) evidence to support decision making evidence to support decision making

–– Identifying, evaluating and summarizing scientific Identifying, evaluating and summarizing scientific evidence about outcomes or interventions or policiesevidence about outcomes or interventions or policies

–– Translating evidence into practice recommendations Translating evidence into practice recommendations


USPSTF Grades of RecommendationsUSPSTF Grades of Recommendations

Certainty of Net Certainty of Net BenefitBenefit

Magnitude of Net BenefitMagnitude of Net Benefit

SubstantialSubstantial ModerateModerate SmallSmall Zero/negativeZero/negative

HighHigh AA BB CC DD

ModerateModerate BB BB CC DD

LowLow InsufficientInsufficient


Communicating USPSTF RecommendationsCommunicating USPSTF Recommendations

GradeGrade Grade DefinitionGrade Definition Suggestion for PracticeSuggestion for Practice

AA The USPSTF recommends the service. There is high certainty The USPSTF recommends the service. There is high ce rtainty that the net benefit is substantial.that the net benefit is substantial.

Offer or provide this service.Offer or provide this service.

BB The USPSTF recommends the service. There is high certainty The USPSTF recommends the service. There is high ce rtainty that the net benefit is moderate or there is modera te certainty that the net benefit is moderate or there is modera te certainty that the net benefit is moderate to substantial. that the net benefit is moderate to substantial.

Offer or provide this service. Offer or provide this service.

CC The USPSTF recommends against routinely providing the The USPSTF recommends against routinely providing t he service. There may be considerations that support p roviding service. There may be considerations that support p roviding the service in an individual patient. There is mode rate or high the service in an individual patient. There is mode rate or high certainty that the net benefit is small.certainty that the net benefit is small.

Offer or provide this service only if there Offer or provide this service only if there are other considerations in support of are other considerations in support of the offering or providing the service in the offering or providing the service in an individual patient.an individual patient.

DD The USPSTF recommends against the service. There is The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits.or that the harms outweigh the benefits.

Discourage the use of this service. Discourage the use of this service.

II The USPSTF concludes that the current evidence is The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or c onflicting, service. Evidence is lacking, of poor quality, or c onflicting, and the balance of benefits and harms cannot be det ermined.and the balance of benefits and harms cannot be det ermined.

Read “Clinical Considerations” section Read “Clinical Considerations” section of USPSTF Recommendation Statement. of USPSTF Recommendation Statement. If offered the service, patients should If offered the service, patients should understand the uncertainty about the understand the uncertainty about the balance of benefits and harms.balance of benefits and harms.

3


�� Your 40Your 40--yearyear--old sister has received an eold sister has received an e--mail encouraging mail encouraging her to get a CAher to get a CA--125 for ovarian cancer screening.125 for ovarian cancer screening.


Steps in the Process Steps in the Process for Development of Recommendationsfor Development of Recommendations

1.1. Define questions and outcomes of interestDefine questions and outcomes of interest2.2. Define and retrieve relevant evidenceDefine and retrieve relevant evidence

3.3. Evaluate QUALITY of individual studiesEvaluate QUALITY of individual studies4.4. Synthesize and judge STRENGTH of available Synthesize and judge STRENGTH of available

evidenceevidence5.5. Determine balance of benefits and harmsDetermine balance of benefits and harms

6.6. Link recommendation to judgment about net Link recommendation to judgment about net benefitsbenefits


Step 1: Analytic Framework on Step 1: Analytic Framework on Screening for a DiseaseScreening for a Disease

8 Key Questions (KQ)



1.1. Define questions and outcomes of interestDefine questions and outcomes of interest2.2. Define and retrieve relevant evidenceDefine and retrieve relevant evidence3.3. Evaluate QUALITY of individual studiesEvaluate QUALITY of individual studies4.4. Synthesize and judge STRENGTH of available Synthesize and judge STRENGTH of available



4


Define and Retrieve Relevant EvidenceDefine and Retrieve Relevant Evidence

�� For each KQ developed from AF: For each KQ developed from AF: –– Create inclusion/exclusion criteria based on the Create inclusion/exclusion criteria based on the

key questions defined from the analytic key questions defined from the analytic frameworkframework

–– PubMed searchPubMed search

–– Cochrane searchCochrane search–– Other database search (CINAHL, etc)Other database search (CINAHL, etc)

–– References from key articles, editorials, review References from key articles, editorials, review articlesarticles

–– Expert consultation (others, TF members)Expert consultation (others, TF members)


Step 1: Analytic Framework on Step 1: Analytic Framework on Screening for a DiseaseScreening for a Disease

8 Key Questions (KQ)


Steps in the ProcessSteps in the Processfor Development of Recommendationsfor Development of Recommendations


3.3. Evaluate quality of individual studiesEvaluate quality of individual studies4.4. Synthesize and judge STRENGTH of available Synthesize and judge STRENGTH of available



Advancing Advancing Excellence in Excellence in Health CareHealth Care Evaluate Quality of Individual StudiesEvaluate Quality of Individual Studies� Good::

–– Uses a credible reference standardUses a credible reference standard–– Reliability of test assessedReliability of test assessed–– Includes large number of subjectsIncludes large number of subjects

� Fair: –– Uses reasonable although not best standardUses reasonable although not best standard–– Interprets reference std independent of screening t estInterprets reference std independent of screening t est–– Moderate sample size Moderate sample size

�� Poor:Poor: Has fatal flaw such as: Has fatal flaw such as: –– Uses inappropriate reference standardsUses inappropriate reference standards–– Biased ascertainment of reference standardBiased ascertainment of reference standard

–– Very small sample size or very selected patients.Very small sample size or very selected patients.

5


Steps in the ProcessSteps in the Processfor Development of Recommendationsfor Development of Recommendations


3.3. Evaluate quality of individual studiesEvaluate quality of individual studies4.4. Synthesize and judge STRENGTH of overall Synthesize and judge STRENGTH of overall



Advancing Advancing Excellence in Excellence in Health CareHealth Care Critical Appraisal QuestionsCritical Appraisal Questions�� Do the studies have the appropriate research design to answer the Do the studies have the appropriate research design to answer the

key questions?key questions?

�� To what extent are the existing studies high quality? To what extent are the existing studies high quality?

�� To what extent are the results of the studies generalizable (or To what extent are the results of the studies generalizable (or “applicable”) to the general US primary care population and “applicable”) to the general US primary care population and situation? situation?

�� How many studies have been conducted that address the key How many studies have been conducted that address the key questions? How large are the studies? questions? How large are the studies?

�� How consistent/coherent are the results of the studies?How consistent/coherent are the results of the studies?

�� Are there additional factors that assist us in drawing conclusions Are there additional factors that assist us in drawing conclusions about the certainty of the evidence? (e.g., presence or absence of about the certainty of the evidence? (e.g., presence or absence of dosedose--response effects; fit within a biologic model)response effects; fit within a biologic model)


Step 4: Synthesize and Step 4: Synthesize and Judge Strength of Overall EvidenceJudge Strength of Overall Evidence

�� Evidence reportsEvidence reports–– Evidence tables summarizing studiesEvidence tables summarizing studies

–– Narrative discussing overall strength of evidenceNarrative discussing overall strength of evidence

�� MetaMeta--analysisanalysis

�� ModelingModeling

�� Systematic reviews from others Systematic reviews from others ––

Cochrane, etc. Cochrane, etc.


Step 4: Synthesize & Judge Step 4: Synthesize & Judge Strength of Key Question EvidenceStrength of Key Question Evidence

ConvincingConvincing :: WellWell--designed, welldesigned, well--conducted studies in conducted studies in representative populations that directly assess effects on health representative populations that directly assess effects on health outcomesoutcomes

Adequate:Adequate: Evidence sufficient to determine effects on health Evidence sufficient to determine effects on health outcomes, but limited by number, quality, or consistency of outcomes, but limited by number, quality, or consistency of studies, generalizability to routine practice, or indirect nature of studies, generalizability to routine practice, or indirect nature of the evidence. the evidence.

Inadequate Inadequate :: Insufficient evidence to determine effect on health Insufficient evidence to determine effect on health outcomes due to limited number or power of studies, important outcomes due to limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, flaws in their design or conduct, gaps in the chain of evidence, or lack of information on important health outcomesor lack of information on important health outcomes

6


�� Your 28Your 28--yearyear--old patient saw a television commercial old patient saw a television commercial encouraging her to get an HPV test. She wants one. Now.encouraging her to get an HPV test. She wants one. Now.


Steps in USPSTF ProcessSteps in USPSTF Processfor Development of Recommendationsfor Development of Recommendations


3.3. Evaluate quality of individual studiesEvaluate quality of individual studies4.4. Synthesize and judge STRENGTH of overall evidenceSynthesize and judge STRENGTH of overall evidence

5.5. Determine balance of benefits and harmsDetermine balance of benefits and harms6.6. Link recommendation to net benefitsLink recommendation to net benefits


Step 5: Determine BalanceStep 5: Determine Balanceof Benefits and Harmsof Benefits and Harms

Estimate Magnitude of Net BenefitEstimate Magnitude of Net Benefit

Benefits Benefits –– Harms = Net BenefitHarms = Net Benefit


Step 5: Determine Balance of Step 5: Determine Balance of Benefits and Harms: Benefits and Harms: Assessing HarmsAssessing Harms

�� Potential harms real but hard to quantifyPotential harms real but hard to quantify�� Include psychological and physical consequences Include psychological and physical consequences

of falseof false--positives, falsepositives, false--negatives, “labeling”, over negatives, “labeling”, over treatmenttreatment�� Opportunity costsOpportunity costs

�� Magnitude and duration of harm subjective, hard to Magnitude and duration of harm subjective, hard to compare to benefits compare to benefits –– May translate into QALYs to compareMay translate into QALYs to compare–– NNHNNH

7


Step 5: Determine Balance of Benefits & Harms: Step 5: Determine Balance of Benefits & Harms: Assessing Magnitude of Net BenefitAssessing Magnitude of Net Benefit

�� No explicit criteria for magnitude (substantial, No explicit criteria for magnitude (substantial, moderate, small, zero/negative) moderate, small, zero/negative)

�� SubstantialSubstantial benefitbenefit : impact on high burden : impact on high burden orormajor effect on uncommon outcomemajor effect on uncommon outcome

�� Problems: requires evidence on harms and Problems: requires evidence on harms and common metric for benefit and harmscommon metric for benefit and harms

Advancing Advancing Excellence in Excellence in Health CareHealth Care CertaintyCertainty

�� Definition: Definition: The U.S. Preventive Services Task Force The U.S. Preventive Services Task Force defines certainty as defines certainty as ““ likelihood that the USPSTF likelihood that the USPSTF assessment of the net benefit of a preventive service assessment of the net benefit of a preventive service is correctis correct”.”. The net benefit is defined as benefit minus The net benefit is defined as benefit minus harm of the preventive service as implemented in a harm of the preventive service as implemented in a general, primary care population. The USPSTF general, primary care population. The USPSTF assigns a certainty level based on the nature of the assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of overall evidence available to assess the net benefit of a preventive service.a preventive service.

Sawaya GF, Guirguis-Blake J, LeFevre M. Barton M, Harris R, Petitti D, US Preventive Services Task Force. Update on the methods of the US Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med. 2007;147:871-875.

Advancing Advancing Excellence in Excellence in Health CareHealth Care Levels of CertaintyLevels of Certainty�� High:High: The available evidence usually includes consistent results from wellThe available evidence usually includes consistent results from well--

designed, welldesigned, well--conducted studies in representative primary care populations. conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health This These studies assess the effects of the preventive service on health This outcomes. conclusion is therefore unlikely to be strongly affected by the results of outcomes. conclusion is therefore unlikely to be strongly affected by the results of future studies.future studies.

�� Moderate: Moderate: The available evidence is sufficient to determine the effects of the The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is preventive service on health outcomes, but confidence in the estimate is constrained by factors such as: the number, size, or quality of individual studies; constrained by factors such as: the number, size, or quality of individual studies; inconsistency of findings across individual studies; limited generalizability of inconsistency of findings across individual studies; limited generalizability of findings to routine primary care practice; or lack of coherence in the chain of findings to routine primary care practice; or lack of coherence in the chain of evidence. As more information becomes available, the magnitude or direction of evidence. As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the observed effect could change, and this change may be large enough to alter the conclusion.the conclusion.

�� Low:Low: The available evidence is insufficient to assess effects on health outcomes. The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient because of: the limited number or size of studies; Important Evidence is insufficient because of: the limited number or size of studies; Important flaws in study design or methods; inconsistency of findings across individual flaws in study design or methods; inconsistency of findings across individual studies gaps in the chain of evidence; findings not generalizable to routine primary studies gaps in the chain of evidence; findings not generalizable to routine primary care practice; or a lack of information on important health outcomes. More care practice; or a lack of information on important health outcomes. More information may allow an estimation of effects on health outcomes. information may allow an estimation of effects on health outcomes.


�� Your 40 year old male patient is considering aspirin therapy Your 40 year old male patient is considering aspirin therapy for the prevention of cardiovascular events. He is otherwise for the prevention of cardiovascular events. He is otherwise healthy. Yes? No? Maybe?healthy. Yes? No? Maybe?

8



1.1. Define questions and outcomes of interestDefine questions and outcomes of interest

2.2. Define and retrieve relevant evidenceDefine and retrieve relevant evidence

3.3. Evaluate quality of individual studiesEvaluate quality of individual studies

4.4. Synthesize and judge STRENGTH of overall evidenceSynthesize and judge STRENGTH of overall evidence

5.5. Determine balance of benefits and harmsDetermine balance of benefits and harms

6.6. Link recommendation to net benefitsLink recommendation to net benefits


Step 6: Step 6: Link recommendation to net benefits:Link recommendation to net benefits:USPSTF Grades of RecommendationsUSPSTF Grades of Recommendations

Certainty of Net Certainty of Net BenefitBenefit

Magnitude of Net BenefitMagnitude of Net Benefit

SubstantialSubstantial ModerateModerate SmallSmall Zero/negativeZero/negative

HighHigh AA BB CC DD

ModerateModerate BB BB CC DD

LowLow InsufficientInsufficient


Step 6: Step 6: Link recommendation to net benefits: Link recommendation to net benefits: Insufficient EvidenceInsufficient Evidence

�� Lack of evidence on harms or benefitsLack of evidence on harms or benefits

�� Poor quality of existing studiesPoor quality of existing studies

�� Good quality studies with conflicting resultsGood quality studies with conflicting results


Electronic Resources for Electronic Resources for CliniciansClinicians

�� ePSS ePSS –– electronic Preventive Services Selector electronic Preventive Services Selector Tool Tool

–– Search USPSTF recommendations by age, sex Search USPSTF recommendations by age, sex and risk factors and risk factors

–– Available as a webAvailable as a web--based tool or can be based tool or can be downloaded to your PDAdownloaded to your PDA

–– www.epss.ahrq.govwww.epss.ahrq.gov

9



Advancing Advancing Excellence in Excellence in Health CareHealth Care Cases: A, B, C, D or I?Cases: A, B, C, D or I?�� Your 74Your 74--yearyear--old father is found to have a slightly elevated old father is found to have a slightly elevated

serum PSA drawn as part of a routine examserum PSA drawn as part of a routine exam�� II�� He says his doctor also recommends he be screened for an He says his doctor also recommends he be screened for an

abdominal aortic aneurysmabdominal aortic aneurysm�� BB�� lung cancerlung cancer�� II�� COPDCOPD�� DD�� carotid artery stenosiscarotid artery stenosis�� DD


Analytic Framework Analytic Framework --Prostate Cancer ScreeningProstate Cancer Screening

Early ProstateCancer

Reduced prostate cancer morbidity,

mortality

AsymptomaticMen

Screen:PSA, DRE

Treatradiation,

prostatectomy

3

Adverse effectsof screening:false positive, false negative, inconvenience,labeling

Adverse effects of Rx:Impotence, incontinence,death, overtreatment

1

2

45

10

Advancing Advancing Excellence in Excellence in Health CareHealth Care Cases: A, B, C, D or I?Cases: A, B, C, D or I?�� Your 40Your 40--yearyear--old sister has received an eold sister has received an e--mail encouraging mail encouraging

her to get a CAher to get a CA--125 for ovarian cancer screening.125 for ovarian cancer screening.�� DD�� Your 28Your 28--yearyear--old patient saw a television commercial old patient saw a television commercial

encouraging her to get an HPV test. encouraging her to get an HPV test. �� II�� Your 40 year old male patient is considering aspirin therapy Your 40 year old male patient is considering aspirin therapy

for the prevention of cardiovascular events.for the prevention of cardiovascular events.�� AA


Resources for Resources for Clinicians and Clinicians and ConsumersConsumers

�� Based on Based on recommendations of the recommendations of the USPSTF. USPSTF.

�� AtAt--aa--glance glance wall chart wall chart for for appropriate preventive appropriate preventive services based on age, services based on age, sex, and risk status. sex, and risk status.

�� To be used in prompting To be used in prompting shared decisionshared decision--making making between consumers and between consumers and their primary care their primary care clinician.clinician.


Resources for Resources for Consumers Consumers

[email protected]@ahrq.hhs.gov


11


Advancing Advancing Excellence in Excellence in Health CareHealth Care SummarySummary

• Preventive interventions require a high burden of proof: the “do no harm” principle.• The Task Force takes this seriously.• Guidelines are designed to maximize population screening benefits and minimize population screening harms• Guidelines involve a lot of evidence, but a certain degree of judgment.

Advancing Advancing Excellence in Excellence in Health CareHealth Care Task Force MantraTask Force Mantra

“An ounce of prevention…“An ounce of prevention…

… is a whole lot of work.”… is a whole lot of work.”

-------- Paul Frame, MD Paul Frame, MD (Task Force Member Emeritus)(Task Force Member Emeritus)

Advancing Advancing Excellence in Excellence in Health CareHealth Care Questions?Questions?

case 1 - ucsf cme grade definition grade definition suggestion for practicesuggestion for practice...

Documents