pharmacoepidemiology and drug safety 2011; 20: 807–809Published online 17 June 2011 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.2173

COMMENTARY

Starting the conversation

William Lawrence*

Center for Outcomes and Evidence, Agency for Healthcare Research and Quality, 540 Gaither Rd., Rockville, MD 20850, USA

Comparative effectiveness research (CER) has drawnincreasing attention since the passing of the MedicarePrescription Drug, Improvement, and ModernizationAct of 2003. In section 1013 of this Act, the Agencyfor Healthcare Research and Quality (AHRQ) wasauthorized to conduct and support research with afocus on outcomes, comparative clinical effectiveness,and appropriateness of pharmaceuticals, devices, andhealth care services.1 The interest in this research hasskyrocketed with the $1.1 billion Federal investmentin CER with the 2009 American Recovery andReinvestment Act (ARRA). In 2003, MEDLINEreports 20 articles using the term “comparativeeffectiveness”; twice as many articles published usingthis term in the first month of 2011 alone. But, what isCER, and is this research really something new? Anumber of organizations have defined CER, but twoare worth particular mention. The Federal Coordinat-ing Council for Comparative Effectiveness Research,established under ARRA, defines comparative effec-tiveness as follows:

Comparative effectiveness research is the conductand synthesis of research comparing the benefitsand harms of different interventions and strategiesto prevent, diagnose, treat and monitor healthconditions in “real world” settings. The purpose ofthis research is to improve health outcomes bydeveloping and disseminating evidence‐based in-formation to patients, clinicians, and other deci-sion‐makers, responding to their expressed needs,about which interventions are most effective forwhich patients under specific circumstances.2

The Institute of Medicine (IOM) has a similardefinition3 and defines six characteristics of CER. Thisresearch:

*Correspondence to: W. Lawrence, Center for Outcomes and Evidence, Agencyfor Healthcare Research and Quality, 540 Gaither Rd., Rockville, MD 20850,USA. E‐mail: William.lawrence@ahrq.hhs.gov

Published 2011. This article is a US Government workand is in the public domain in the USA.

• Has the objective to inform a particular clinicaldecision, for either individuals from a patient perspec-tive or for populations from a policy perspective;

• Compares at least two alternatives, each of whichcould potentially be “best practice”;

• Describes results at the population and the subgrouplevel to move beyond average effect towardsassisting with individualizing decisions;

• Measures outcomes, including both benefits andharms, that are important to patients;

• Employs methods and data sources that areappropriate to the decision, including potentiallyexperimental studies, non‐experimental designs, andresearch syntheses; and

• Is conducted in settings similar to those in which theintervention would be used in “real‐world” practice.

Both the Federal Coordinating Council and the IOMhave emphasized training and the development of askilled research workforce as part of CER. AHRQ, aspart of the ARRA awards, has invested in bothinstitutional and individual awards for training andcareer development in CER.The implications of these approaches to CER are

important and reflect the multifaceted nature of thisresearch. First, the focus on informing decisionsimplies that researchers need the input of thedecision‐makers, to understand the appropriate ques-tions and the factors CER could address to improvedecision‐making.3 The focus on decision‐making alsohighlights the need for expertise in the science oftranslation and dissemination of research findings. IfCER is to inform decision‐making at the patient orpolicy level, then peer‐reviewed publication of CERfindings is insufficient to ensure its use. Not only asound evidence base, but research and tools totranslate evidence into actionable information forinformed and shared decision‐making are essential todiminishing the chances of making decisions in theface of “avoidable ignorance”.4

w. lawrence808

Second, wide varieties of clinical and methodologicexpertise are represented in the context of this research.The field involves clinical specialties such as medicine,pharmacy, and nursing, and research areas such asepidemiology, biostatistics, economics, informatics,survey research, and health services research. Method-ologic approaches aimed at improving internal andexternal validity of observational studies include rapidlyexpanding advances in both study design and analyticstrategies.5 While emphasis has been given to observa-tional studies and analysis of administrative databases,traditional clinical trials and pragmatic clinical trials arealso valid tools for CER.6 The synthesis of currentlyavailable research requires expertise in systematicreview and critical appraisal of research studies.7 Thisdiversity of expertise and approach optimally shouldbe represented when thinking about training futureresearchers in comparative effectiveness.The demand for researchers conversant with CER

and the increasing interest in new methods insystematic review, observational research, and inter-ventional research emphasize the need for training ofnew researchers. But what are the most criticalknowledge areas that we need to emphasize in trainingfor these researchers? And, equally importantly, whatare the training needs of future audiences of thisresearch, such as policy makers?The paper by Dr. Murray8 takes an impressive first

step in addressing these needs for training researchersand policy makers in the fundamentals of CER forpharmaceuticals. The report describes the convening ofa multidisciplinary panel comprising academic re-searchers, pharmaceutical industry scientists, healthprofession educators, and an AHRQ program official todiscuss a core curriculum in CER for researchers anddecision makers. The results of the panel deliberationsappear to have face validity, and were consistent with apharmacoepidemiology‐based approach to CER. Topknowledge areas for researchers in comparativeeffectiveness included understanding observationalresearch methods, data resources, interpreting researchresults, and understanding the US health care system.Top skills emphasized critical appraisal of CER studies,interpreting results, and developing analytic plans. Thisemphasis on methodologic rigor is appropriate forthose entering a career in research, and an emphasis onunderstanding the health care system and dataresources is critical for those researching outcomes inreal‐world settings. This work serves as an inventory ofa basic knowledge base and skill sets in the field.What are the lessons learned from this curriculum

development exercise? First, CER embraces a widerange of research methodology. The author reports

Published 2011. This article is a US Government workand is in the public domain in the USA.

considerable panel debate about whether or not CER issomething “new”; from the standpoint of this exercise,the question is moot. More relevant is that CER,whether or not a new entity, builds upon the work fromestablished areas, and that these should be representedin a CER curriculum. Database studies, prospective andretrospective observational studies, traditional clinicaltrials and newer approaches such as cluster randomizedtrials and adaptive trials, are all valid mechanisms withinthe field of CER, as are the requisite methodologicwork from fields such as the statistical, epidemiologic,economic, behavioral, and survey research fields. Inaddition, the panel recognized the importance of ex-pertise in systematic review and the methodologies thatsupport research synthesis; this synthesis can allow us tosystematically elucidate evidence from prior research,and to use these data to improve identification of prioritygaps for new research. Touched upon in the paneldeliberations, but perhaps deserving of more futurework, is the area of translation and disseminationscience. If CER is to inform decisions, key findingsfrom the research need translated into decision makingcontexts aimed at specific audiences faced with specificdecisions; products such as informational guides,decision aids, and clinical decision support can helpplace these findings into relevant context and ensureeasy access to the information at the time it is needed.Translation requires representation from fields ofbehavioral research, behavioral economics, communi-cations and decision sciences, and informatics.Not only are multiple methodologic areas involved in

CER, but multiple clinical areas of expertise may beinvolved too. The expertise of the panel participants andthe focus of the panel discussions centers on pharma-ceuticals, which is appropriate for the topic and for thisjournal. However, CER should also be consideredwithin the context of helping to inform decision makers,whether patients, clinicians, or policy makers, about thebenefits or harms of alternative choices in the decisionthey are making. Sometimes the alternatives are broaderthan a choice between pharmaceuticals; for example,Cooperberg and colleagues9 compared commonly usedtherapies for localized prostate cancer, including pri-mary androgen deprivation therapy (pharmaceuticals),external beam radiation therapy (devices), and radicalprostatectomy (surgical procedures). Indeed, alterna-tives for some issues may include choices betweentherapeutics, further diagnostic evaluation, and/or ex-pectant management. How does this issue impact on thedevelopment of a curriculum? In order to be preparedfor these challenges, future work could be aimedtowards having input from multiple disciplines for thecurriculum development. Of particular importance is to

Pharmacoepidemiology and Drug Safety, 2011; 20: 807–809DOI: 10.1002/pds

starting the conversation 809

have the input of the potential users of CER, thedecision‐makers, at the table.Dr. Murray’s work represents an important start in

helping to define the future training needs of newgenerations of researchers interested in CER forpharmaceuticals. Interest has been growing in CERover time, and the investment in CER through ARRAhas created huge interest both from researchers andfrom those who are interested in using this research toimprove care. For a sustained effort in CER we need todevelop the field further to ensure the highest qualityresearch that is aimed at helping to inform real‐worlddecisions in health care. One of the critical componentsof this field is the training of new researchers; Dr.Murray’s work can serve as a model for assessing theseresearch needs. This model can, and should, be sharedacross fields of clinical and methodologic specializa-tion. Given the multidisciplinary nature of CER, thedialog between the disciplines is crucial; the currentwork is a critical start to that ongoing conversation.

CONFLICT OF INTEREST

The author declares no conflict of interest.

Published 2011. This article is a US Government workand is in the public domain in the USA.

DISCLAIMER

The views expressed in this article are those ofthe author, and no official endorsement by the Agencyfor Healthcare Research and Quality or the USDepartment of Health and Human Services is intendedor should be inferred.

REFERENCES

1. Slutsky JR, Clancy CM. AHRQ’s Effective Health Care Program: whycomparative effectiveness matters. AJMQ 2009; 24: 67–70.

2. Federal Coordinating Council for Comparative Effectiveness Research. Report tothe President and the Congress. US Department of Health and Human Services:Washington, DC, June, 2009.

3. IOM (Institute of Medicine). Initial National Priorities for ComparativeEffectiveness Research. The National Academies Press: Washington, DC, 2009.

4. Moulton B, King J. Aligning ethics with medical decision‐making: The quest forinformed patient choice. J Law Med Ethics 2010; 38: 85–97.

5. Lohr KN. Comparative effectiveness research methods: symposium overview andsummary. Med Care 2010; 48(Suppl1): S3‐S6.

6. Sullivan P, Goldman D. The promise of comparative effectiveness research.JAMA 2011; 305: 400–401.

7. Slutsky J, Atkins D, Chang S, Collins Sharp BA. AHRQ Series Paper 1:Comparing medical interventions: AHRQ and the Effective Health‐Care Program.J Clin Epidemiol 2010; 63: 481–483.

8. Murray MD. Curricular considerations for pharmaceutical comparative effective-ness research. Pharmacoepidemiol Drug Saf 2011; 20(8): 797–804.

9. Cooperberg MR, Vickers AJ, Broering JM, et al. Comparative risk‐adjustedmortality outcomes after primary surgery, radiotherapy, or androgen‐deprivationtherapy for prostate cancer. Cancer 2010; 116: 5226–5234.

Pharmacoepidemiology and Drug Safety, 2011; 20: 807–809DOI: 10.1002/pds