Original article

Evidence-based decision making and asthma in the internet age:

the tools of the trade

We all want to make decisions that are based on the bestavailable knowledge. However, keeping up with existingand new knowledge is not an easy task. Each year, thebody of literature on asthma expands by thousands ofnew articles published in hundreds of journals and booksand, increasingly, on the internet. Because of the speedwith which the literature is growing and the limitedavailability of resources, most clinicians, researchers andpatients may never benefit from the information thatcould best guide their decisions.The need to handle information in efficient and

responsible ways is what motivates the supporters ofevidence-based decision making (EBDM), a process thatinvolves the explicit, conscientious and judicious consid-eration of the current best available evidence fromresearch to guide health-care decisions (1). There areother related terms, most of which are used interchange-ably. The first term used, evidence-based medicine,referred primarily to physicians and medicine (2). Otherterms include evidence-based health care, which refers tothe process in relation to the health care system at large(3); evidence-based practice, used by the US Agency forHealthcare Research and Quality (known formerly as theAgency for Health Care Policy and Research) to desig-nate a series of centres in North America, that areproducing evidence reports and technology assessmentsto support guideline development by other groups (4).More specific terms are emerging rapidly. These include

evidence-based management (5); evidence-based nursing(6); evidence-based mental health (7); and so on. Notsurprisingly, evidence-based child health care has beenalready introduced into the health lexicon (8).Regardless of the term or focus, the process of EBDM

includes the following steps (the description has beenmodified from another source (9)):• formulation of answerable questions;• systematic efforts to locate research evidence that couldbe used to answer the question(s);

• evaluation of the validity, relevance and applicability ofthe research evidence identified;

• integration of the evidence, if applicable, with othertypes of information, the values and preferences of thepeople involved in the decision, and the circumstancesin which the decision is being made; and

• evaluation of the outcomes of the decision and of thedecision makers’ own performance.

Potential benefits of EBDM in asthma

EBDM could enhance the appropriateness of asthma-related decisions and lead to better patient outcomes, byencouraging the use of interventions of establishedefficacy and indicating those with dubious effects orclear harm. It could also provide clinicians, patients,policy makers, educators, researchers, funders and other

At the dawn of the Information Age, the practice of evidence-based decisionmaking (EBDM) is still hindered by many important barriers related to thedecision makers, to the evidence per se or to the health system. Some of thesebarriers, particularly those related to the distillation, dissemination and pack-aging of research evidence, could be overcome by recent and ongoing develop-ments in portable/wearable computers, internet appliances, multimedia andwireless broadband internet traffic. This article describes specific EBDM-relatedtools, with emphasis on internet-enabled ‘‘how to’’ books; and tools to improvethe quality of reporting research, to formulate questions; to search for evidence;to access journals, systematic reviews and guidelines; to interact with organi-zations promoting EBDM; and to tailor evidence to individual cases. However,thinking that all barriers to the practice of EBDM could be solved by fancyinformation technology is naı̈ve. Barriers related to the generation, interpret-ation, integration and use of the evidence demand more complex and perhapsunfeasible solutions, as overcoming them will require substantial changes in thestructure of the health system, in the politics of science and in the way in whichhumans think and behave.

A. R. JadadCentre for Global eHealth Innovation, UniversityHealth Network and University of Toronto, Toronto,Canada

Keywords: asthma; bias; children; critical appraisal;education; eHealth; evidence-based decision making;innovation; internet.

Professor Alejandro R. Jadad, MD, DPhil, FRCPCCentre for Global eHealth Innovation4th FloorR. Fraser Elliott BuildingToronto General Hospital190 Elizabeth StreetToronto, M5G 2C4Canada

Allergy 2002: 57 (Suppl. 74): 15–22Printed in UK. All rights reserved

Copyright � Blackwell Munksgaard 2002

ALLERGYISSN 0108-1675

15

members of the public with a common platform on whichthey could interact. In addition, it could improve theefficiency with which health-related resources are allocat-ed, as resources spent on interventions shown to beineffective are shifted to more effective ones. Some claimthat the most important benefit associated with EBDM isits ability to help decision makers identify knowledgegaps, leading to research with maximum validity, preci-sion and relevance to fill these gaps (10). For instance, asystematic review showed recently that most questionsaround the risk of fetal malformations associated withasthma therapies remain unanswered (11).

Potential dangers of EBDM in asthma

The main dangers associated with the practice of EBDMare related to the manipulation and subversion of theprocess that generates the evidence or to misuse ofexisting evidence after it becomes available. Although adetailed description of different sources and types ofmanipulation and misuse of evidence is beyond the scopeof this article, the following is a brief summary of somekey issues.

Manipulation during the generation of evidence

Most research tools are very vulnerable to manipulation.The design, execution and reporting of research can beeasily ‘‘engineered’’ by the people involved in the researchprocess, to obtain results that meet their needs, expecta-tions or prior beliefs.The risk of manipulation is likely to increase in direct

proportion to the presence and prominence of thefollowing factors:• a series of interventions that belong to the same group,but that have wide variations in price;

• potentially huge financial gains or losses for industry,depending on the effectiveness and safety of theinterventions;

• ambitious researchers with unfulfilled dreams of fameand wealth;

• patients desperate for a cure; and• a government or health-care provider organizationdesperate to cut costs.The treatment of asthma provides ideal conditions for

manipulation of research. Although manipulation can bedetected easily in many cases, it can be impossible todetect, particularly when it is the result of small, verycreative and subtle efforts at different points during theresearch process.

Misuse and abuse of EBDM after the evidence is generated

EBDM can be abused by almost any decision maker atany step of the decision making process. One frequentform of misuse takes place when a person decides to pay

attention only to research evidence that supports hisor her prior beliefs or expectations, overriding allcontradicting evidence or other sources of information.For instance, sensation-seeking journalists can portray aweak piece of evidence as a ‘‘breakthrough’’ to make agood case for a headline. One form of abuse particularlyfeared by clinicians relates to the interpretation of lack ofevidence of effectiveness as evidence of lack of effective-ness. In these cases, payers may decide to fund only thosetreatments supported by strong evidence in order to savemoney or increase profits. The practice of EBDM canalso become cult-like. Some decision makers may bewilling to adhere to research evidence blindly, applying itin circumstances where it may not be appropriate whileignoring its limitations, the role of other types ofinformation and the values and preferences of otherdecision makers (10).These dangers may explain, at least in part, the

resistance experienced by some decision makers inrelation to the practice of EBDM.

Barriers to the practice of EBDM

Although EBDM has been generally assumed to be a‘‘good thing’’, its putative benefits have never been testedempirically. In addition, despite worldwide efforts topromote and implement it, EBDM has had a remarkablysmall impact on ‘‘real-world’’ decisions.The relatively small impact of EBDM could be

attributed to the existence of a series of barriers thathinder its implementation. These barriers could beassociated with the decision makers, the evidence per se,and the health-care system. In theory, the introduction ofeffective and efficient strategies to minimize or eliminatethese barriers would result in the rapid adoption andapplication of the principles of EBDM.The following is a brief discussion of some of the most

prominent barriers that hinder the practice of EBDM inasthma. They have been adapted from another source(10).

Barriers related to the decision makers

Despite the impressive efforts of the past 20 years toimprove the use of evidence in decisions, there are stillmany barriers shared by all decision makers. Some aremore prominent among health-care providers, whileothers are more important among those who lack clinicaltraining, such as consumers, journalists, lawyers andplanners. Although they are likely to affect all groups ofdecision makers, the barriers presented below do notaffect all decision makers in the same way or order.Some of the most important barriers to the practice of

EBDM from a decision makers’ perspective include:• Lack of awareness: some decision makers may not

know about EBDM, and if they do, they may have a poor

Jadad

16

understanding of it and therefore not realize its fullbenefits. Lack of awareness is likely to affect more thosedecision makers that do not have formal training inhealth and health-care issues.

• Lack of time: this is an important barrier thataffects mainly the health-care providers. Most clinicians,for instance, are facing increasing workloads and areseeing how the amount of time available to study and tokeep up to date is shrinking rather than expanding. Thisis compounded by the lack of reward systems that fosterthe practice of EBDM. With the advent and rapidgrowth of the internet, many patients are now havingmore time to explore the available research evidencethan the clinicians involved in their care. Althoughrigorously generated data are not yet available, itappears that clinicians are often finding themselvesupstaged by patients who bring along internet down-loads with research evidence related to their medicalconditions and treatments (12).

• Lack of motivation: some decision makers, even ifaware of and with time to engage in EBDM, may not feelmotivated to practise it. This could be explained, amongother reasons, by their lack of trust in research and thewhole EBDM approach, or by their perceived lack ofskills to practise EBDM efficiently.

• Inadequate skills and resources to access the evidence:most decision makers, including prominent clinicians,have never had the opportunity to participate in coursesand practical sessions on how to search the biomedicalliterature. Even if they did have such opportunity, mostdecision makers are unlikely to have access to pro-grammes designed to maintain these skills and keep upwith the developments in bibliographic databases. Inother cases, particularly in developing countries, decisionmakers may have the skills but no access to sources ofresearch evidence, namely computers, bibliographicdatabases or even journals.

• Limited critical appraisal skills: few decision makershave received formal courses on how to judge the validityand relevance of research evidence. Even if they had, theeffects of such courses are still unclear (13).

• Limited knowledge about how to integrate researchevidence with other types of information: research evidenceis just one of many different types of information thatdecision makers can use to support and guide their de-cisions. Most people are likely to be influenced, to variousdegrees, by clinical, political, financial or anecdotalinformation. The interaction between different types ofinformation in health-care decision making is perhaps ascomplex and poorly understood as the interactionbetween information, values, preferences and circum-stances.

• Limited knowledge about how to integrate researchevidence with the values, preferences and circumstancesof the decision makers: planners, politicians, adminis-trators, physicians, nurses, patients and the public basehealth-care decisions on external circumstances (such

as setting, financial and other resources available,marketing efforts by industry and the political climate)and internal circumstance (individual priorities, values,potential personal gains and losses and feelings aboutwhat is or is not important). Little is known, however,about the complex interaction among these factors. Inparticular, we have a poor understanding of how dif-ferent types of information modulate each other, andhow they are filtered by the values and preferences ofthe decision makers, and the circumstances under whichdecisions are made.

• Poor understanding of the interaction between EBDMand other ‘‘ways of knowing’’: it could also be argued thatthe lack of impact of EBDM is best explained by theextent to which nonevidential factors influence decisions.These factors, which the traditional approach to EBDMdoes not address satisfactorily, include other very power-ful ‘‘ways of knowing’’ that are essential to the decisionmaking process, such as personal experience, rules ofthumb and what is commonly labelled as ‘‘instincts’’ or‘‘intuition’’.

• Poor understanding of the interaction among differentdecision makers: the lack of understanding of theinteraction among the different factors that influencedecisions within individual decision makers makes iteven more difficult to understand the interactions amongdifferent decision makers participating in the samedecision.

Barriers related to the evidence

Even if decision makers have optimal skills to practiseEBDM, they would face many barriers emanating fromthe current status of research evidence. Some of the mostprominent evidence-related barriers include:

• Abundance: the amount of research evidence that isbeing produced in the world makes it impossible forany decision maker to keep up to date or even moreimportantly, to find valid and clinically relevantmaterial in the overwhelming body of literature. InApril 2000, a search of the Cochrane Controlled TrialsRegister, the largest collection of clinical trials in theworld (see below), found that ‘‘asthma’’ was mentionedin more than 6000 citations or almost 2.5% of the totalnumber of clinical trials identified to date. Clinical tri-als, of course, are just one of many researchapproaches that could be used to study asthma-relatedissues.

• Poor internal validity (great risk for bias): as men-tioned above, human beings are prone to bias.Empirical research, for instance, has shown consistentlythat most clinical trials exaggerate the benefits oftreatments studied (14). Several factors contribute tothis exaggeration of benefits. Many studies aredesigned, conducted and reported by researchers whosecareers are closely linked to the interventions theyevaluate. Most patients want interventions to be effective.

The tools of the trade

17

Funding is often provided by organizations that thrive onbreakthroughs and positive results. The latter is particu-larly true for studies funded by pharmaceutical compa-nies (15), which represent a substantial proportion of thetotal volume of research evidence available.• Limited relevance: many research efforts fail to meet

the needs of clinicians, policy makers and patientsbecause they are designed to meet the needs of academics,funding organizations and regulatory agencies. Fewstudies include measurements of quality of life, patientpreferences or the resource implications of interventions,and few last long enough to produce clinically mean-ingful answers. Most studies do not express their resultsin ways that clinicians, patients and policy makers canapply easily to their decisions, and are inadequate to helpclinicians and patients select treatments among manyoptions that have never been compared directly. Wecannot afford to perpetuate this situation.• Limited precision: most studies are too small to

provide precise answers, particularly around adverseeffects.

Barriers related to the health-care system

The barriers imposed by the health care system are verystrong and without exception present all over the world.One of the most important barriers is the lack of financialincentives for clinicians, planners and policy makers toparticipate in EBDM. In the case of clinicians, forinstance, performance tends to be judged by the numberof patients they see and the resources they generate,rather than on their efforts to make decisions based onthe best available evidence. Another important barrier isthe lack of tools that could help decision makers to accessand use the best available evidence to guide theirdecisions, at the right time, in the right place, in theright format and at the right speed. Most clinical settings,for instance, were designed and built before the era of theWorld Wide Web. Many settings, on the other hand, haveto cope with rapidly increasing workloads that make iteven more difficult to introduce information technologythat could support EBDM.

Overcoming the barriers: the tools of the trade

As highlighted above, the fate of EBDM will depend, to agreat extent, on the availability of effective and efficientstrategies to minimize or eliminate the existing barriersfaced by decision makers in the real world. Until recently,there were few solutions in sight.Since the last decade of the 20th century, develop-

ments in information technology are allowing thecreation of tools with unprecedented power that couldfoster the practice of EBDM. Some of these tools havebeen designed to help decision makers formulateanswerable questions; find research evidence which

may answer them; appraise such evidence; consider itsvalidity, relevance and applicability; and use and sharethe best available knowledge at the point of decisionmaking (1).The remainder of this article will provide a description

of some of the most prominent tools for the practice ofEBDM, with a focus on child health and the managementof asthma in children. Emphasis has been placed onresources that are available on the internet, mainly free ofcharge.

How-to books

There are over 100 textbooks whose main purpose is toprovide basic concepts and tools to support peopleinterested in the practice of EBDM. For the purposes ofthis article, two books have been selected; one because itcovers a wide spectrum of basic issues of EBDM and theother because it is available, free of charge, on the internet.A more extensive list can be found at the BMJ bookstore,one of the most comprehensive online resources to identifybooks on the basic principles of EBDM (16).

Evidence-based medicine: how to practise and teachEBM. This pocket-sized book is the best-seller in thisfield. Now in its second edition, this book contains 260pages and a free CD-ROM produced by five of theforemost proponents of EBDM in the world. A descrip-tion of the contents of the book is available on theinternet (17).Written for the practitioner, this short and highly

practical book makes a strong case about the feasibility ofthe practice of EBDM in busy clinical environments. Thechapters cover the whole spectrum of clinical practice andalso practical tips on how to formulate answerablequestions and find the best evidence to answer them.The accompanying CD-ROM contains backgroundpapers from 14 other health disciplines, clinical examplesand extended descriptions (including sample Web pages)of several key evidence sources, some of which aretargeted specifically to child health or provide access totool for handheld devices (18).

Randomized controlled trials: a user’s guide. This bookcontains eight chapters with answers to the 100 mostfrequently asked questions about randomized controlledtrials (RCTs) and EBDM. It is included in this articlebecause it is now available on the internet, full text andfree of charge (19).

Tools to improve the quality of reporting of research

Authors interested in reporting RCTs and systematicreviews can find checklists and flowcharts prepared byinternational collaborative efforts (20, 21). Similar effortsare under way to assist and improve the reporting ofother types of research articles.

Jadad

18

Tools to formulate answerable questions

The formulation of answerable questions is one of themost important, and yet overlooked and underestimated,aspects of EBDM. Under ideal circumstances, the prac-titioner of EBDM should be able to assemble questionsthat include information on the broad characteristics ofthe participants (i.e. children between 8 and 12 years ofage), the condition (i.e. mild asthma), the setting (i.e.school), the interventions (i.e. inhaled steroids) and theoutcomes (i.e. compliance) (22). Educational resourcesand tips on how to formulate questions can be foundelsewhere (23).

Tools to search for research evidence

Tools to access MEDLINE on the Internet. MEDLINEbecame available on the Internet, free of charge, in 1997(24). This service, also known as PubMed, includesrefined search strategies for optimizing searches forclinically useful studies (25). An ambitious (and verycontroversial) project led by the US National Institutes ofHealth aims at extending access to the full text of articlespublished in peer-reviewed journals, free of charge, on theinternet. If successful, this project (also known asPubMed Central), will provide links to PubMed abstractsof the references used in the articles, figures sized foron-screen viewing and support for supplementary infor-mation such as data tables, streaming video and high-resolution images (26).

Free access to journal contents

While PubMed Central evolves to include all journalsin biomedical sciences, many individual journals arealready making their contents available free of chargeon the internet. There are several collections of asthma-related journals available online on the internet (27, 28).A new breed of internet-based journals is emerging.

These are journals that are produced and disseminatedentirely through the internet. An example is BioMed-Central (29).

Appraisal tools

Validated tools to appraise evidence. There are severalvalidated tools to assess the quality of RCTs, systematicreviews and clinical practice guidelines (30–32).

The CATMaker. This is an internet-based software toolthat helps clinicians produce short summaries of art-icles on clinically relevant topics, known as ‘‘criticallyappraised topics’’ (CATs). These CATs can then be usedand enhanced by the clinician who produced them orby colleagues who come across the same clinical issuesand relevant publications elsewhere. The CATMakeris associated with a CATbank, a storage and retrieval

facility for a collection of CATs available on the internet(33).The tool includes the following functions:

• prompts clinical questions, search strategy and keyinformation about any relevant study found;

• online critical appraisal guides for assessing the validityand usefulness of the study;

• automatic calculation of clinically useful measures (andtheir 95% confidence intervals);

• assistance to formulate clinical ‘‘Bottom Lines’’ fromthe study once it is read;

• creation of one-page summaries (CATs) that are easyto store, print, retrieve and share (as both text andHTML files);

• reminders on when to update each CAT; and resourceson how to teach others how to practice EBM.

Evidence-based publications

This is a group of secondary or ‘‘new breed’’ publicationsthat perform the initial sorting of evidence using explicitmethods to identify valid and clinically useful articlesfrom a large number of journals, and then provideconcise informative titles, abstracts and commentariesthat help readers discern whether the information appliesto their own decisions. Surprisingly, these publicationsare typically thin and relatively infrequent in publication(e.g. bimonthly or quarterly), reflecting the fact that avery small proportion of the literature is really worthreading. ACP Journal Club (ACPJC), the first of these‘‘evidence-based’’ publications, appeared in 1991. It wasfollowed by Evidence-Based Medicine (EBM) in 1996 andEvidence-Based Nursing and Evidence-Based MentalHealth in February of 1998. Similar journals are alreadyemerging in many other areas in health care.

Bandolier, another important member of this group oftools, is produced monthly by the Oxford Anglia NHSRegion in the United Kingdom (34). It contains veryuseful ‘‘bullet points (hence Bandolier) of evidence-basedmedicine’’, which are available on the internet, free.

Specialized compendia of evidence for clinical practice

Three superb examples are the Cochrane Library, BestEvidence and Clinical Evidence.

The Cochrane Library. This is the main product of theCochrane Collaboration, an international organizationthat aims to help people make informed decisions abouthealth, by preparing, maintaining and ensuring theaccessibility of rigorous, systematic and up-to-date re-views (and where possible, meta-analyses) of the benefitsand risks of health-care interventions (35). The rapidgrowth and recognition of the Cochrane Collaborationhas been due, to a great extent, to its extensive use of theInternet to provide interested individuals with access to

The tools of the trade

19

manuals, tools and training materials to facilitate thedesign and conduct of the reviews, easy access to thereviews, and continuous open peer-review (1).The Cochrane Library is a regularly updated electronic

library designed to give decision makers the evidencethey need to make informed health-care decision making,with special emphasis on data from RCTs. Launched inApril 1995 as the Cochrane Database of SystematicReviews (CDSR), it was renamed to reflect the inclusionof additional important related databases, making itperhaps the most comprehensive source of evidence forall those interested in EBDM. The Cochrane Libraryis issued quarterly and contains general information onthe Cochrane Collaboration, a handbook on how toconduct systematic reviews, and the following fourdatabases:• The Cochrane Database of Systematic Reviews

(CDSR): this is a rapidly growing collection of regu-larly updated systematic reviews of the effects of healthcare prepared by members of Collaborative ReviewGroups. It also includes protocols of reviews inprogress. A search of the Cochrane Library in March ofthe year 2000 revealed 59 completed reviews and 58protocols under development by members of the Coch-rane Collaboration (35). In a recent review of systematicreviews and meta-analyses on the treatment of asthma,Cochrane reviews were shown to be more rigorousthan those published in paper-based peer-reviewedjournals (36).• The Database of Abstracts of Reviews of Effectiveness

(DARE): this is a database of structured abstracts ofthousands of systematic reviews from around the world,all of which have been completed outside of the CochraneCollaboration. These reviews have been approved byreviewers at the National Health Service Centre forReviews and Dissemination at the University of York,England. DARE also includes brief records of reviewsthat may be useful for background information,abstracts of reports of health technology agenciesworldwide, and abstracts of reviews in the journals ACPJournal Club and Evidence-based Medicine. In March2000, DARE included information from 35 reviewsrelated to asthma.• The Cochrane Controlled Trials Register: this is the

most comprehensive collection of RCTs in the world,which results from worldwide ongoing searches of thebiomedical literature, using manual and electronic means.As mentioned above, this database included over 6400citations related to asthma in the first issue of the Coch-rane Library of the year 2000.• The Cochrane Review Methodology Database

(CRMD): this is a database that contains hundreds ofcitations of articles on the science of research synthesisand on practical aspects of preparing systematic reviews.

The Cochrane Library is available on CD-ROM forWindows and Macintosh and should be regarded as an

evolving product. More details on the Cochrane Libraryand on distributors can be found on the internet (35).

Best evidence. This is an easy-to-search database thatincludes the contents of all the issues of the journals ACPJournal Club and Evidence-based Medicine (37).

Clinical evidence. This is a new addition to this group (38).It is a 6-monthly updated compendium of evidence behindcommon clinical interventions and practices, producedjointly by the BMJ Publishing Group and the AmericanCollege of Physicians ) American Society of InternalMedicine. It provides ‘‘a concise account of the currentstate of knowledge, ignorance, and uncertainty about theprevention and treatment of a wide range of clinicalconditions based on thorough searches of the literature’’.The producers emphasize that Clinical Evidence is not atextbook of medicine nor a book of guidelines. Itsummarizes the best available evidence, and where thereis no good evidence, it says so. The entire contents ofClinical Evidence will be available on the Internet shortly.

The National Guideline Clearinghouse. The NationalGuideline ClearinghouseTM is an internet-based publicresource that enables access to evidence-based clinicalpractice guidelines and that allows comparisons ofrecommendations produced by different organizationsin North America (39).The National Guideline ClearinghouseTM (NGC) is a

comprehensive database of evidence-based clinical prac-tice guidelines and related documents produced by theAgency for Healthcare Research and Quality (AHRQ)(formerly the Agency for Health Care Policy andResearch [AHCPR]), in partnership with the AmericanMedical Association (AMA) and the American Associa-tion of Health Plans (AAHP).The NGC mission is to provide physicians, nurses and

other health professionals, health-care providers, healthplans, integrated delivery systems, purchasers and othersan accessible mechanism for obtaining objective, detailedinformation on clinical practice guidelines and to furthertheir dissemination, implementation and use. Key com-ponents of NGC include:• structured abstracts (summaries) about the guidelinesincluded and their development;

• a tool for comparing two or more guidelines side-by-side;

• syntheses of guidelines covering similar topics, high-lighting areas of similarity and difference;

• links to full-text guidelines, where available, and/orordering print copies;

• an electronic forum, NGC-L, for exchanging infor-mation on clinical practice guidelines, their develop-ment, implementation and use; and

• annotated bibliographies on guideline developmentmethodology, implementation, and use.

Jadad

20

Collections of websites of resources and organizationssupporting EBDM

These are Internet-based ‘‘look up’’ services that provideaccess to evidence-based information and to sites thatsupport EBDM. Examples include the Center forEvidence-based Child Health Web site; The HealthInformation Research Unit at McMaster University;The Center for Evidence-based Medicine, Mount SinaiHospital and University Health Network, University ofToronto and Netting the Evidence (8, 40–42).

Tools to tailor evidence to individual patient characteristics

There is a proliferation of Internet ‘‘calculators’’ designedto match unique individual profiles with an underlyingevidence-based knowledge repository and sources ofpotential feedback. Examples include tools for morbidityrisk assessments, drug dosing and delivery of up-to-dateevidence-based care recommendations, all tailored to aperson’s unique values, preferences and circumstances(1). To increase compliance with the recommendations,the Internet can also be used to deliver reminder messagesto providers and consumers.

Looking ahead: beyond text, desktop computersand providers

To date, most efforts to promote EBDM have been text-based and focused on the needs of health providers. Untilnow, text has been the most efficient way for humans todisseminate scientific information. With the rapid evolu-tion of information technology we already have tremen-dous opportunities to exchange research evidence usingformats that aremore consistent with our human nature. Ifcurrent trends in the development of multimedia andbroadband communications continue, it will soon bepossible to anyone to produce and receive knowledgeinvolving visual, auditory, tactile and even olfactorystimuli. These new modalities of communication mayprovide the long awaited solutions to increase the level offunctional health literacy among the public, which arealarmingly low even in developed countries (43).Thanks to existing and ongoing developments in porta-

ble and wearable computers, internet appliances (e.g.fridges with internet access) and wireless networks, allgroups of decision makers now have the opportunity toexchange knowledge to and from anywhere in the world.Clinical settings and homes are rapidly becoming knowl-edge management centres. This is creating unprecedentedopportunities for new partnerships between health-careproviders and the public (44, 45). However, the rapidproliferation of consumer-orientated communicationstechnology is also creating important threats to thetraditional role of the health-care system and its profes-sionals, who will lose their ‘‘mind share’’ unless they take

effective measures to keep up with the changes in the worldaround them (46).All these changes will have a profound effect on the

management of diseases of childhood. With the rapidproliferation of access to the Internet at home and atschools, there will be an increasing number of opportuni-ties to engage parents and children in the decision makingprocess (47). One example of this is LinkMedica’s Knowl-edge Management Center, a joint effort by industry,patient groups and academics, to provide people affectedby asthma with access to interactive glossaries and multi-level summaries of the best available evidence on themanagement of asthma (48). Another good example is thatof interactive evidence-based stories designed to helpchildren with asthma, their parents and teachers to learnabout asthma and to access the best available knowledgethrough the internet. The story, entitled ‘‘Mr Knowsitlearns about asthma’’ includes a cartoon character withinsatiable curiosity, Mr Knowsit, who helps readers bringintuitive insight, their own values, the experience of others,research evidence and internet technology together, tocontribute to their learning process (49). It is throughinitiatives such as these that children could become familiarwith basic principles of health-related decisionmaking andresearch evidence and obtain high levels of health literacy.If efforts such as this succeed, children will be in a betterposition to participate in health-care decisions that af-fect them directly. Interventions targeting children mayalso have long-lasting effects that could reduce the need foradditional education and reinforcement once they becomeadults, leading to potentially stronger collaborative effortswith future generations of children and parents.

Closing remarks

At the dawnof the InformationAge, the practice of EBDMis still hindered by many important barriers. Some of thesebarriers, particularly those related to the distillation, dis-semination and packaging of research evidence, could beovercome by recent and ongoing developments in virtualreality, portable/wearable computers, internet appliances,multimedia andwireless broadband internet traffic. Think-ing that all barriers to the practice ofEBDMcould be solvedby fancy information technology is naive. Barriers related tothe generation, interpretation, integration and use of theevidence demand more complex and perhaps unfeasiblesolutions, as overcoming them will require substantialchanges in the structure of the health system, in the politicsof science and in theway inwhich humans think andbehave.

Note added in proof

For biographical details of the author, please visithttp://www.uhnres.utoronto.ca/ehealth/html/who/eh_who_bio_ jadad.shtml.

The tools of the trade

21

References

1. Jadad AR, Haynes RB, Hunt D,Browman GP. The Internet andevidence-based decision making: aneeded partnership for efficient knowl-edge management in health care. CMAJ2000;162:362–365.

2. Guyatt G, Rennie D. Users’ guides tothe medical literature. JAMA1993;270:2096–2097.

3. Gray JAM. Evidence-Based Healthcare:How to Make Health Policy and Man-agement Decisions. London: ChurchillLivingstone, 1997.

4. AtkinsD,KamerowD, EisenbergGM.Evidence-based medicine at the Agencyfor Health Care Policy and Research.ACP J Club 1998;128:A14–A16.

5. Mulhall A. Nursing research and theevidence. Evidence-Based Nursing1998;1:4–6.

6. Smith K. Evidence-based managementin health care. In: Peckham M, Smith R,eds. Scientific Basis of Health Service.London: BMJ Publishing Group, 1996,92–98.

7. Geddes J, Reynolds S, Streiner Det al. Evidence-based practice in mentalhealth. Evidence-Based Mental Health1998;1:4–5.

8. Centre for Evidence-based Child

Health. London, UK: Institute of ChildHealth and Great Ormond Street Hos-pital for Children NHS Trust [http://www.ich.bpmf.ac.uk/ebm/ebm.htm].

9. Sackett DL, Straus SE, Richardson

WS, et al. Evidence-Based Medicine.How to Practise and Teach EBM, 2ndedn. Edinburgh: Churchill Livingstone,2000, 261 pp. [includes CD-ROM].

10. Jadad AR. From trials to decisions: thebasis on evidence-based health. In:Jadad AR ed. Randomised ControlledTrials. A User�s Guide. London: BMJBooks, 1998 [http://www.bmjpg.com/rct/contents.html].

11. Jadad AR, Sigouin C, Mohide PTet al. Risk of congenital malformationsassociated with the treatment of asthmaduring the first trimester of pregnancy: asystematic review. Lancet 2000;355:119.

12. Coiera E. The Internet’s challenge tohealth care provision. BMJ 1996;31:3–4.

13. Norman GR, Shannon SI. Effectivenessof instruction in critical appraisal(evidence-based medicine) skills: acritical appraisal. Can Med Assoc J1998;158:177–181.

14. Jadad AR, Rennie D.The randomizedcontrolled trial gets a middle-aged check-up [editorial]. JAMA 1998;279:319–320.

15. Bero LA, Rennie D. Influences on thequality of published drug studies. Int JTechnol Assess Health Care1996;12:209–237.

16. BMJ Bookstore

[http://www.bmjbookshop.com].

17. Table of contents. Evidence-based Med-icine: How to Practise and Teach EBM[http://www.cebm.utoronto.ca/].

18. Syllabi for Practicing EBM [http://www.cebm.utoronto.ca/].

19. Jadad AR. Randomised ControlledTrials: a User’s Guide. London, UK:BMJ Books, 1998 [http://www.bmjpg.com/rct/contents.html].

20. Moher D, Cook DJ, Eastwood Set al. for the QUOROM Group.Improving the quality of reports ofmeta-analyses of randomised controlledtrials: the QUOROM Statement [http://www.cheori.org/tcc/Viewpoints/quorom.htm].

21. The CONSORT Statement [http://ja-ma.ama-assn.org/info/auinst_pchk.html].

22. Richardson W, Wilson M, Nishikawa

J, et al. The well-built clinical question: akey to evidence-based decisions. ACP JClub 1995;123:12–13.

23. Teaching Tips For FormulatingAnswerable Clinical Questions [http://www.cebm.utoronto.ca/practise/formulate/].

24. PubMed [http://www4.ncbi.nlm.nih.gov/PubMed/].

25. PubMed’s Clinical Queries [http://www.ncbi.nlm.nih.gov/PubMed/clinical.html].

26. PubMed Central

[http://www.pubmedcentral.nih.gov/].27. Respiratory Medicine Journals. Med-

Nets [http://www.mednets.com/pulmonarjournals.html].

28. MedBioWorld [http://www.sciencekomm.at/med/journals/pulmonary.html].

29. BioMedCentral[http://www.biomedcentral.com/].

30. Jadad AR. Assessing the quality ofRCTs: why, what, how, and bywhom? In:Jadad AR, ed. Randomised ControlledTrials: a User’s Guide, Chapter 4. Lon-don, UK: BMJ Books, 1998 [http://www.bmjpg.com/rct/chapter4.html #fig4–1].

31. Jadad AR. Oxman and Guyatt’s indexof the scientific quality of researchoverviews. In: Jadad AR, ed. Rando-mised Controlled Trials: a User’s Guide,Chapter 6. London, UK: BMJ Books,1998 [http://www.bmjpg.com/rct/chapter6.html #fig6_1].

32. Appraisal Instrument for ClinicalGuidelines. London, UK: St George’sHospital, Health Care Evaluation Unit[http://www.sghms.ac.uk/Departments/phs/hceu/form.htm] (the validation pro-cess for this instrument is currentlyunder way).

33. The CatMaker [http://cebm.jr2.ox.ac.uk/docs/catmaker.html].

34. Bandolier. UK NHS R & D Programme[http://www.jr2.ox.ac.uk/bandolier/bandopubs/bandopubs.html]

35. The Cochrane Collaboration [http://www.cochrane.org/].

36. Jadad AR, Moher M, Browman GP,et al. Systematic reviews and meta-ana-lyses on the treatment of asthma: a crit-ical evaluation. BMJ 2000;320:537–440.[http://www.bmj.com/cgi/content/full/320/7234/537].

37. Best Evidence and Evidence-basedMedicine [http://hiru.mcmaster.ca/acpjc/acpod.htm].

38. Clinical Evidence [http://www.clinicalevidence.com/lpBinCE/lpext.dll?f=templates&fn=main-h.htm&2.0].

39. National Guideline Clearinghouse

[http://www.guidelines.gov].40. Evidence-based Health Care Organiza-

tions, Health Information ResearchUnit, McMaster University. Hamilton,Ontario [http://hiru.mcmaster.ca/cochrane/centres/canadian/evidence-based.html].

41. Evidence Resources. Toronto: Centrefor Evidence-based Medicine, MountSinai Hospital and University HealthNetwork, University of Toronto[http://www.cebm.utoronto.ca/resources/].

42. Netting the Evidence. A ScHARRIntroduction to Evidence Based Practiceon the Internet [http://www.sheffield.ac.uk/�scharr/ir/netting/].

43. Ad Hoc Committee on Health

Literacy for the Council of

Scientific Affairs, American

Medical Association. Health literacy.Report of the Council of ScientificAffairs. JAMA 1999; 281:552–557.

44. Jadad AR. Promoting partnerships:challenges for the Internet age. BMJ1999;319:761–764 [http://www.bmj.com/cgi/content/full/319/7212/761].

45. Eysenbach G, Sa ER, Diepgen TL.Shopping around the Internet todayand tomorrow: towards themillennium of cybermedicine. BMJ2001;319:1294 [http://www.bmj.com/cgi/content/full/319/7220/1294].

46. Eysenbach G, Jadad AR. Evidence-based patient choice and consumerhealth informatics in the Internet age.J Med Internet Res 2001;3:E19 [http://www.jmir.org//2/e19/].

47. Holmes-Rovner M, Llewellyn-Thomas H, Entwistle V et al. Patientchoice modules for summaries of clinicaleffectiveness: a proposal. BMJ2001;322:664–667.

48. LinkMedica’s Knowledge Manage-

ment Centre [http://www.linkmedi-ca.co.uk/Asthma/KC/KC_Home.asp].

49. Foresight Links Corporation.Mr. Knowsit[http://www.foresightlinks.com/default_mrk.html].

Jadad

22