physicians' awareness regarding evidence-based medicine

Original Articles

Physicians' Awareness Regarding Evidence-based Medicine, Practice

Guidelines and Clinical Information Resources in Japan

Needs Assessment Prior to the Initiation of •gMedical Information Network Distribution Service (Minds)•h

Toshihiko Satohl'2, Takeo Nakayama3, Yasuto Sato1,4, Keika Hoshil, Koichi Miyaki1,5, Noriko

Kojimahara1, 4, Narumi Eguchi6, Takahiro Okamoto7, Yoko Hayashil, Naohito Yamaguchi1, 4

1 Medical Information Service Center, Japan Council for Quality Health Care2 Department of Preventive Medicine and Public Health

, School of Medicine, Kitasato University, Kanagawa

3 Department of Health Informatics, Kyoto University Graduate School of Public Health, Kyoto4 Department of Hygiene and Public Health II

, Tokyo Women's Medical University, Tokyo5 Department of Preventive Medicine and Public Health

, School of Medicine, Keio University, Tokyo 6 Japan Medical Association Research Institute

Department of Endocrine Surgery, Tokyo Women's Medical University, Tokyo

BACKGROUND: physicians' awareness regarding evidence-based medicine(EBM), clinical practice guidelines, and

clinical information resources were rarely examined in Japan. We need to know them prior to the initiation of the

Medical Information Network Distribution Service (Minds) by the Japan Council for Quality Health Care(JCQHC).

METHODS: A total of 10,000 directors/owners of private clinics (CDs: clinic physicians) affiliated with the Japan

Medical Association(JMA)and 8682 physicians working for hospitals certified by the JCQHC(HDs: hospital physi-

cians)were randomly selected and surveyed by a mailed questionnaire.

RESULTS: The response rate to the questionnaire was 18.7%(n=1865)among CDs and 67.8%(n=5885) among

HDs. The percentage of respondents who uses internet was 39.9% among CDs and 69.3% among HDs. The infor-

mation resource most commonly used by all respondents was medical journals, followed by textbooks. The percent-

age of respondents who used medical literature database was 10.8% among CDs and 49.7% among HDs, respec-

tively. Approximately 80% of all respondents approved implementing EBM in daily practice. Fifty percent of all re-

spondents indicated to have used clinical practice guidelines, and 90% of all the guideline users replied that clinical

practice guidelines are useful tools for clinical decision-making. Over half of HDs required to access to the abstracts of the literature cited in the guidelines.

CONCLUSIONS: Many physicians who responded to the survey acknowledged that EBM will contribute to im-

proving the quality of medical services. They are positive in using clinical practice guidelines that include a series of recommendations proposed by specialists in the relevant field(s) in accordance to the reviewed evidence.

KEY WORDS: Evidence-based Medicine(EBM), clinical guideline, awareness, clinical information resources

Gen Med: 2004; 5: 13-20

Author for Correspondence: Toshihiko Satoh Department of Preventive Medicine and Public Health, School of Medicine, Kitasato University 1-15-1 Kitasato, Sagamihara, Kanagawa 228-8555, Japan. Phone:+81-42-778-9352, Fax:+81-42-778-9257, E-mail: [email protected]

14 General Medicine, Vol. 5, No. 1, 2004

It is widely accepted today that the practice of

evidence-based medicine (EBM) can contribute to

improving the quality of patient care. EBM meth-

odology is rapidly becoming an important topic to be

included in undergraduate, postgraduate, and continuing

medical education, and researchers all over the world

are trying to collect current best evidence to promote

EBM1-3.

On the other hand, clinical practice guidelines were

defined by the U.S. Institute of Medicine as systemati-

cally developed statements to assist practitioners and

patients in making decisions about appropriate health

care for specific clinical circumstances4. According to

Grimshaws and Koyama6, the use of clinical practice

guidelines can lead to a better patient care and im-

proved patient outcome. Fukui and Tango released A

Guide to the Development of Clinical Practice Guide-

lines', in which they claimed that guideline develop-

ment should involve the participation of not only physi-

cians and clinicians specializing in the medical condi-

tion addressed by the guidelines but also clinical epide-

miologists, medical librarians8 and representatives of pa-

tients.

In Japan, the first governmental report on health

technology assessment which officially introduced EBM was issued in 19979. The second report was issued in

1999 and it referred to the need to develop evidence-based clinical practice guidelines10. The Ministry of

Health, Labour and Welfare (MHLW) established pri-orities for the development of clinical practice guide-lines in the 1999 report, and started several projects to

develop them. During the 2001 Forum for the Dissemi-nation of Information on Health and Medical Technol-

ogy, establishment of an Internet data center aiming at

providing access to clinical practice guidelines and in-formation on EBM was officially suggested. In view of

this, Japan Council for Quality Health Care (JCQHC), the independent third party, initiated to prepare an in-

formation service called Medical Information Network Distribution Service (Minds) in 2002. Minds, granted from the MHLW, mainly provides access to clinical

practice guidelines via Internet to improve the quality of patient care in Japan by providing information that

can assist both healthcare providers and consumers in

making better healthcare decisions. Upon initiating Minds, we have carried out a large-scale, cross-

sectional survey about Japanese physicians in order to identify their needs for clinical information, including

the types of information resources used for clinical decision-making, and their level of awareness and ex-

pectation regarding EBM and evidence-based clinical

practice guidelines.

SUBJECTS AND METHODS

Firstly, ten thousand subjects were randomly chosen

from 70,000 directors/owners of private clinics and hos-

pitals (hereinafter referred to as CDs: clinic physicians) who are also the Japanese Medical Association (JMA) members. Secondly, in order to perform a random sam-

pling of physicians working at 773 hospitals certified by the JCQHC (hereinafter referred to as HDs: hospital

doctors), HDs were weighted based on the number of full-time doctors available at the hospitals they worked for. Thus, 8682 HDs were chosen for the survey. Ques-

tionnaires were mailed to all subjects in December 2002. The cover letter of the questionnaire provided the

summary and purpose of Minds as well as a statement

describing that the questionnaire was part of a needs as-sessment conducted prior to the initiation of Minds.

The questionnaire sent to the subjects was made up of 11 pages and consisted of 24 items, including the

items related to 1) the attributes of respondents (e.g.,

gender, age, years of experience in clinical practice, and the area of specialty), 2) Internet accessibility, 3) level of awareness regarding EBM and evidence-based clini-

cal guidelines, and 4) the types of medical information service needed by the respondents. Subjects were asked

to return the questionnaire by post. A reminder was sent out to all subjects who had not responded by Janu-ary 2003. Only those respondents who returned the

questionnaire by the end of January were included in the survey, whose responses were then aggregated and

analyzed. In order to adjust for the differences in attrib-utes between those subjects who did and did not re-

spond to the questionnaire, weights were assigned to

CDs according to their age and to HDs according to the size of hospitals (the number of full-time physicians) for which they worked (See Appendix).

RESULTS

Attributes of Respondents

Responses were obtained from 1865 CDs (response rate: 18.7%) and 5885 HDs (response rate: 67.8%).

Table 1 shows separately the attributes of respon-

dents for CDs and HDs. On account of the fact that

CDs were generally older than HDs, the percentage of

respondents who uses computers and/or having access

to the internet at work was lower among CDs than

T.Satoh, et al. Physicians' Awareness Regarding EBM and Practice Guidelines 15

Table 1 Attributes of the respondents

+ : Multiple responses allowed

$ : Concerning only the workplace

Table 2 Sources of information used in clinical decision-making +

+ : Multiple responses allowed

HDs. The number of respondents specializing in inter-

nal medicine was over 50% among CDs while it was

only about 25% among HDs.

Information Resources Used for Clinical Decision-making

Table 2 shows the information resources used by re-

spondents to solve problems and uncertainties associ-ated with individual patient care. Medical journals were

the most commonly used information resource among

both CDs and HDs. According to Table 2, all types of information resources excepting the manuals were more

actively used by HDs than by CDs. The percentage of

HDs using medical information database, Internet, ad-vice of colleagues/supervisors, and expert opinion was

particularly high. The percentage of respondents using medical information database, such as MEDLINE, was

about 50% among HDs while it was only 10% among CDs. Only 15% of CDs used Internet to search clinical

information.


Table 3 Awareness regarding Evidence-based Medicine and clinical guidelines

Respondents' Awareness of EBM and Clinical

Practice Guidelines

Table 3 shows the respondents' awareness regarding EBM and clinical practice guidelines. The percentage of

respondents who replied that they were very familiar with EBM was slightly higher among HDs than among

CDs. About 80% of CDs and 90% of HDs were either very familiar with or somewhat familiar with EBM.

About 80% of CDs and 90% of HDs either simply agreed or strongly agreed with implementing EBM into

clinical practice.The percentage of respondents who indicated as actu-

ally using clinical practice guidelines was 56% among CDs and 52% among HDs. Most of the respondents

(85% CDs and 89% HDs) felt that clinical practice

guidelines are either sometimes useful or very useful as a tool for making clinical decisions.

Information Required to Make Appropriate Clini-

cal Decisions

Table 4 summarizes the types of information the re-

spondents feel that should be included in clinical prac-

tice guidelines, including whether or not they wish to

have access to the articles cited in clinical practice

guidelines. Compared to CDs, HDs tended to require more detailed information in the guidelines; in fact, 56

%of HDs indicated that the abstracts of the articles cited in the guidelines should be made accessible to

guideline users. Furthermore, over 90% of all respon-dents indicated that they would refer to the full text of

the articles cited in the guidelines provided that they were readily available to the guideline users.

Reasons for Using Medical Information and Tim-

ing for Referring to Such Information

Table 5 summarizes the reason for using medical in-

formation and the timing for referring to such informa-

tion. The most common reason for using medical infor-

mation was •gto make decisions about the care of indi-

vidual patients•h (68% CDs and 80% HDs), followed by

•g to obtain up-to-date medical information•h and •gto ob-

tain information required to give advice to a patient and

to his/her family•h. Approximately half of all respon-

dents replied that they referred to such information

either during the interval between patient examinations

or after hospital hours. Only 15% of all respondents re-

plied that they referred to medical information during a


Table 4 Information Required to Make Appropriate Clinical Decisions

Table 5 Reasons for Using Medical Information and Timing for Referring to Such Information

+: Multiple responses allowed

patient examination.

DISCUSSION

Methodological Issues The findings from this large-scale survey, which cov-

ers approximately 15 % of all directors/owners of pri-vate clinics/hospitals in Japan and physicians working at 8% of all hospitals located in Japan, were initially

expected to represent the opinions of Japanese physi-

cians. However, the response rate was as low as less than 20% for CDs. Furthermore, the surveyed HDs in-

cluded only those that worked at the hospitals certified by the JCQHC. So, one cannot deny the possibility that both CDs and HDs included in the present survey have

a higher awareness about EBM than the average CDs

and HDs in Japan. However, the degree of selection bias cannot be measured unless the characteristics of the survey respondents can be compared with those of

the subjects who did not respond to the questionnaire.

Variation in the respondents' answers will be discussed by the respondents' age and area of specialty in another

paper.

Information Sources and EBM Activities in the

previous surveySome previous by conducted surveys found a fre-

quent usage of traditional (non-EBM) information sources and a low level of awareness and usage of

EBM sources. A questionnaire survey conducted in Canada, 294 participating internist reported that they

are using EBM in their clinical practice always (11 %), often (59%), sometimes (27%), or rarely/never (3%).

Popular information sources they reported were clinical experience (93%), review articles (73%), the opinion

of the colleagues (61%), and textbooks (45%)"A UK survey of general practitioners showed that 40

% respondents knew the Cochrane Library, but, many

did not use12. In Australia, a cross-sectional postal sur-

vey of general practitioners showed 14% were •gon

line•h at their workplace, 22% were aware of the Co-

chrane Library, although only 6% had access of it and

4% had ever used it13. In Japan, Kassai conducted a

questionnaire survey of which subjects were members


of the Japanese Academy of Primary Care Physicians

(JAPCP) and reported as follows: the top 4 popular in-formation sources reported by respondents are Konnichi no Chiryo Shishin (a compendium of experts' sugges-

tions) (44.5% respondents used at least once a week),

journals (36.4%), discussion with colleagues (31.8%), and textbooks (27.9%). EBM information source, both MEDLINE and Japanese translation of Clinical Evi-dence were not frequently used (11.0% and 9.9%, re-

spectively)14

Interpretation of Findings

Farquhar et al. reviewed 30 studies about attitudes to clinical practice guidelines. They reported 70% of clini-

cians agreed that guidelines were helpful sources of ad-vice, good educational tools and intended to improve

quality15. In the present survey, approximately half of all surveyed CDs and HDs referred to clinical practice

guidelines in a number of ways that suited their indi-vidual purposes. The rates of referring to clinical prac-

tice by physicians seem to be rather lower than those in the Farquhar's findings, however, almost all respon-

dents replied that clinical practice guidelines are useful for making clinical decisions. Although the survey did not investigate how the guidelines were used by each

respondent, it is quite possible that many of the respon-dents merely used the guidelines to refer, for example,

to the strength of recommendation concerning the treat-ment of hypertension and hypercholesterolemia. Never-theless, as described below, many respondents were in-

terested not only in the strength of the recommendation for therapy but also in the rationale for such recommen-

dation. Therefore, in addition to treatment recommenda-

tions, clinical practice guidelines should also include summaries of the information on which each recom-

mendation is based, the bibliographic information for the cited articles and the abstracts of the literature cited in the guidelines. In fact, the majority of HDs in the

present survey indicated that they would like to know the contents of the cited articles if possible. HDs'

eagerness to know the contents of the article cited in

the guidelines can be attributed to the fact that the ma-

jority of HDs in the present survey are specialist doc-tors and they are therefore very much interested in the

matters concerning the area of their specialty. In fact, over 90% of all respondents indicated that they would

refer to the full text of the articles cited in the guide-lines provided that they were readily available to the

guideline users. However, as shown in Table 2, only 10.8% of CDs and 49.7% of HDs indicated to have

ever carried out a literature search to obtain necessary

information. This is because literature search and re-

view is laborious, which is especially true when Internet

is not available at work.

Because CDs are generally older than HDs, the per-

centage of internet users is lower among CDs than

among HDs. The governmental report showed that 54.5

%in 2002 and 60.6% in 2003 among all Japanese peo-

ple can access to internet16. Considering age of the re-spondents, physicians' Internet accessibility is almost

the same as those of the nation average. In any case,

there is an issue of linguistic barrier to be concerned

about, for many of the articles cited in the guidelines

may be written in English.

Among respondents, the most common reason why

they used medical information was •gto make decisions

about the care of individual patients•h. Although most

physicians in Japan tend to depend on textbooks and/or

relevant medical journals to obtain information neces-

sary to support their decision-making, there is a need

for more efficient ways of obtaining the necessary in-

formation.

Since many respondents indicated the need for medi-

cal information to support their clinical decision-

making, few of them referred to such information while

examining an outpatient. In fact, most respondents indi-

cated that they refer to medical information during the

interval between examinations or after hospital hours.

The use of secondary information resources available via Internet, such as clinical practice guidelines, allows

physicians to efficiently obtain a range of information necessary for clinical decision-making within a limited

time frame, and it is considerably more convenient than

undergoing the lengthy process of literature search, critical appraisal of the retrieved articles, and evidence synthesis. Thus, measures should be taken to ensure

that individuals working at medical institutions and pri-vate clinics have access to Internet information re-

sources. Most of the surveyed physicians indicated that they obtain evidence necessary for clinical decision-making not through literature search but through the

use of clinical practice guidelines, which include a se-

ries of recommendations made according to the evi-dence collected through literature search by specialists

in the relevant field(s). Today, researchers and experts in various medical fields are vigorously involved in de-

veloping evidence-based clinical practice guidelines. Awareness and understanding of EBM in Japan can be increased not only by encouraging physicians to use

and evaluate the existing clinical guidelines but also by


promoting the knowledge regarding EBM process of

decision-making.

On the other hand, although it is widely accepted to-

day that implementation of EBM in daily practice and

the use of clinical practice guidelines would be useful

in improving the quality of patient care in Japan, both

the concept of EBM and the roles of clinical practice

guidelines are often misunderstood in the clinical set-tings17.

Minds, which provides access to clinical practice

guidelines via the Internet, was released to the public in May, 2004. Increased accessibility to the guidelines will

promote a lively debate among medical practitioners and the general public on the issues surrounding the use

of clinical practice guidelines. We hope that, through

critical appraisal and constructive feedback from health-

care providers and consumers, Minds will not only de-

velop into a reliable information support system but

will also contribute to improving the quality of medical

practice in Japan.

References

1. Shannon S. Evidence-based programme design for CME. Lancet

2003; 361(9370):1752.

2. McColl A, Smith H, White P, Field J. General practitioners' per-

ceptions of the route to evidence based medicine: a question-

naire survey. BMJ 1998; 316(7128):361-365.

3. Jacobson LD, Edwards AGK, Granier SK, Butler CC. Evidence

based medicine and general practice. Br L Gen Pract 1997; 47:

449-452.

4. M.J. Field and K.N. Lohr (eds.) Clinical practice guidelines: Di-

rections for a new program. Washington, DC: National Academy

Press 2000; 38.

5. Grimshaw JM, Russell IT. Effect of clinical guidelines on medi-

cal practice: a systematic review of rigorous evaluations. Lancet

1993; 342(8883):1317-1322.

6. Koyama H, Fukui T. A review of research on the clinical effec-

tiveness of therapeutic practice guidelines, 1991-2000: the need

for standardization of reporting style. General Medicine 2002; 1:

1-7.

7. Fukui T, Tango T. The procedure of developing clinical practice

guidelines, version 4.3 (in Japanese), 2001.11.7 (http://minds.

jcqhc.or.jp/st/svc 115.aspx accessed June 18, 2004)8. Nakayama T, Fukuhara S, Kodanaka T. Contributions of clinical

epidemiologists and medical librarians to developing evidence-

based clinical practice guidelines in Japan: A case of the treat-

ment of rheumatoid arthritis. General Medicine 2003; 4(1):21-

28.

9. The committee for examination of health technology assessment. A report about need and future direction of health technology

assessment (in Japanese). Health Policy Bureau, Ministry of

Health and Welfare, Japan. July 27, 1997 (http://www 1.mhlw.

go.jp/houdou/0906/h 0627-6.htm1 accessed June 18, 2004)10. The committee for promoting health technology assessment. A

report about promotion of health technology assessment (in

Japanese). Health Policy Bureau, Ministry of Health and Wel-

fare, Japan. March 23 1999 (http://www 1.mhlw.go.jp/houdou/

1103/h 0323-1 _10.html accessed June 18, 2004)11. McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-based

medicine and the practicing clinician. J Gen Intern Med 1999

14: 236-242.

12. McColl A, Smith H, White P, Field J. General practitioner's per-

ceptions of the route to evidence based medicine: a question-

naire survey. BMJ 1998;316:361-365.

13. Young JM, Ward JE. General practitioners' use of evidence da-

tabases. Med J Aust 1999;170:56-58.

14. Kassai R. Verifying the effectiveness of tools for evidence-based

primary care in Japan. Primary Care Japan 2003;1(1):14-29.15. Farquhar CM, Kofa EW, Slutsky JR. Clinicians' attitudes to

clinical practice guidelines: a systematic review. Med J Aust.

2002 Nov 4;177(9):502-506.

16. Ministry of Public Management, Home Affairs, Posts and Tele-

communications. Communications Usage Trend Survey, 2003.

Tokyo (http://www.johotsusintokei.soumu.go.jp/tsusin _riyou/data/eng_ tsusin_riyou 02_2003.pdf accessed on July 1, 2004)

17. Nakayama T, Budgell B, Tsutani K. Confusion about the clinical

practice guidelines in Japan: on the way to a social consensus. Int J Qual Health Care 2003;15: 359-360

Appendix

Method of Weighing

1. Method Used to Assign Weights to Japan Medical Asso-

ciation (JMA) Members

The weights were assigned according to the following proce-

dure:

(1) The number of JEA members by age group (1) and the per-centage of (1) against the total number of JEA members are

shown below in columns (1) and (2). The number of responses obtained from JEA members by age group (3) and the percentage of (3) against the total number of responses obtained are shown in columns (3) and

(4). Three of the JEA members who did not provide any an-swer to the question concerning age were excluded from the survey.

(3)The response rate was higher among the subjects aged in their 20 s and 30 s than those aged 50 or above.

(4) Weights were calculated in order to make adjustments to account for imperfect response rate. The calculated weights are shown below in column (5).


(5) The number of responses obtained was multiplied by the calculated weights to obtain the adjusted sample size.

(6) When calculating the adjusted sample size, all fractions un-

der 0.5 were rounded down and all fractions over 0.5 were rounded up to the nearest whole unit; thus, the total value of the adjusted sample size may not equal the total number of

JEA members.

2. Method Used to Assign Weights to Physicians Working at Authorized Hospitals

Weights were assigned according to the following procedure:

ED Hospitals were grouped into five categories based on the number of full-time physicians (i.e. 1-9, 10-19, 20-49, 50-99 and 100 and above) (See Column (1) below).

(2) The number of full-time physicians by hospital size (2) and the percentage of (2) against the total number of full-time

physicians are shown below in columns (2) and (3) (based

on the data released by JCQHC).(3) The number of responses obtained from the physicians by

hospital size (4) and the percentage of (4) against the total number of responses obtained are shown in columns (4) and

(5). The response rate was higher among physicians working for smaller-size hospitals than among those working for larger-size hospitals.

(4) Weights were calculated in order to make adjustments to ac-

count for imperfect response rate. (5) The number of responses obtained was multiplied by the

calculated weights to obtain the adjusted sample size (See Column (7) below).

(6) When calculating the adjusted sample size, all fractions un-der 0.5 were rounded down and all fractions over 0.5 were

rounded up to the nearest whole unit; thus, the total value of the adjusted sample size may not equal the total number of full-time physicians.

physicians' awareness regarding evidence-based medicine

Documents