How we should approach the future

Download How we should approach the future

Post on 06-Jul-2016




2 download

Embed Size (px)


<ul><li><p>STATISTICS IN MEDICINE, VOL. 9, 1063-1068 (1990) </p><p>HOW WE SHOULD APPROACH THE FUTURE </p><p>GORDON D. MURRAY Medical Statistics Unit, Department of Surgery, Western Infirmary, Glasgow GI1 6NT, U.K. </p><p>SUMMARY There is little doubt that there are still major problems with the statistical aspects of published medical research. This paper argues that it is not sufficient to teach statistics to medical undergraduates, when the bulk of their courses are being taught by the very same individuals who continue to undertake and publish substandard research. Instead, any undergraduate teaching needs to advance in parallel with courses given to the clinical staff within medical schools. </p><p>INTRODUCTION </p><p>It is hardly novel to point out that there are major problems with the statistical aspects of published medical research. Twenty-five years ago Yates and Healy remarked that It is depressing to find how much good biological work is in danger of being wasted through incompetent and misleading analysis of numerical results, and there is little evidence that matters have improved substantially since then. There is perhaps now a wider recognition of the problem, and certain journals are taking a much stronger line on statistical issues (notably the British Medical Journal), but large numbers of review papers continue to typically revealing statistical flaws in around 50 per cent of published papers. It is difficult to assess the seriousness of these errors. In many cases even an inappropriate analysis will lead to the correct qualitative conclusions, but at the very least these errors must introduce serious biases into quantitative conclusions. It should also be remembered that this figure of 50 per cent will give an under- estimate of the true prevalence of substandard statistical methodology, because it has tended to be the leading journals, which adhere to strict clinical refereeing, which have been reviewed, and moreover, not all statistical errors are apparent on even a careful reading of a published report. This is especially the case when the statistical methods section is abbreviated to a single inane sentence along the lines of A p-value of -= 0.05 was chosen as the level of statistical confidence (reference withheld to protect the guilty!). </p><p>PROBLEMS IN TEACHING </p><p>What has become even more apparent over the last 25 years is that the problem of teaching statistics to medical students is a difficult one. It is equally apparent that a standard course of perhaps 10 to 20 lectures in the first undergraduate year, covering material from measures of central tendency through normality and t-tests to perhaps ANOVA and a touch of regression can be counter productive, and gives no feel for what medical statistics is actually about. The ideas </p><p>0277-67 15/90/09 1063-06$05.W 0 1990 by John Wiley &amp; Sons, Ltd. </p><p>Received February 1990 </p></li><li><p>1064 G. D. MURRAY </p><p>which we attempt to get over in an introductory course must surely be more fundamental-ideas of design, the types of question which can be answered by a clinical study (with an emphasis on estimation), the difference between an explanatory and a pragmatic study, etc. The actual mechanics of specific tests are trivial (and boring) against such a background, and can easily be filled in at a later stage when the students are motivated by having met genuine (pre-) clinical problems where they need to apply statistical methods. </p><p>Even if it were possible to fit a good methods course into the early part of an undergraduate medical course, there seems to be little point when it would run parallel to hundreds of lecture and practical hours taught by the very individuals who write, referee or edit the substandard papers which appear throughout the medical literature. I believe, given the meagre staff resources which are available for teaching medical statistics in most British medical schools, that the most efficient use of time over the short to medium term is to focus our efforts on running courses for active research workers and for the teaching staff. This ultimately is the only way in which statistical thinking can be incorporated and integrated into the entire medical syllabus, rather than appearing as a semi-optional disjoint subject which is skipped as soon as a crucial anatomy examination approaches. </p><p>There is certainly a demand for such courses. A general statistics course for research workers run recently by members of the Department of Statistics at Glasgow University attracted over 100 participants, and a Staff Development Course on medical statistics which I ran this year for the first time was fully subscribed (24 places), and will require to be run twice next term to cover those individuals already on the waiting list. I would like to believe that raising the statistical awareness of 100 members of the Universitys teaching staff will in the long run have a far more beneficial effect on the statistical education of our students than could be achieved by lecturing directly to our annual intake of 300 medical and dental undergraduates. </p><p>THE PROFILE OF MEDICAL STATISTICS </p><p>In some ways I view this Staff Development Course as a public relations exercise, that is, an attempt to raise the profile of statistics within the Medical School. In the long run I believe that this is essential if our speciality is ever to progress and attract the staff resources which are enjoyed by our American and many European colleagues. I was struck recently by a report in the Biometric Bulletin on the development of medical statistics in the Indian subcontinent. B. L. Verma writes </p><p>By contrast with the developed world, staffing is utterly inadequate; . . . medical schools do not have separate departments of biostatistics; teachers of biostatistics do not enjoy the same status as their counterparts in other medical specialities;. . ~ wrong or imprecise use of biostatistics is rife in medical research publications; . . . Biostatistics must be recognised as an independent discipline requiring its own department within every medical school, equipped with senior faculty positions and modern data-analytic facilities. Biostatistics must be involved in research projects from the outset (not at the end), and refresher courses are needed for biostatisticians to keep abreast of current applications. </p><p>While I would agree very much with these sentiments, I suspect that by these criteria, many British medical schools would need to be regarded as developing rather than developed. </p><p>Indeed I think that it is essential not only to raise the profile of statistics within medical schools, but also to raise the profile of medical statistics within the (academic) statistics profession. Again </p></li><li><p>HOW WE SHOULD APPROACH THE FUTURE 1065 </p><p>turning to the writings of Healy; in a paper discussing whether a medical statistician should be viewed as a scientist or a technician, he writes </p><p>If this is being a mere technician, I for one find it nothing to be apologetic about. And if I am concerned with assessing a fellow-statistician for employment or promotion and find that the bulk of his publications are collaborative and have appeared outside the purely statistical journals, I shall not feel like asking him (as I was once asked): When are you going to get down to your own work?. </p><p>I suspect that few members of this workshop would argue against these views, but equally I suspect that there are few members who have not at some stage of their careers been conscious of being assessed by senior statisticians with a less enlightened attitude. </p><p>Looking at my own publications, it is certainly not those with the most high powered statistical analyses of which I feel most proud. It is far more satisfying to have been involved in a research project from the outset, to have bounced statistical ideas off receptive clinical colleagues, and to have seen the whole enterprise through to becoming the umpteenth author in a report in a leading medical journal, with perhaps nothing more exciting by way of analysis than a well- chosen t-test. </p><p>To make the issue less personal, I would give as an example a recent Lancet paper with which I have no association, but which I regard as being an excellent example of the medical statisticians art.7 The paper is by the Canadian Collaborative CVS-Amniocentesis Clinical Trial Group, and describes a randomized trial of chorionic villus sampling (CVS) at 9-12 weeks gestation against amniocentesis at 15-17 weeks for the detection of a chromosomal abnormality in the foetus, involving 2787 women who were aged 35 years or over at the expected date of delivery. Superficially the statistics are rather uninteresting, with a simple power calculation and a couple of 2 x 2 tables leading to confidence intervals comparing two proportions. There was not even a suggestion of a multivariate regression analysis to correct for baseline differences, and so surely this study would have a lowly standing in the academic statistics stakes. However, a careful reading of the report shows that an enormous degree of effort has gone into the wider statistical aspects of the trial. For example, CVS is a relatively new technique, and care was taken to ensure that all participants had adequate experience in its use, so that there was no learning curve effect during the trial. (Some ardent supporters of randomize from the first case might criticize this approach, but at least the problem is recognized and addressed.) Then a substantial proportion of the women were found to be ineligible after they had been randomized, and so the difficult question of how to handle these potential exclusions was addressed. Another difficult question was what to take as the primary endpoint, and, in particular, whether to count only spontaneous foetal losses or to include planned terminations as well, as part of the cost of the screening method. Finally there was the wording of the conclusions. The results showed no statistically significant difference in total foetal losses in the two groups, but avoiding the classic pitfall of deducing that there was no difference between the two approaches, the authors used the confidence interval comparing the proportions of foetal losses in the two groups to conclude that it was most unlikely that any excess losses associated with CVS were greater than 2-7 per cent. This surely is precisely the information which a woman would want to know when attempting to balance the obvious attractions of CVS against the potential worry of risk to the foetus. Clearly one could quibble with some of the details of the study, as one could with any multicentre randomized trial which has ever been conducted, but the authors have revealed a deep understanding of what medical statistics is actually about; achieving a fine balance between statistical niceties and the pragmatics of clinical medicine. </p></li><li><p>1066 G. D. MURRAY </p><p>STAFF RESOURCES </p><p>This again raises the question of staff resources. To be able to collaborate fully with clinical colleagues, a statistician needs to win their trust and respect, and an important prerequisite for this is to have a grounding in the clinical background to the problem. One requires to have sufficient clinical common sense to be sure that ones advice is relevant and realistic. In my experience, the most successful way to achieve this is to align oneself as strongly with the clinicians as with academic statisticians, attending clinical meetings and conferences, reading specialist medical journals, and, when the opportunity arises, serving on the editorial boards of medical journals. All of these activities are time consuming, and are perhaps not an efficient use of ones time. Certainly there are many 80/20 situations where one can achieve 80 per cent of the likely returns by expending only 20 per cent of a full effort. We are all familiar with reports (published or otherwise) where after 10 minutes of reading it is clear that there is a good study fighting to get out, but the report is marred by a simple statistical flaw which could easily be corrected. However, to limit oneself to such situations is to undersell our discipline. </p><p>I prefer to make a 100 per cent effort, which inevitably limits the number of projects which one can cover. This is certainly not done as a deliberate tactic, but it is maybe still a good strategy for medical statisticians to aim to give an excellent service in a limited number of areas rather than to overstretch and give an indifferent service to all comers. The 100 per cent approach certainly gives the greater opportunity of demonstrating the invaluable contribution which a statistician can make to a research programme, and also emphasizes the scarcity of medical statisticians. </p><p>THE ROLE OF COMPUTERS IN TEACHING </p><p>A further point to consider as we look towards the future must be the role of computers. Indeed, in many respects one could argue that for statistical computing the future has already arrived! For well under one thousand pounds one can obtain a modest but adequate personal computer, complete with a database package and a statistical package. However, such technological advances are clearly two-edged. Used sensibly, they obviate tedious and error-prone manual calculation, and encourage the user to plot their data and hence to think critically about assumptions and the interpretation of their findings. Equally they provide more than enough rope to hang the unwary. As I write this paper, I come from having just refereed a paper based around a proportional hazards regression analysis looking at predictors of local recurrence following surgical excision of primary colorectal cancer. The data came from a retrospective review of case records from eight hospitals, and the second most significant predictor was hospital, arranged alphabetically and treated as a continuous variable ranging from 1 to 8. </p><p>I frequently cringe to open a popular weekly medical journal to find advertisements for statistical packages offering features such as ANOVA, ANCOVA, MANOVA, principal com- ponents, multi-dimensional scaling, multiple and canonical discriminant analysis, cluster and time series analysis . . ., but taking a head-in-the-sand attitude does nothing to prevent the nonsenses which result when these powerful tools are used by individuals with inadequate training. Certainly such facilities are becoming increasingly available, and friendly front ends will make it even simpler for individuals with little knowledge of statistics or computing to analyse their data. I believe that we need to take the initiative here, and mount courses which demonstrate the appropriate use of these dangerous but invaluable tools. </p><p>In the Glasgow Medical School we have installed a cluster of IBM PS/2 microcomputers, and we are in the process of installing a second cluster. These machines have many uses, but inter uliu, they have meant that the Staff Development Course in medical statistics could be supported by </p></li><li><p>HOW WE SHOULD APPROACH THE FUTURE 106...</p></li></ul>


View more >