Journal of Small Animal Practice • Vol 53 • April 2012 • © 2012 British Small Animal Veterinary Association 195

EDITORIAL

Systematic reviews in small animal veterinary medicine: what are they and why do we need them?This issue of JSAP contains a systematic review looking at the effect of neutering on the risk of urinary incontinence in bitches (Beauvais and others 2012). Systematic reviews are relatively uncommon in the veterinary literature and yet they are tremen-dously influential in human medicine. What exactly is a sys-tematic review, and why would we like to see more of these in veterinary journals?

Traditional narrative reviews provide an overview of literature in a particular area and are often written by someone with expe-rience, or particular expertise, in the subject being reviewed. They often contain a useful up-to-date summary of clinical research in that area combined with an expert’s opinion on what he or she believes is important or relevant. No two authors might be expected to come to the same conclusions as each would be likely to put an emphasis on aspects reflecting their own beliefs. A systematic review on the other hand attempts to provide an objective summary of the published literature on a particular topic. Rather than requiring specific knowledge of the topic being reviewed the authors require expertise and skills in finding the literature, and its appraisal. In a systematic review the review is undertaken using the same scientific process, or method, that is normally used to perform the research in the first place. Before the authors describe the results of their review they describe the actual methods that they used. They describe where and how they looked for the papers they reviewed. They describe what criteria were used to select and omit papers. They say how data was extracted, how it was assessed and how it was analysed. In a systematic review (just like in a report of a scien-tific study) the materials and methods section should contain sufficient information to enable any suitably educated reader to reproduce the analysis and (hopefully) obtain the same or a similar result. There remains room for a degree of subjectivity in the discussion of the results of the systematic review and, as in any scientific study, errors may be made in the methods used or the way they are applied. In a clinical trial we can bias the results through poor patient inclusion or exclusion criteria; in a systematic review we can bias the outcome through including poor studies in the analysis and omitting good studies. System-atic reviews should be subject to rigorous peer review, a pro-cess that should prevent the publication of poorly designed or conducted work. The discussion in a systematic review should consider the strengths and weaknesses of the review and place the results in the context of the current understanding of the subject area.

The value in systematic reviews is in their relative transpar-ency. The reader can see how the results were obtained and make a judgement about the quality of the process which generated the results. Any reader who questioned the thoroughness of the literature search could compare their own results with those of the authors and identify wrongly included or excluded literature. In other words, the reader can reach their own conclusion about

the quality of the systematic review through an appraisal of the way the review was performed.

A clinical study which involves small numbers of patients will often be underpowered. An underpowered study may fail to detect the effect being measured simply due to the random variation in the thing being measured, and it is why most clini-cal researchers would perform a sample size estimation before embarking on the study. Similarly systematic reviews may fail to identify the true ‘effect’ or clinical ‘truth’, even when there is one, due to the paucity of clinical studies available in the literature. A systematic reviewer has no control over this, and when there is insufficient data to work from can only report this. This brings us on to a particularly useful feature of systematic reviews in vet-erinary medicine which is that they clearly identify areas where further clinical research is needed. Thus when Beauvais and oth-ers (2012), conclude that “the evidence is not consistent or strong enough to make recommendations about the age of neutering and its effect of urinary incontinence”, they are highlighting the need for more research. Potential authors (and journal ref-erees) of systematic reviews should be aware that the failure to write or publish systematic reviews that have inconclusive results robs us of this information. Systematic reviews provide objective identification of research deficits and can help guide researchers and research funders to identify priorities. The use of systematic review in human medicine has clearly identified how the failure to publish negative findings from clinical studies leads to publica-tion bias. Authors and/or publishers are sometimes reluctant to publish clinical research that fails to identify a statistically signifi-cant effect. However this research may have been well-designed and be able to help identify the true magnitude of the effect when included in a systematic review. Another hindrance to our being able to conduct systematic reviews is the unwillingness to repeat studies that have already been published. It is essential to appre-ciate that no one clinical study can produce a definitive result. Each time a study is performed it increases our confidence in the precision and generalizability of the result. The experience in human medicine is that the collation of results from many similar studies into systematic reviews has had a considerable influence on changing clinical practice. However, in order for us to be able to perform systematic reviews we need to have the results from these similar studies. So next time you are thinking of doing some clinical research please don’t let the fact that it has been done once before put you off; and when reviewing a paper don’t let the fact that a study is similar to one performed before lead you to reject it.

The generation of systematic reviews within a discipline dem-onstrates its scientific maturity. The ability to reflect upon the research that informs clinical practice using a well-defined meth-odology (i.e. using a ‘systematic’ approach) can only be a posi-tive development. There are challenges in undertaking systematic reviews in veterinary medicine and there may be a temptation to

http

://w

ww

.w

ww

.b

sa

va

.co

msa

va

.co

msa

va

.co

msa

va

.co

m/

Editorial

196 Journal of Small Animal Practice • Vol 53 • April 2012 • © 2012 British Small Animal Veterinary Association

desire of clinicians to turn the results of clinical research into improved patient care.

ReferenceBEAUVAIS, W., CARDWELL, J. M. & BRODBELT, D. C. (2012) The effect of neutering on

the risk of urinary incontinence in bitches – a systematic review. Journal of Small Animal Practice 53, 198-204

Dr Mark A. HolmesUniversity of Cambridge

arrive at unjustified conclusions when the studies being included are too heterogeneous or use poor study designs. Some of the methodology developed for analysis of human clinical trials may be inappropriate for the analysis of animal studies. The accumu-lation of veterinary expertise in systematic review will increase over time and the methodology will evolve. The success of the Cochrane Collaboration (a collection of consortia in different disciplines) in human medicine should serve as an example to the veterinary profession. Not only because it provides a valuable information resource, but also because it arose from the need and

Mark Holmes is a Senior Lecturer in Preventive Veterinary Medicine at the University of Cambridge. His research interests include veterinary immunology, epidemiology and more recently the discovery, in his laboratory, of a new MRSA in animals and people. He is an author of books on evidence based veterinary medicine and veterinary clinical research. He runs a clinical research outreach program for veterinary clinicians providing training for practitioners interested in performing clinical research. He is a former Honorary Secretary of the BSAVA and has been a regular speaker at BSAVA congress promoting practice-based clinical research and evidence based medicine.

jsap_1194.indd 196jsap_1194.indd 196 09/03/12 2:48 PM09/03/12 2:48 PM