ARTICLE IN PRESS
1356-689X/$ -
doi:10.1016/j.m
Manual Therapy 12 (2007) 1–2
www.elsevier.com/locate/math
Editorial
Dealing with heterogeneity in clinical trials
Clinical trials would be a simpler enterprise if patientspresented with homogenous clinical presentations towhich we could assign simple diagnoses, if therapistsuniformly applied standardised therapies using mechan-istic decision rules based on objective and universallyaccepted criteria, and if patients responded in more orless uniform ways to intervention. Unfortunately, theclinical presentations for each diagnosis are varied,diagnosis can be difficult, therapists choose to intervenevery differently for the same condition or presentation,and patients’ outcomes often appear hard to predict.For almost any clinical problem, clinical presentations,diagnoses, interventions and outcomes are heteroge-nous. This makes clinical trials difficult. Clinicalresearch is a messy business.
One way to deal with heterogeneity is to attempt tominimise it. For example a trial could recruit fromhomogeneous populations by defining stringent inclu-sion and exclusion criteria. The process of clinicaldecision-making could be tightly constrained by requir-ing that the experimental intervention is always admi-nistered in a particular way, or by defining precisealgorithms for decisions about intervention. Trials withnarrowly defined populations and tightly constrainedinterventions are sometimes called ‘‘explanatory’’ clin-ical trials (Schwartz and Lellouch, 1967; McMahon,2002; Herbert et al., 2005). Typically this approachmaximises the effects of intervention and reducesvariability of outcomes. So the explanatory approachis often preferred by researchers who are intent onproving the efficacy of intervention.
Alternatively, a trial might recruit from the diversepopulations for whom therapy is usually provided in thecourse of normal clinical practice. Such populations willnot usually be those in whom the intervention is mosteffective because therapists often offer intervention evenwhen they are not optimistic of success. Therapists couldbe given freedom in exactly how they provide theexperimental intervention, and they might be allowed tocustomise the intervention to particular needs ofindividual patients. These ‘‘pragmatic’’ trials reflect theway intervention is administered in the course of normalclinical practice. By giving in to heterogeneity, prag-
see front matter r 2006 Elsevier Ltd. All rights reserved.
ath.2006.11.001
matic trials may be less likely to find large effects ofinterventions, but they have the advantage of telling usabout the real-world effects of intervention.
The primary purpose of clinical trials is to estimate‘‘the effect’’ of intervention. But interventions do nothave one effect on all patients; typically they are veryeffective for some patients and ineffective or evenharmful for others. In other words, effects of interven-tion are heterogenous. Unfortunately, clinical trialscannot provide unbiased estimates of the effects ofintervention on each participant in a trial. Alternativemethods, such as single case experiments (also calledn-of-1 designs) may permit conclusions to be drawnabout effects of interventions on individuals (Barlowand Hersen, 1984), but these methods do not provide abasis for robust inference about effects of interventionson individuals other than those who participated in thestudy. Ultimately neither clinical trials nor single caseexperiments can provide what we most want: thecapacity to make specific predictions about what theeffect of intervention will be on our next patient.
The best we can hope for from most clinical trials isan unbiased estimate of the average effect of theexperimental intervention in a population. This hasbeen a frequent source of criticism. Some have arguedthat by focusing on averages we ignore the heterogeneityof effects of interventions. After all, it is argued, we treatindividual patients, not average patients. Of course thatis true, but it fails to recognise why we might beinterested in the average effect of an intervention: in theabsence of better information about how individuals willrespond, the average effect of intervention provides uswith a ‘‘best guess’’ of what the effect of interventionwill be on any individual (Herbert, 2000). Clinical trialscannot give us specific estimates of the effects of anintervention on our next patient but they can help usmake unbiased guesses about effects of intervention thatcan form an appropriate basis for clinical decisionmaking.
That is not to say we should not try to identify thosepatients most likely and least likely to benefit from anintervention. One of the next big challenges for clinicalresearchers investigating manual therapies is to identify
ARTICLE IN PRESSEditorial / Manual Therapy 12 (2007) 1–22
characteristics of people who respond to therapy.(Sometimes such characteristics are called ‘‘effectmodifiers’’, and in the context of clinical trials identifi-cation of responders is sometimes referred to as‘‘subgroup analysis’’.) Recently there has been a flurryof interest in development of new taxonomies of lowback pain (O’Sullivan, 2006) driven partly by a desire tobe able to target interventions at people who will benefitmost from intervention. Manual therapy researchershave begun to think about the best methodologies foridentifying effect modifiers (Beattie and Nelson, 2006).And the first well designed studies have begun toidentify which patients respond well to manual therapyinterventions (Childs et al., 2004).
Unfortunately, identification of effect modifiers is amethodologically hazardous undertaking. A simple andcommon mistake is to confuse prognostic factors(predictors of outcomes) with effect modifiers (predic-tors of response to therapy). Prognostic factors can beidentified using cohort studies, but effect modifiers canonly be identified with controlled clinical trials. Rigor-ous identification of effect modifiers involves contrastingeffects of interventions across subgroups in randomisedtrials.
The perils of naı̈ve analyses have been widelydiscussed (Yusuf et al., 1991) and extensively analysed(Brookes et al., 2004). The message from this literatureis that robust identification of effect modifiers can onlybe carried out within the context of a randomised trial.Identification of effect modifiers must involve priorspecification of a small number of specific hypothesesrather than undisciplined dredging of numerous hy-potheses. Analysis must involve examination of themagnitude of the interaction between patient character-istics and intervention (Brookes et al., 2004). Aconsequence of the need to examine interactions is thatsample size requirements are quadrupled (Brookes et al.,2004). Rothman and Greenland (1998) have pointed outthat particular care must be taken in defining what ismeant by an interaction, because the magnitude of anyinteraction will depend on how the effect of interventionis measured. (For example, an interaction observedwhen the effect of an intervention is measured as anabsolute risk reduction may evaporate when the effect isre-expressed as a relative risk.) A consequence is that
certain patient characteristics may appear to predicteffects of intervention when effects are measured withone metric, but not when effects are measured withanother.
Heterogeneity is a universal feature of clinical practicethat presents a challenge for clinical trialists. Carefulconsideration of sampling, intervention and analysisshould make it possible to design trials which cansupport real-world clinical decision-making.
References
Barlow DH, Hersen M. Single case experimental designs: strategies for
studying behavior change. Boston: Allyn and Bacon; 1984.
Beattie P, Nelson R. Clinical prediction rules: what are they and what
do they tell us? Australian Journal of Physiotherapy
2006;52(3):157–63.
Brookes ST, Whitely E, Egger M, Smith GD, Mulheran PA, Peters TJ.
Subgroup analyses in randomized trials: risks of subgroup-specific
analyses; power and sample size for the interaction test. Journal of
Clinical Epidemiology 2004;57(3):229–36.
Childs JD, Fritz JM, Flynn TW, Irrgang JJ, Johnson KK, Majkowski
GR, et al. A clinical prediction rule to identify patients with low
back pain most likely to benefit from spinal manipulation: a
validation study. Annals of Internal Medicine 2004;141(12):920–8.
Herbert RD. How to estimate treatment effects from reports of clinical
trials. I: continuous outcomes. Australian Journal of Physiotherapy
2000;46(3):229–35.
Herbert RD, Jamtvedt G, Mead J, Hagen KB. Practical evidence-
based physiotherapy. Oxford: Elsevier; 2005.
McMahon AD. Study control, violators, inclusion criteria and defining
explanatory and pragmatic trials. Statistics in Medicine
2002;21(10):1365–76.
O’Sullivan P. Classification of lumbopelvic pain disorders—why is it
essential for management? Manual Therapy 2006;11(3):169–70.
Rothman KJ, Greenland S. Modern epidemiology. Philadelphia, PA:
Lippincott-Raven; 1998.
Schwartz D, Lellouch J. Explanatory and therapeutical attitudes in
therapeutical trials. Journal of Chronic Diseases 1967;20:
637–48.
Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpreta-
tion of treatment effects in subgroups of patients in randomized
clinical trials. Journal of the American Medical Association
1991;266(1):93–8.
Rob HerbertSchool of Physiotherapy, University of Sydney, Sydney,
Australia
E-mail address: [email protected]
