meta-analysis: sports medicine canadian academy of sport medicine lacadémie canadienne de médecine...

Meta-Analysis: Sports MedicineMeta-Analysis: Sports Medicine

Canadian Academy of Sport MedicineCanadian Academy of Sport MedicineL’Académie Canadienne de Médecine du SportL’Académie Canadienne de Médecine du Sport

Ian ShrierIan Shrier MD, PhD, Dip Sport Med, FACSMMD, PhD, Dip Sport Med, FACSM

Centre for Clinical Epidemiology and Community Studies, SMBD-Centre for Clinical Epidemiology and Community Studies, SMBD-Jewish General Hospital and McGill UniversityJewish General Hospital and McGill UniversityPast-president, Canadian Academy of Sport MedicinePast-president, Canadian Academy of Sport Medicine

OBJECTIVESOBJECTIVES

• How do we think?How do we think?

• A Workshop Example: Does Stretching A Workshop Example: Does Stretching Prevent Injury?Prevent Injury?

• What parameter are you estimating?What parameter are you estimating?

• RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses

• It’s all about Bias!It’s all about Bias!

INTERPRETATIONSINTERPRETATIONS

““It’s a rather interesting It’s a rather interesting phenomenon. Every time I phenomenon. Every time I

press this lever, the press this lever, the graduate student breathes graduate student breathes

a sigh of relief”a sigh of relief”


• Shrier, Platt, Steele. Mega-trials vs. meta-analysis: Precision vs. heterogeneity? Contemp Clin Trials 2007

• Shrier et al. Should Meta-Analyses of Interventions Include Observational Studies in Addition to Randomized Controlled Trials? A Critical Examination of Underlying Principles. Am J Epi 2007

• Shrier et al. The interpretation of systematic reviews with meta-analyses: an objective or subjective process? BMC Med Inform Dec Making 2008


I believe magnesium has now been shown to be beneficial for I believe magnesium has now been shown to be beneficial for patients during the post-MI periodpatients during the post-MI period (SD-SA) (SD-SA)

Rev 8Rev 8

Rev 7Rev 7

Rev 6Rev 6

Rev 5Rev 5

Rev 4Rev 4

Rev 3Rev 3

Rev 2Rev 2

Ag

DA

Ag

Ag

DA

DA

SA

Ag

DA

-

Ag

Ag

DA

-

SA

Ag

SD

-

Ag

Ag

DA

-

Ag

AgRev 1Rev 1

II22

Rand. ORRand. OR

Fixed ORFixed OR

Ag

DA

Ag

Ag

SD

SD

Ag

59%59%

0.75 (0.61-0.92)0.75 (0.61-0.92)

1.01 (0.96-1.07)1.01 (0.96-1.07)

69,50569,505

61%61%

0.65 (0.48-0.87)0.65 (0.48-0.87)

1.02 (0.96-1.08)1.02 (0.96-1.08)

63,04763,047

-

DA

Ag

Ag

Ag

-

SA

Ag

14%14%

0.66 (0.53-0.81)0.66 (0.53-0.81)

0.64 (0.52-0.79)0.64 (0.52-0.79)

3,6853,685

0%0%

0.38 (0.21-0.66)0.38 (0.21-0.66)

0.40 (0.28-0.61)0.40 (0.28-0.61)

597597

0%0%

0.40 (0.18-0.86)0.40 (0.18-0.86)

0.40 (0.19-0.83)0.40 (0.19-0.83)

415415NN

1-231-231-201-201-101-101-51-51-31-3# RCTs# RCTs

HOW DO WE THINK?

* * * * *

Clue: wantClue: want crave, covet, yearn, fancycrave, covet, yearn, fancy

(Vandenbroucke et al, 2001)(Vandenbroucke et al, 2001)

HOW DO WE THINK?

J U G

U

I

C * * * *

E



HOW DO WE THINK?

J U G

U

I

C * * * *

E H I S

QQuueeuueess



HOW DO WE THINK?

J U G L

U I

I N

C * * * E

E H I S

QQuueeuueess



HOW DO WE THINK?

J U G L

U I

I N

C R A V E

E H I S

L

E

QQuueeuueess



Does Stretching Prevent Injury?

(adapted from Shrier, Evidence-Based Sports Medicine 2007)(adapted from Shrier, Evidence-Based Sports Medicine 2007)

AnalysisAnalysisAnalysisAnalysis

YesYes No No

I will now tell my patients to I will now tell my patients to

stretch to prevent injurystretch to prevent injury

YesYes No No

I will now tell my patients not to I will now tell my patients not to


YesYes No No

I will now tell my patients that I have no idea I will now tell my patients that I have no idea

whether they should stretch to prevent injurywhether they should stretch to prevent injury


Review: StretchingComparison: 01 Injuries Outcome: 04 Stretching and Injury All Studies

Study OR or RR or HR (random) OR or RR or HR (random)or sub-category log[OR or RR or HR] (SE) 95% CI 95% CI Year Quality

01 MenAmako (2003) -0.3011 (0.1976) 0.74 [0.50, 1.09] 2003 DBixler (52) Not estimable 1992 DCross (1999) -0.7134 (0.2372) 0.49 [0.31, 0.78] 1999 DEkstrand (1983a) -1.7047 (0.3536) 0.18 [0.09, 0.36] 1983 DEkstrand (1983b) Not estimable 1983 DHartig (1999) -0.5551 (0.2180) 0.57 [0.37, 0.88] 1999 DHilyer (1990) -0.1985 (0.1654) 0.82 [0.59, 1.13] 1990 DMacera (1989) 0.0953 (0.1582) 1.10 [0.81, 1.50] 1989 DMcKay (2001) -0.9637 (0.4509) 0.38 [0.16, 0.92] 2001 DPope (1998) -0.0834 (0.2855) 0.92 [0.53, 1.61] 1998 DPope (2000) 0.0392 (0.1255) 1.04 [0.81, 1.33] 2000 DWalter (1989 never) 0.1398 (0.2823) 1.15 [0.66, 2.00] 1989 DWalter (1989 some) -0.4463 (0.1796) 0.64 [0.45, 0.91] 1989 DWalter (1989 usual) 0.2231 (0.1917) 1.25 [0.86, 1.82] 1989 Dvan Mechelin (1993) 0.2311 (0.2844) 1.26 [0.72, 2.20] 1993 D

Subtotal (95% CI) 0.77 [0.61, 0.96]Test for heterogeneity: Chi? = 46.20, df = 12 (P < 0.00001), I? = 74.0%Test for overall effect: Z = 2.27 (P = 0.02)

02 WomenMacera (1989) 0.4700 (0.3994) 1.60 [0.73, 3.50] 1989 DWalter (1989 never) 0.1655 (0.5831) 1.18 [0.38, 3.70] 1989 DWalter (1989 some) -0.5798 (0.3445) 0.56 [0.29, 1.10] 1989 DWalter (1989 usual) 0.0488 (0.3536) 1.05 [0.53, 2.10] 1989 D

Subtotal (95% CI) 0.98 [0.61, 1.57]Test for heterogeneity: Chi? = 4.27, df = 3 (P = 0.23), I? = 29.8%Test for overall effect: Z = 0.09 (P = 0.93)

Total (95% CI) 0.80 [0.65, 0.98]Test for heterogeneity: Chi? = 50.92, df = 16 (P < 0.0001), I? = 68.6%Test for overall effect: Z = 2.15 (P = 0.03)

0.1 0.2 0.5 1 2 5 10

Favour Stretching Favour No Stretching

Does Stretching Prevent Injury?Does Stretching Prevent Injury?



YesYes No No

I will now tell my patients to I will now tell my patients to


YesYes No No

I will now tell my patients not to I will now tell my patients not to


YesYes No No

I will now tell my patients that I have no idea I will now tell my patients that I have no idea

whether they should stretch to prevent injurywhether they should stretch to prevent injury


70% 80% 90% 100% 110% 120% 130%

% Unstretched Condition% Unstretched Condition

Acute Stretching: Force (MVC/1RM)Acute Stretching: Force (MVC/1RM)

(adapted from Shrier, Clin J Sport Med 2004)(adapted from Shrier, Clin J Sport Med 2004)

70% 80% 90% 100% 110% 120% 130%

Hortobagyi (38): Knee extension

Handel (29): Knee

Wilson (34): Bench Press

Dintiman (35): Knee extensionSprint & Weight Training

Sprint Training

Pure Concentric Bench Press

Rebound Bench Press

Extension

Flexion


Regular Stretching: Force (MVC/1RM)Regular Stretching: Force (MVC/1RM)

PNFPNF

StaticStatic


Acute Stretching: Force (Isokinetic)Acute Stretching: Force (Isokinetic)

80% 85% 90% 95% 100% 105% 110% 115% 120%


Slow (30-60 deg/s)Slow (30-60 deg/s)

Fast (>180 deg/s)Fast (>180 deg/s)


Regular Stretching: Force (Isokinetic)Regular Stretching: Force (Isokinetic)

80% 85% 90% 95% 100% 105% 110% 115% 120%

Knee flexion

Extension

Knee extension

Flexion


Slow (30-60 deg/s)Slow (30-60 deg/s)

Fast (>180 deg/s)Fast (>180 deg/s)


90% 95% 100% 105% 110%


Acute Stretching: Jump HeightAcute Stretching: Jump Height

StaticStatic

CMJCMJ


90% 95% 100% 105% 110%

HunterStretch

Pow er & Stretch


Regular Stretching: Jump HeightRegular Stretching: Jump Height

StaticStatic

CMJCMJ


Stretching and Force

• Overstretching occurs with as little as 20% stretchOverstretching occurs with as little as 20% stretch

• ProtocolProtocol Skinned bullfrog muscle fibers stretched and released Skinned bullfrog muscle fibers stretched and released

to different lengthsto different lengths

(adapted from Higuchi et al, J Mus Res Cell Motil 1988)(adapted from Higuchi et al, J Mus Res Cell Motil 1988)

• ResultsResults

Regular Stretching: Force

• ProtocolProtocol Weights attached to left wing of Japanese Quail x 30 Weights attached to left wing of Japanese Quail x 30

daysdays Animals killed and ant. lat. dorsi. placed in vitroAnimals killed and ant. lat. dorsi. placed in vitro

(adapted from Alway, J Appl Physiol 1984)(adapted from Alway, J Appl Physiol 1984)

Con StretchMass 25.7 66.8Force 55.0 120.0Velocity 2.7 4.1

• ResultsResults

Study OR or RR or HR (random) Weight OR or RR or HR (random) or sub-category 95% CI % 95% CI Year

Hilyer (68) 43.93 0.82 [0.59, 1.13] 1990 Hartig (67) 25.29 0.57 [0.37, 0.88] 1999 Am ako (6) 30.78 0.74 [0.50, 1.09] 2003

Total (95% CI) 100.00 0.73 [0.59, 0.90] Test for heterogeneity: Chi? = 1.71, df = 2 (P = 0.42), I? = 0% Test for overall effect: Z = 2.92 (P = 0.003)

0.1 0.2 0.5 1 2 5 10 Favour Stretching Favour No Stretching

Regular Stretching: Injury


Study OR or RR or HR (random) Weight OR or RR or HR (random) or sub-category 95% CI % 95% CI Year Men Ekstrand (52) 5.52 0.18 [0.09, 0.36] 1983 Ekstrand (64) Not estimable 1983 M acera (58) 8.98 1.10 [0.81, 1.50] 1989 W alter (59 never) 6.69 1.15 [0.66, 2.00] 1989 W alter (59 som e) 8.59 0.64 [0.45, 0.91] 1989 W alter (59 usual) 8.36 1.25 [0.86, 1.82] 1989 Bixler (51) Not estimable 1992 van M echelin (61) 6.65 1.26 [0.72, 2.20] 1993 Pope (63) 6.63 0.92 [0.53, 1.61] 1998 Cross (49) 7.51 0.49 [0.31, 0.78] 1999 Pope (62) 9.53 1.04 [0.81, 1.33] 2000 M cKay (7) 4.24 0.38 [0.16, 0.92] 2001 Am ako (6) 8.25 0.74 [0.50, 1.09] 2003

Subtotal (95% CI) 80.94 0.78 [0.59, 1.02] Test for heterogeneity: Chi? = 42.89, df = 10 (P < 0.00001), I? = 76.7% Test for overall effect: Z = 1.83 (P = 0.07) W om en M acera (58) 4.87 1.60 [0.73, 3.50] 1989 W alter (59 never) 3.02 1.18 [0.38, 3.70] 1989 W alter (59 som e) 5.66 0.56 [0.29, 1.10] 1989 W alter (59 usual) 5.52 1.05 [0.53, 2.10] 1989

Subtotal (95% CI) 19.06 0.98 [0.61, 1.57] Test for heterogeneity: Chi? = 4.27, df = 3 (P = 0.23), I? = 29.8% Test for overall effect: Z = 0.09 (P = 0.93)


Acute Stretching: Injury

Excluding multiple co-intervention studiesExcluding multiple co-intervention studies (adapted from Shrier, Evidence-Based Sports Medicine 2007)(adapted from Shrier, Evidence-Based Sports Medicine 2007)

Study OR or RR or HR (random) Weight OR or RR or HR (random) or sub-category 95% CI % 95% CI Year Men Ekstrand (52) 1983 Ekstrand (64) Not estimable 1983 M acera (58) 8.98 1.10 [0.81, 1.50] 1989 W alter (59 never) 6.69 1.15 [0.66, 2.00] 1989 W alter (59 som e) 8.59 0.64 [0.45, 0.91] 1989 W alter (59 usual) 8.36 1.25 [0.86, 1.82] 1989 Bixler (51) Not estimable 1992 van M echelin (61) 6.65 1.26 [0.72, 2.20] 1993 Pope (63) 6.63 0.92 [0.53, 1.61] 1998 Cross (49) 1999 Pope (62) 9.53 1.04 [0.81, 1.33] 2000 M cKay (7) 4.24 0.38 [0.16, 0.92] 2001 Am ako (6) 2003

Subtotal (95% CI)

W om en M acera (58) 4.87 1.60 [0.73, 3.50] 1989 W alter (59 never) 3.02 1.18 [0.38, 3.70] 1989 W alter (59 som e) 5.66 0.56 [0.29, 1.10] 1989 W alter (59 usual) 5.52 1.05 [0.53, 2.10] 1989

Subtotal (95% CI) 19.06 0.98 [0.61, 1.57] Test for heterogeneity: Chi? = 4.27, df = 3 (P = 0.23), I? = 29.8% Test for overall effect: Z = 0.09 (P = 0.93)


29.41 0.97 [0.79, 1.19] Test for heterogeneity: Chi? = 13.27, df = 7 (P = 0.07), I? = 47.2% Test for overall effect: Z = 0.27 (P = 0.79)

Excluding multiple co-intervention studiesExcluding multiple co-intervention studies

Acute Stretching: Injury



YesYes No No

I will now tell my patients to I will now tell my patients to stretchstretch before exercisebefore exercise to prevent injury to prevent injury

YesYes No No

I will now tell my patients not to I will now tell my patients not to stretch stretch before exercisebefore exercise to prevent injury to prevent injury

YesYes No No

I will now tell my patients that I have no idea whether I will now tell my patients that I have no idea whether they should they should stretch before exercisestretch before exercise to prevent injury to prevent injury



YesYes No No

I will now tell my patients to I will now tell my patients to stretch stretch regularlyregularly to prevent injury to prevent injury

YesYes No No

I will now tell my patients not to I will now tell my patients not to stretch stretch regularlyregularly to prevent injury to prevent injury

YesYes No No

I will now tell my patients that I have no idea whether I will now tell my patients that I have no idea whether they should they should stretch regularlystretch regularly to prevent injury to prevent injury


Clinical Trials and Meta-Analysis 1994;29:41–47 Clinical Trials and Meta-Analysis 1994;29:41–47

Effect of RCT on Outcomes

RCT vs. Observational: TheoreticalRCT vs. Observational: Theoretical• HughesHughes

Balanced placebo/traditional designBalanced placebo/traditional design RCT Informed: pts told Nicotine or PlaceboRCT Informed: pts told Nicotine or Placebo Balanced placebo: Pts randomized to be told Balanced placebo: Pts randomized to be told

Nicotine or Placebo, but random 50% given what Nicotine or Placebo, but random 50% given what they were told (4 groups)they were told (4 groups)

# D

ays

# D

ays

Sm

oke

dS

mo

ked

NicNicoo

PlaPlacc

BlinBlindd

% C

om

ple

te

% C

om

ple

te

Ab

stai

nm

ent

Ab

stai

nm

ent

NicNicoo

PlaPlacc

BlinBlindd

NicoNico

PlacPlac

Psychopharmacology 1989Psychopharmacology 1989

• ITT: Patient wants to know effect of intervention conditional on them ITT: Patient wants to know effect of intervention conditional on them receiving the intervention (per protocol?)receiving the intervention (per protocol?)

RCT vs. Observational: Evidence

• Linde: Linde: Observational studies about 10-20% better for Observational studies about 10-20% better for acupuncture/headacheacupuncture/headache (J Clin Epi 2002)(J Clin Epi 2002)

• Concato: Concato: No difference in well-designed studiesNo difference in well-designed studies (NEJM 2000)(NEJM 2000)

• MacLehose: MacLehose: discrepancies for high quality studies were small but discrepancies for high quality studies were small but discrepancies for low quality studies were largediscrepancies for low quality studies were large (HTA 2000) (HTA 2000)

• Benson: Benson: No difference after 1984No difference after 1984 (Am J Opthalmol 2000)(Am J Opthalmol 2000)

• Britton: Britton: “Non-randomized overestimated magnitude of effect”“Non-randomized overestimated magnitude of effect” (HTA 1998)(HTA 1998)

• Linde: Linde: Observational studies about 10-20% better for Observational studies about 10-20% better for acupuncture/headacheacupuncture/headache (J Clin Epi 2002)(J Clin Epi 2002)

• Concato: Concato: No difference in well-designed studiesNo difference in well-designed studies (NEJM 2000)(NEJM 2000)

• MacLehose: MacLehose: discrepancies for high quality studies were small but discrepancies for high quality studies were small but discrepancies for low quality studies were largediscrepancies for low quality studies were large (HTA 2000) (HTA 2000)

• Benson: Benson: No difference after 1984No difference after 1984 (Am J Opthalmol 2000)(Am J Opthalmol 2000)

• Britton: Britton: “Non-randomized overestimated magnitude of effect”“Non-randomized overestimated magnitude of effect” (HTA 1998)(HTA 1998)

more extrememore extreme

RCT vs. Observational: Theoretical

• All StudiesAll Studies Adjust for known confoundersAdjust for known confounders


• Unknown confounders likely to be Unknown confounders likely to be equally distributedequally distributedRCTRCT

ConConProProDesignDesign

Concealed randomization specifically removes the possibility of selection Concealed randomization specifically removes the possibility of selection bias or confounding in RCTs, i.e. any differences between the groups are bias or confounding in RCTs, i.e. any differences between the groups are attributable to chance or to the intervention, all else being equal.attributable to chance or to the intervention, all else being equal.

Deeks et al, Health Tech Asess 2003Deeks et al, Health Tech Asess 2003

Based on assumption of randomization in infinite population, or opposite Based on assumption of randomization in infinite population, or opposite distribution of confounders if many trials examineddistribution of confounders if many trials examined

Example: Confounder present in 20% of population. N= 400Example: Confounder present in 20% of population. N= 40095% Prob. Distr. = 15.6%-24.4%. If 5 confounders, 23% chance that at least 95% Prob. Distr. = 15.6%-24.4%. If 5 confounders, 23% chance that at least one is outside the range (95% Prob. Dist. = 14.2%-25.8%)one is outside the range (95% Prob. Dist. = 14.2%-25.8%) (Shrier et al, AJE 2007)(Shrier et al, AJE 2007)

RCT vs. Observational: Theoretical



DesignDesign ProPro ConCon

RCTRCT• Unknown confounders likely to be Unknown confounders likely to be

equally distributedequally distributed

• Control participants likely to Control participants likely to do better than non-do better than non-participantsparticipants

CohortCohort

• More representative sampleMore representative sample

• CheaperCheaper

• Historical Cohort: answer fasterHistorical Cohort: answer faster

• Increased sample size for Increased sample size for adjustmentadjustment

• Confounding by indication: Confounding by indication: patient/physician - randompatient/physician - random

There may be important prognostic factors that the There may be important prognostic factors that the investigators do not know about or have not investigators do not know about or have not measured which are unbalanced between groups and measured which are unbalanced between groups and responsible for differences in outcome.responsible for differences in outcome.

Deeks et al, Health Tech Asess 2003Deeks et al, Health Tech Asess 2003(Shrier et al, AJE 2007)(Shrier et al, AJE 2007)

turn on your IPOD now!turn on your IPOD now!

IF YOU WANT THE BLUE PILL….IF YOU WANT THE BLUE PILL….

QuickTime™ and aCinepak decompressor

are needed to see this picture.

FORMS OF BIAS

Structural Approach to BiasStructural Approach to Bias

• Confounding BiasConfounding Bias• Failure to condition on a common causeFailure to condition on a common cause• Do not condition on a variable (or marker of a Do not condition on a variable (or marker of a

variable) that lies along the causal pathwayvariable) that lies along the causal pathway

• Selection BiasSelection Bias• Conditioning on a common effectConditioning on a common effect

(Pearl, Hern(Pearl, Hernáán, Greenland)n, Greenland)

CONFOUNDING BIAS

OsteoarthritisOsteoarthritis(indirect)(indirect)

Gait DisorderGait Disorder(direct)(direct)

ActivityActivity

XX MM YY

Gait DisorderGait Disorder

OAOA ActivityActivityXX

CC

YY

XXXX

CONFOUNDING BIAS?

• Exposure: smokingExposure: smoking

• Outcome: spont. abortionOutcome: spont. abortion

• Confounding?: previous spont. abortionConfounding?: previous spont. abortion

(Weinberg Am J Epid 1993)(Weinberg Am J Epid 1993)

SmokingSmokingSmokingSmoking Spont. AbortionSpont. AbortionSpont. AbortionSpont. Abortion

Previous Sp. Ab.Previous Sp. Ab.Previous Sp. Ab.Previous Sp. Ab.

CONFOUNDING BIAS?







TissueTissueAbnormalityAbnormality


• Underlying abnormality: intrinsic tissue Underlying abnormality: intrinsic tissue abnormalityabnormality

CONFOUNDING BIAS?









• Underlying abnormality: intrinsic tissue Underlying abnormality: intrinsic tissue weaknessweakness

XX XX

CONFOUNDING BIAS?


• Univariate RR for smoking/no smokingUnivariate RR for smoking/no smoking =1.85=1.85

• Stratified RRStratified RR RR for smoking/no smoking (Previous Sp. Ab.)RR for smoking/no smoking (Previous Sp. Ab.) =1.32=1.32 RR for smoking/no smoking (No Previous Sp. Ab.) RR for smoking/no smoking (No Previous Sp. Ab.) =1.32=1.32

• However, whether or not someone had a previous spontaneous However, whether or not someone had a previous spontaneous abortion does not change the effects of smokingabortion does not change the effects of smoking

• Including this covariate results in an invalid estimateIncluding this covariate results in an invalid estimate





XX XX

CONFOUNDING BIAS?CONFOUNDING BIAS?

(Hern(Hernánán Am J Epid Am J Epid 2002)2002)

CC ExEx OutcomeOutcome

ExEx UU

OutcomeOutcome

CC

CCCC ExExExEx OutcomeOutcomeOutcomeOutcome

UUUU

CCCC ExExExEx OutcomeOutcomeOutcomeOutcome

UUUU

Condition on C?

(Cole & Hern(Cole & Hernánán Int J Epid 2002) Int J Epid 2002)

EE OutcomeOutcome

CC

U2U2U1U1

FORMS OF BIAS

Structural Approach to BiasStructural Approach to Bias

• Confounding BiasConfounding Bias• Failure to condition on a common causeFailure to condition on a common cause• Do not condition on a variable (or marker of a Do not condition on a variable (or marker of a

variable) that lies along the causal pathwayvariable) that lies along the causal pathway

• Selection BiasSelection Bias• Conditioning on a common effectConditioning on a common effect

(Pearl, Hern(Pearl, Hernáán, Greenland)n, Greenland)

(Pearl. Causality Book)(Pearl. Causality Book)

Step 4: Connect any two parents sharing a common child.Step 4: Connect any two parents sharing a common child. Including “colliders” opens up path for confoundingIncluding “colliders” opens up path for confounding

XX11

XX33 XX22

XX44

XX55

SprinklerSprinkler RainRain

SeasonSeason

WetWet

SlipperySlippery

If one knows the value If one knows the value of the “collider”, the of the “collider”, the parents are associated. parents are associated.

If wet:If wet: the sprinkler is the sprinkler is more likely to be more likely to be on if there was no on if there was no rain.rain.

PEARL’S RULES - EXPLANATION

UNBIASED EFFECT ESTIMATE?UNBIASED EFFECT ESTIMATE?

XX OutcomeOutcome


Which measurements should be included in the model if we are interested in Which measurements should be included in the model if we are interested in the relation between X and Outcome? the relation between X and Outcome?

XX OutcomeOutcome

ZZ11


ZZ22

Which measurements should be included in the model if we are interested in Which measurements should be included in the model if we are interested in the relation between X and Outcome? the relation between X and Outcome? Do ZDo Z11 and Z and Z22 remove confounding? remove confounding?


XX OutcomeOutcome

ZZ11


ZZ22

If X is disconnected from Outcome (d-separation), there is no confounding

Which measurements should be included in the model if we are interested in Which measurements should be included in the model if we are interested in the relation between X and Outcome? the relation between X and Outcome? Do ZDo Z11 and Z and Z22 remove confounding? remove confounding?

UNBIASED EFFECT ESTIMATE!UNBIASED EFFECT ESTIMATE!


XX OutcomeOutcome

ZZ11


ZZ22

XX OutcomeOutcome

ZZ11


ZZ22

X is NOT disconnected from Outcome

Which measurements should be included in the model if we are interested in Which measurements should be included in the model if we are interested in the relation between X and Outcome? the relation between X and Outcome? Do ZDo Z11, Z, Z22 and Z and Z33 remove confounding? remove confounding?

ZZ33

INCLUDING Z3 INTRODUCES BIAS!


SELECTION BIAS EXAMPLES

• Observational SpecificObservational Specific Berkson’s BiasBerkson’s Bias Volunteer / Self-selection BiasVolunteer / Self-selection Bias Healthy worker BiasHealthy worker Bias

• Meta-analysis specificMeta-analysis specific Reporting biasReporting bias Publication biasPublication bias

• RCT or ObservationalRCT or Observational Differential loss to follow-upDifferential loss to follow-up Non-response / Missing data biasNon-response / Missing data bias Adjustment for variables affected by previous exposureAdjustment for variables affected by previous exposure

ATTRITION BIASATTRITION BIAS

RCT Complex Attrition biasRCT Complex Attrition bias

TreatmentTreatment

DeathDeath

Side effectsSide effects Drop OutDrop Out

Mild diseaseMild disease

Condition on common effectCondition on common effect

EFFECT OF BIAS• Can one sum probability distributions for different risks of bias?Can one sum probability distributions for different risks of bias?

Already being done “intuitively” and informallyAlready being done “intuitively” and informally Some beginnings: response-surface estimation (Greenland), multiple Some beginnings: response-surface estimation (Greenland), multiple

bias modeling (Greenland), adjusted likelihoods (Wolpert), bias against bias modeling (Greenland), adjusted likelihoods (Wolpert), bias against bias (Kaufman)bias (Kaufman)

Treatment BeneficialTreatment Beneficial Treatment HarmfulTreatment Harmful

Bias Towards BenefitBias Towards BenefitBiased Against BenefitBiased Against Benefit

Probability of BiasProbability of Bias

ESTIMATING BIAS?

XX OutcomeOutcome

(Pearl. Causality Book)

CensoredCensored

Unknown VariableUnknown Variable

SUMMARY• Objective is to obtain an unbiased estimate of the Objective is to obtain an unbiased estimate of the

parameter of interestparameter of interest

• Study design is only one source of biasStudy design is only one source of bias

• Mathematics underlying statistical analyses do not care Mathematics underlying statistical analyses do not care what the names of the nodes arewhat the names of the nodes are

• Causal maps make assessing bias more transparentCausal maps make assessing bias more transparent

• Meta-analyses should be able to treat all potential biases Meta-analyses should be able to treat all potential biases regardless of causeregardless of cause

Estimating the wrong parameter - RCT, ITT?Estimating the wrong parameter - RCT, ITT? Conditioning on a variable that lies along the causal path or is a Conditioning on a variable that lies along the causal path or is a

marker for a variable lying along the causal pathmarker for a variable lying along the causal path Absence of conditioning on a common causeAbsence of conditioning on a common cause Conditioning on a common effectConditioning on a common effect

INTERPRETATIONS

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Canadian Academy of Sport MedicineL’Académie Canadienne de Médecine du Sport

meta-analysis: sports medicine canadian academy of sport medicine lacadémie canadienne de médecine...

Documents

clin j sport med

dip sport med

interpretations shrier

rcts slide

sport ian shrier md

ag da ag sd ag

da ag sa ag

sigh of relief slide