1

STAT 5500/6500 – Handout #24:

Meta-Analysis Examples

Example 1 A RCT was conducted to consider whether steroid therapy for expectant

mothers affects the survival rate of premature [less than 37 weeks] newborns. Expectant

mothers at risk were randomly assigned to steroid and placebo. At some set time after

birth, observe numbers alive and dead.

Alive (1) Dead (0)

Steroid (1) 496 36

Placebo (0) 478 60

First, look at some things in SAS …

/* Read in data (line 1 from Table 12.1 of Matthews text)

*/

data rct; input Steroid Alive Count @@; cards;

1 1 496 1 0 36 0 1 478 0 0 60

;

/* Chi-square test of independence */

proc freq data=rct;

tables Steroid*Alive / chisq nocol norow nopercent;

weight Count;

title1 'Chi-square test of independence';

run;

Statistic DF Value Prob

Chi-Square 1 6.2992 0.0121

Likelihood Ratio Chi-Square 1 6.3631 0.0117

Fisher's Exact Test

Cell (1,1) Frequency (F) 60

Two-sided Pr <= P 0.0137

Sample Size = 1070

2

/* How to quantify the difference? */

proc logistic data=rct;

model Alive(event='1') = Steroid;

weight Count;

title1 'RCT data';

run;

RCT data

Probability modeled is Alive=1.

Analysis of Maximum Likelihood Estimates

Parameter DF Estimate Standard

Error

Wald

Chi-Square

Pr > ChiSq

Intercept 1 2.0753 0.1370 229.5855 <.0001

Steroid 1 0.5478 0.2203 6.1804 0.0129

Odds Ratio Estimates

Effect Point Estimate 95% Wald

Confidence Limits

Steroid 1.729 1.123 2.664

So the odds of “Alive” is 73% higher in Steroid group than Placebo.

3

##########################################################

# Now do some things in R

##########################################################

# Look at test for a single RCT

rct1 <- matrix(c(496,478,36,60),ncol=2)

rct1

[,1] [,2]

[1,] 496 36

[2,] 478 60

r1 <- fisher.test(rct1)

r1$p.value

.01368349

# Get magnitude and direction

steroid <- c(1,0,1,0)

alive <- c(1,1,0,0)

r.wts <- c(496,478,36,60)

result <- glm( alive~steroid, weight=r.wts,

family='binomial')

result.1 <- summary(result)$coeff

round(result.1,4)

Estimate Std. Error z value Pr(>|z|)

(Intercept) 2.0753 0.1370 15.1521 0.0000

steroid 0.5478 0.2203 2.4860 0.0129

exp(0.54779)

1.729427

4

Example 2 The RCT in Example 1 was not the only study conducted to address the

question of whether steroid therapy for expectant mothers affects death rate of premature

newborns. A total of 12 RCT’s (including the RCT in Example 1) were identified:

By systematically combining results across these multiple studies, we can arrive at a

clearer understanding of the true treatment effect. [Column order for 2x2 tables: b a d c]

# Get results for all RCT's

filepath <-

"http://www.stat.usu.edu/jrstevens/biostat/data/steroid.csv"

data <- read.csv(filepath)

Fishers.p <- rep(NA,nrow(data))

for(i in 1:nrow(data))

{

mat <- matrix(as.numeric(data[i,]),ncol=2)

Fishers.p[i] <- fisher.test(mat)$p.value

}

logOR <- log( data[,2]*data[,3] /

(data[,1]*data[,4]) )

# 'fix' study 6 [no evidence of effect]

logOR[6] <- 0

round(cbind(data,Fishers.p, logOR),4)

# Look at Fishers p-values

hist(Fishers.p, main=NA)

Fishers.p

Fre

quency

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

5

SteroidDead SteroidAlive PlaceboDead PlaceboAlive Fishers.p logOR 1 36 496 60 478 0.0137 0.5478 2 1 68 5 56 0.0982 1.8036 3 3 61 12 46 0.0112 1.6685 4 5 51 7 64 1.0000 0.1094 5 2 79 10 53 0.0052 2.0086 6 0 38 0 42 1.0000 0.0000 7 14 117 20 117 0.3637 0.3567 8 36 335 37 335 1.0000 0.0274 9 7 114 13 111 0.2436 0.6457 10 1 70 5 70 0.2101 1.6094 11 2 65 7 52 0.0816 1.4759 12 5 29 5 26 1.0000 0.1092

# Look at Fishers composite testing approach

S <- -2*sum(log(Fishers.p))

1-pchisq(S,2*length(Fishers.p))

# For Stouffer's, need to convert 1-sided to 2-sided p-vals

# Use p2.p1 function from SigTree package:

# * p is a vector of two-tailed p-vals, w/ null M2=M1

# * diff is a vector of differences M2-M1

# This returns a vector of one-sided p-values for alt M2>M1

p2.p1 <- function (p, diff)

{ p1 <- rep(NA, length(p)); t <- diff >= 0

p1[t] <- p[t]/2; p1[!t] <- 1 - p[!t]/2; return(p1)

}

# Look at Stouffers composite testing approach

F.p1 <- p2.p1(Fishers.p, logOR)

z <- qnorm(1-F.p1)

zs <- sum(z)/sqrt(length(z))

1-pnorm(zs)

######################################

# Now look at effect size approaches

######################################

# Define effect size estimate and variance

theta.hat <- log( (data[,2]+0.5)*(data[,3]+0.5) /

((data[,1]+0.5)*(data[,4]+0.5)) )

var.theta.hat <- 1/(data[,2]+0.5) + 1/(data[,3]+0.5) +

1/(data[,1]+0.5) + 1/(data[,4]+0.5)

0.004789277

1.38417e-05

6

# Look at fixed effects model

w.FE <- 1/var.theta.hat

theta.FE <- sum(w.FE*theta.hat) / sum(w.FE)

var.FE <- 1/sum(w.FE)

z.FE <- theta.FE/sqrt(var.FE)

p.FE <- 2*(1-pnorm(abs(z.FE)))

round(c(z.FE,p.FE),5)

3.81645 0.00014

# Look at overall odds ratio from fixed effects model

OR.FE <- exp(theta.FE)

OR.FE

1.616437

# Now get an approximate 95% confidence interval for OR

CI.FE.dn <- exp(theta.FE - 1.96*sqrt(var.FE))

CI.FE.up <- exp(theta.FE + 1.96*sqrt(var.FE))

round(c(CI.FE.dn,CI.FE.up),5)

1.26314 2.06856

# automate this

library(metafor)

# may first need: install.packages("metafor")

?rma.uni

Description

Function to fit the meta-analytic fixed- and random/mixed-effects models with or without

moderators via linear (mixed-effects) models. See the documentation of the metafor-

package for more details on these models.

Usage

rma.uni(yi, vi, sei, weights, ai, bi, ci, di, n1i, n2i, x1i, x2i,

t1i, t2i, m1i, m2i, sd1i, sd2i, xi, mi, ri, ti, sdi, ni, mods,

measure="GEN", intercept=TRUE, data, slab, subset,

add=1/2, to="only0", drop00=FALSE, vtype="LS",

method="REML", weighted=TRUE, knha=FALSE,

level=95, digits=4, btt, tau2, verbose=FALSE, control)

7

Arguments

yi vector of length k with the observed effect sizes or outcomes. See ‘Details’.

vi vector of length k with the corresponding sampling variances. See ‘Details’.

...

ai vector to specify the 2x2 table frequencies (upper left cell).

bi vector to specify the 2x2 table frequencies (upper right cell).

ci vector to specify the 2x2 table frequencies (lower left cell).

di vector to specify the 2x2 table frequencies (lower right cell).

n1i vector to specify the group sizes or row totals (first group/row).

n2i vector to specify the group sizes or row totals (second group/row).

...

mods optional argument to include one or more moderators in the model. A single

moderator can be given as a vector of length k specifying the values of the

moderator. Multiple moderators are specified by giving a matrix with k rows

and p' columns. Alternatively, a model formula can be used to specify the

model. See ‘Details’.

measure character string indicating the type of data supplied to the function. When

measure="GEN" (default), the observed effect sizes or outcomes and

corresponding sampling variances (or standard errors) should be supplied to the

function via the yi, vi, and sei arguments (only one of the two, vi or sei,

needs to be specified). Alternatively, one can set measure to one of the effect

size or outcome measures described under the documentation for the escalc

function and specify the needed data via the appropriate arguments.

data optional data frame containing the data supplied to the function.

slab optional vector with unique labels for the k studies.

add a non-negative number indicating the amount to add to zero cells, counts, or

frequencies. See ‘Details’.

to a character string indicating when the values under add should be added (either

"all", "only0", "if0all", or "none"). See ‘Details’.

method character string specifying whether a fixed- or a random/mixed-effects model

should be fitted. A fixed-effects model (with or without moderators) is fitted

when using method="FE". Random/mixed-effects models are fitted by setting

method equal to one of the following: "DL", "HE", "SJ", "ML", "REML", "EB",

or "HS". Default is "REML". See ‘Details’.

8

Measures for Dichotomous Variables

In various fields (such as the health and medical sciences), the response variable measured

is often dichotomous (binary), so that the data from a study comparing two different

groups can be expressed in terms of a 2x2 table, such as:

outcome 1 outcome 2 total

group 1 ai bi n1i

group 2 ci di n2i

The options for the measure argument are then:

• "RR" for the log relative risk.

• "OR" for the log odds ratio.

• "RD" for the risk difference.

• "PETO" for the log odds ratio estimated with Peto's method (Yusuf et al., 1985).

result <- rma.uni(

ai = SteroidAlive,

bi = SteroidDead,

ci = PlaceboAlive,

di = PlaceboDead,

measure = 'OR', # report LOG odds ratio scale

add = 0.5, to='all',

method = 'DL', # Dersimonian-Laird

slab = 1:12, # study labels

data=data # dataset containing named variables

)

summary(result)

Random-Effects Model (k = 12; tau^2 estimator: DL)

tau^2 (estimated amount of total heterogeneity): 0.0607 (SE = 0.1228)

I^2 (total heterogeneity / total variability): 21.70%

H^2 (total variability / sampling variability): 1.28

Test for Heterogeneity:

Q(df = 11) = 14.0484, p-val = 0.2303

Model Results:

estimate se zval pval ci.lb ci.ub

0.5490 0.1614 3.4011 0.0007 0.2326 0.8654 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

9

# Transform LOG odds ratio estimate and CI to OR scale:

exp(c(.5490,.2326,.8654))

[1] 1.731521 1.261877 2.375956

# visualize results in a forest plot

forest(result)

# Check for publication bias using Funnel plot

funnel(result)

10

# Check for publication bias using Galbraith (radial) plot

radial(result)

# Get Galbraith test of significance:

z <- result$yi/sqrt(result$vi + result$tau2)

x <- 1/sqrt(result$vi + result$tau2)

fit <- lm(z~x)

summary(fit)$coeff

Estimate Std. Error t value Pr(>|t|)

(Intercept) 1.18340465 0.6446501 1.8357318 0.09626887

x -0.05588472 0.3604818 -0.1550279 0.87988381

Linear regression test of funnel plot asymmetry

abline(1.18340465,-.05588472, lty=2)

11

# Check for publication bias using normal quantile plot

qqnorm(result)

# Include 'moderator' [covariate] -- studies 4,6,7, 8, and 12

# include only mothers with a certain complication,

# while other studies excluded such mothers

complication <- rep(0,nrow(data))

complication[c(4,6,7,8,12)] <- 1

# [optionally] center non-intercept columns

# to preserve interpretation of intercept

ctr.complication <- complication - mean(complication)

result.mod <- rma.uni(

ai = SteroidAlive, bi = SteroidDead, ci = PlaceboAlive,

di = PlaceboDead, measure = 'OR', add = 0.5, to='all',

method = 'DL', slab = 1:12, data=data,

mods = ctr.complication

)

summary(result.mod)

Test for Residual Heterogeneity:

QE(df = 10) = 6.8025, p-val = 0.7440

Model Results:

estimate se zval pval ci.lb ci.ub

intrcpt 0.5142 0.1265 4.0664 <.0001 0.2664 0.7621

mods -0.6789 0.2522 -2.6918 0.0071 -1.1733 -0.1846

12

# Look at correcting for publication bias

funnel(result)

fn <- function(x){sqrt( ((x)+.0559)^2 / 1.1834^2 - .0607 )}

curve(fn, from=(.2357), to=2.5, col="red", lwd=2,lty=2, add=TRUE)

points(.2357, 0, pch=18, cex=1.5, col='red')

# Can automate using limitmeta function in metasens package,

# which gives:

# Adjusted OR = 1.2656 (log of this is 0.2355)

# with OR CI [.7843, 2.0423]

# and p-value .3346