1954 Salk polio vaccine 1954 Salk polio vaccine trialstrials

► Biggest public Biggest public health experiment health experiment everever

► Polio epidemics hit Polio epidemics hit U.S. in 20U.S. in 20thth century century

► Struck hardest at Struck hardest at childrenchildren

► Responsible for 6% Responsible for 6% of deaths among 5- of deaths among 5- to 9- year-oldsto 9- year-olds

Number of polio cases in the U.S.

1930 to 1955

YEAR

1954

1952

1950

1948

1946

1944

1942

1940

1938

1936

1934

1932

1930

Valu

e C

AS

ES

60000

50000

40000

30000

20000

10000

0

Salk vaccine trial: Salk vaccine trial: BackgroundBackground

► Polio is rare but the virus itself is commonPolio is rare but the virus itself is common► Most adults experienced polio infection Most adults experienced polio infection

without being aware of it.without being aware of it.► Children from higher-income families were Children from higher-income families were

more vulnerable to polio! more vulnerable to polio! ► Children in less hygienic surroundings Children in less hygienic surroundings

contract mild polio early in childhood while contract mild polio early in childhood while still protected from their mother’s still protected from their mother’s antibodies. They develop immunity early.antibodies. They develop immunity early.

► Children from more hygienic surroundings Children from more hygienic surroundings don’t develop such antibodies.don’t develop such antibodies.

Salk trial: The need for testingSalk trial: The need for testing► By 1954, Salk’s research with a vaccine By 1954, Salk’s research with a vaccine

looked promisinglooked promising► Government agencies were ready to try the Government agencies were ready to try the

vaccine in the general population but some vaccine in the general population but some scientists feared the vaccine was unsafe or scientists feared the vaccine was unsafe or ineffective.ineffective.

► There was enormous fear and desperation There was enormous fear and desperation throughout the country.throughout the country.

► Why not just distribute the vaccine to some Why not just distribute the vaccine to some and see if it lowered the polio rate?and see if it lowered the polio rate? A yearly drop might mean the drug was A yearly drop might mean the drug was

effective, or that that year was not an effective, or that that year was not an epidemic yearepidemic year

► Vaccine could not be distributed without Vaccine could not be distributed without testing testing

Salk vaccine trial: Salk vaccine trial: The need for controlsThe need for controls

► An experiment requires An experiment requires controls. controls. ► To test if the vaccine was effective the only variable To test if the vaccine was effective the only variable

that should be considered is the vaccine itselfthat should be considered is the vaccine itself► This means that some children would get the This means that some children would get the

vaccine and some would not.vaccine and some would not.► This raises enormous ethical questions:This raises enormous ethical questions:

Is it ethical to not give children the vaccine?Is it ethical to not give children the vaccine? Imagine yourself as a parent in these desperate Imagine yourself as a parent in these desperate

times. Would you participate in such an times. Would you participate in such an experiment.experiment.

Ultimately, does the benefit to society outweigh Ultimately, does the benefit to society outweigh the risk to those children who would not get the the risk to those children who would not get the vaccine?vaccine?

Salk vaccine:Salk vaccine:The need for massive trialsThe need for massive trials

► Polio rate of occurrence is about 50 per Polio rate of occurrence is about 50 per 100,000100,000

► Suppose the vaccine was 50% effective and Suppose the vaccine was 50% effective and 10,000 subjects were recruited for each of 10,000 subjects were recruited for each of the control and treatment groupsthe control and treatment groups You would expect 5 polio cases in control group You would expect 5 polio cases in control group

and 2-3 in treatment groupand 2-3 in treatment group Such a difference could be attributed to random Such a difference could be attributed to random

variationvariation► Clinical trials were needed on a massive Clinical trials were needed on a massive

scalescale► The ultimate experiment involved over 1.6 The ultimate experiment involved over 1.6

million children, with over 600,000 children million children, with over 600,000 children inoculatedinoculated

Controversy over the Controversy over the design of the experimentdesign of the experiment

► In order to isolate the vaccine as the only variable to In order to isolate the vaccine as the only variable to be considered, the be considered, the treatmenttreatment and and controlcontrol groups groups need to be as similar as possibleneed to be as similar as possible

► But how should subjects be recruited?But how should subjects be recruited?► Fact:Fact: volunteers tend to be better educated and volunteers tend to be better educated and

more well-to-do than those who don’t participatemore well-to-do than those who don’t participate► In the context of the polio disease, relying on In the context of the polio disease, relying on

volunteers could potentially volunteers could potentially biasbias the results the results Subjects would tend to have higher rates of polioSubjects would tend to have higher rates of polio Subjects are not representative of the populationSubjects are not representative of the population Results would be biased Results would be biased against the vaccineagainst the vaccine

► After much debate, the trials proceeded with two After much debate, the trials proceeded with two different protocols.different protocols.

““Observed Control” approachObserved Control” approach

► Administer the experiment to 1Administer the experiment to 1stst, 2, 2ndnd, and 3, and 3rdrd gradersgraders

► Offer the vaccination to 2Offer the vaccination to 2ndnd graders graders This group would rely on volunteers (parental consent)This group would rely on volunteers (parental consent)

► Use 1Use 1stst and 3 and 3rdrd graders as control group graders as control group These children would be observed for incidences of polioThese children would be observed for incidences of polio

► Supporters of this approach argued that there Supporters of this approach argued that there would not be much variability between grades so would not be much variability between grades so treatment and control groups would be similartreatment and control groups would be similar

► And the control group would be “observed controls”And the control group would be “observed controls”► But there were objections . . .But there were objections . . .

NFIP Observed Control studyNFIP Observed Control study

► Volunteers would result in more children from Volunteers would result in more children from higher income families in treatment grouphigher income families in treatment group Treatment group is thus more vulnerable to disease Treatment group is thus more vulnerable to disease

than control groupthan control group Would expect more incidences of polio in the Would expect more incidences of polio in the

treatment group than in the control grouptreatment group than in the control group Biases the experiment against the vaccineBiases the experiment against the vaccine

► How would incidents of the disease be How would incidents of the disease be diagnosed?diagnosed? Many forms of polio are hard to diagnoseMany forms of polio are hard to diagnose In making the diagnosis physicians would naturally In making the diagnosis physicians would naturally

ask whether a child was vaccinated or notask whether a child was vaccinated or not Diagnosis for borderline cases could be affected by Diagnosis for borderline cases could be affected by

knowledge of what grade the child was in and knowledge of what grade the child was in and whether the child was vaccinated or notwhether the child was vaccinated or not

Randomized control approachRandomized control approach

► This experiment relied on volunteer subjects overall.This experiment relied on volunteer subjects overall.► But subjects were But subjects were randomly assignedrandomly assigned to treatment to treatment

and control groupsand control groups► Control group was given a Control group was given a placeboplacebo► Placebo material was prepared to look Placebo material was prepared to look

exactly like the vaccine so subjects didn’t exactly like the vaccine so subjects didn’t know what treatment they were gettingknow what treatment they were getting

► Placebo-control group guards against thePlacebo-control group guards against the““placebo effect”placebo effect”

► Many objected to the design on ethicalMany objected to the design on ethicalgrounds. grounds.

► Jonas Salk himself called it “A `beautiful’ experiment Jonas Salk himself called it “A `beautiful’ experiment over which the epidemiologist could become quite over which the epidemiologist could become quite ecstatic but which would make the humanitarian ecstatic but which would make the humanitarian shudder.”shudder.”

Randomized control approachRandomized control approach

► Subjects were “blind”: they did not know to which Subjects were “blind”: they did not know to which group they were assignedgroup they were assigned

► Also, those doing the evaluationAlso, those doing the evaluationdidn’t know which treatmentdidn’t know which treatmentany subject receivedany subject received

► Each vial was identified by a codeEach vial was identified by a codenumber so no one involved in thenumber so no one involved in thevaccination or the diagnostic vaccination or the diagnostic evaluation could know who gotevaluation could know who gotthe vaccine.the vaccine.

► Experiment was Experiment was double-blind: double-blind: neither subjects nor those doing neither subjects nor those doing the evaluation knew which the evaluation knew which treatment any subject receivedtreatment any subject received

Results of vaccine trialsThe randomized, controlled experiment

SizeSize Rate (per Rate (per 100,000)100,000)

TreatmentTreatment 200,000200,000 2828

ControlControl 200,000200,000 7171

No consentNo consent 350,000350,000 4646

The Observed Control study

SizeSize Rate (per Rate (per 100,000)100,000)

Grade 2 (vaccine)Grade 2 (vaccine) 225,000225,000 2525

Grade 1, 3 (control)Grade 1, 3 (control) 725,000725,000 5454

Grade 2 (no Grade 2 (no consent)consent)

125,000125,000 4444Source: Thomas Francis, J r., “An evaluation of the 1954 Poliomyelitis vaccine trials---summary report,” American Journal of Public Health vol 45 (1955) pp. 1-63.

Comparing the two studiesComparing the two studies

► Results show that the observed control study was Results show that the observed control study was biased against vaccinebiased against vaccine Treatment group got the vaccine but was more prone to Treatment group got the vaccine but was more prone to

higher polio rateshigher polio rates Control group didn’t get the vaccine but was more prone to Control group didn’t get the vaccine but was more prone to

lower polio rateslower polio rates► It’s impossible to determine what’s the effect of the It’s impossible to determine what’s the effect of the

vaccine and what’s the effect of socio-economic vaccine and what’s the effect of socio-economic statusstatus

► This is called This is called confoundingconfounding—the inability to —the inability to distinguish the separate impacts of two or more distinguish the separate impacts of two or more variables on a single outcome. variables on a single outcome.

► In a randomized controlled experiment, by making In a randomized controlled experiment, by making the treatment and control groups as similar as the treatment and control groups as similar as possible (by randomization), we are able to isolate possible (by randomization), we are able to isolate the variable of interest and eliminate confoundingthe variable of interest and eliminate confounding

Comparing the two studies: Comparing the two studies: are the results “significant”?are the results “significant”?

► In the “observed control” approach, In the “observed control” approach, chance enters the study in an chance enters the study in an unplanned and haphazard way based unplanned and haphazard way based on what families will volunteeron what families will volunteer

►By contrast, for the randomized By contrast, for the randomized controlled experiment chance enters controlled experiment chance enters the study in a planned and simple waythe study in a planned and simple way Each child has 50-50 chance to be in the Each child has 50-50 chance to be in the

treatment or control grouptreatment or control group►This allows for the use of probability to This allows for the use of probability to

analyze the resultsanalyze the results

Are the results significant?Are the results significant?► Two competing positions—which side Two competing positions—which side

would you be on?would you be on? ProPro: “The vaccine is effective. There were less : “The vaccine is effective. There were less

cases of polio in the treatment group than in cases of polio in the treatment group than in the control group. We should undertake a the control group. We should undertake a massive vaccination program throughout the massive vaccination program throughout the general population.”general population.”

ConCon: “We are not convinced. The two groups : “We are not convinced. The two groups were randomly divided. There may have been were randomly divided. There may have been fewer polio-prone people in the treatment fewer polio-prone people in the treatment group. It was all done by chance. We can’t be group. It was all done by chance. We can’t be sure and we’re not willing to commit millions of sure and we’re not willing to commit millions of dollars of taxpayer’s money on a vaccination dollars of taxpayer’s money on a vaccination program that might not be effective.”program that might not be effective.”

Are the results Are the results significant?significant?► Assume the cons are right and that the Assume the cons are right and that the

vaccine is worthless. vaccine is worthless. What are the What are the chances of seeing such a large chances of seeing such a large difference in the two groups?difference in the two groups?

► Imagine a “polio” coin where the Imagine a “polio” coin where the chance of heads is equal to the chance of heads is equal to the chance that a person gets polio. chance that a person gets polio. Flip the coin in Room A for 200,000 times. Then Flip the coin in Room A for 200,000 times. Then flip it in Room B for 200,000 times. What’s the flip it in Room B for 200,000 times. What’s the chance that we would get such a large chance that we would get such a large difference as 28 heads in A and 71 heads in B?difference as 28 heads in A and 71 heads in B?

► They are over a billion to one against!They are over a billion to one against!► In the face of such odds, we say that the In the face of such odds, we say that the

outcome is outcome is statistically significant. statistically significant. The effect The effect is so large that it would rarely occur by chance.is so large that it would rarely occur by chance.

Salk vaccine trials aftermathSalk vaccine trials aftermath

► The results, announced in 1955, showed good The results, announced in 1955, showed good statistical evidence that Jonas Salk's vaccine was statistical evidence that Jonas Salk's vaccine was 80-90% effective in preventing paralytic 80-90% effective in preventing paralytic poliomyelitis. poliomyelitis.

► Postscript:Postscript: Polio was virtually eliminated from the Polio was virtually eliminated from the Americas in 1994, but still circulates in Asia and Americas in 1994, but still circulates in Asia and Africa, paralyzing the world’s most vulnerable Africa, paralyzing the world’s most vulnerable children. children.

► The Global Polio Eradication Initiative was begun in The Global Polio Eradication Initiative was begun in 1988. That year, an estimated 350,000 children 1988. That year, an estimated 350,000 children were paralyzed with polio worldwide.were paralyzed with polio worldwide.

► In 2004, polio cases had fallen to just over 1,200 In 2004, polio cases had fallen to just over 1,200 cases globally. cases globally.

The language of experimental design

In an experiment, we have at least one explanatory variable, called a factor, to manipulate and at least one response variable to measure

The specific values that the experimenter chooses for a factor are called the levels of the factor.

A treatment is a combination of specific levels from all the factors that an experimental unit receives.

The ability to manipulate factors, apply treatments, and compare the responses is what differentiates an experiment from an observational study

Observational studies

Nurses Health Study often in the news– Over 100,000 registered nurses aged 30 to 55 have been

followed for more than 30 years– Detailed questionnaires sent out every two years on a wide

variety of health and nutrition issues– 90% response rate– “One of the most significant studies ever conducted on the

health of women.” -- Donna Shalala, Former Secretary of the U.S. Department of Health and Human Services

This is a prospective study. Subjects were identified in advance and data collected as events unfolded.

Many observational studies are retrospective. Subjects are selected and their previous conditions or behaviors are determined.

Confounding

Observational studies can suffer from confounding and lurking variables

You’ll read about this over the weekend in “Hormone Studies: What Went Wrong?”

The ability to control and manipulate variables and compare groups allows for eliminating confounding and the effect of lurking variables

Double-blind, placebo-controlled randomized comparative experiment: The “gold standard” of statistics

Massive clinical trials industry Complex ethical questions for experiments involving human

subjects– Informed Consent, Institutional Review Board, Confidentiality

Placebo effect is a fascinating area of research– In conditions such as pain, the percent of patients responding to

placebos has been shown to be 20% to 50%.– Reflects the amount that the body can be coaxed/empowered to

heal itself, in the absence of other active agents. Today, few clinical trials compare against placebo. Most new

drugs are improvements over existing therapies. If an existing medicine exists it would be unethical to deny it to subjects

Other experimental design issues: Blocking

When groups of experimental units are similar, it’s often a good idea to gather them together into blocks.

Blocking isolates the variability due to the differences between the blocks so that we can see the differences due to the treatments more clearly.

When randomization occurs only within the blocks, we call the design a randomized block design

By contrast, a completely randomized design, all subjects have an equal chance of receiving any treatment.

Diagram of a blocked experiment

Hypertension pharmacogenetics studyHypertension pharmacogenetics study

• Hypertension is most prevalent risk factor for diseases of the heart, brain and kidneys, affecting 43 million in U.S.

• Complex disease affected by physical, physiological and environmental factors

• State-of-the-art for treatment is trial-and-error• Less than 40% of treated patients achieve blood

pressure control (systolic blood pressure < 140)• Ultimate goal of this study is to identify unknown

genes that influence drug response with the potential of tailoring antihypertensive therapy for individuals

GERA Clinical TrialGERA Clinical Trial• Black and white patients react differently to blood pressure

medicine• Blocked experimental design• Mayo Clinic – Rochester, MN

– 300 white subjects with hypertension (150 women and 150 men, ages 30 to 60)

• Emory University – Atlanta, GA– 300 Black subjects with hypertension (150 women and 150 men, ages

30 to 60)• Subjects had previous medications discontinued for 4 weeks;

blood pressure rose and stabilized in hypertensive range• Hydrochlorothiazide administered for 4 weeks• Blood pressure measured at the beginning of therapy and after

4 weeks• In each group, identify 100 “best” responders and 100 “worst”

responders by change in blood pressure

YrYr NN RaceRace DrugDrug RaceRace NN

11 100100 BB HHydrochlorothiazideydrochlorothiazide BB 100100

22 100100 WW WW 100100

BP increaseBP increase BP decreaseBP decrease

GERA clinical trialGERA clinical trial• DNA collected for each patient • Data consists of 100,000 genetic markers called Single-

Nucleotide Polymorphisms (SNPs)• Goal: to find an association between blood pressure response

and genetic makeup• Ultimate goal: to find those genes that affect blood pressure

response• What makes this complicated is that we have only 400

observations (the patients) and over 100,000 variables (the genetic markers)

• Classically in statistics we had a “few” variables and “many” observations. As datasets become larger and more complex, this classic paradigm is shifting and the challenges are enormous!