chapter 2: observational studies - binghamton...

Chapter 2: Observational StudiesPART I : DESIGN OF EXPERIMENTS

Dr. Joseph Brennan

Math 148, BU

Dr. Joseph Brennan (Math 148, BU) Chapter 2: Observational Studies 1 / 21

The TWO Main Types of Statistical Studies

OBSERVATIONAL STUDY

A study based on observing individuals and measuring responses butdoes not attempt to influence responses.

An investigator does not assign individuals to different treatments.He or she selects individuals from the population who have thecondition whose effects are being studied; this is the treatment group.Individuals without the condition comprise the control group.

The smoking study is an observational study. People cannot be coercedinto smoking for the period of a study. Smokers form a treatmentgroup. Nonsmokers form a control group.

EXPERIMENTAL STUDY

A study based on deliberately imposing some treatment(s) onindividuals to observe their responses.

The placebo treatment is only relevent for experimental studies.

The fever study is an experimental study.


Main Types of Observational Studies

Cross-sectional Study (or sample survey): provides informationabout a population based on a sample from the population at aspecific time point.

All opinion polls are cross-sectional studies.

Longitudinal Study (prospective study): selects a sample from thepopulation and follows the sample forward in time in order to observedeveloping certain conditions (for example, the occurrence of adisease).

The smoking example is a longitudinal study. We follow the smokersand nonsmokers for a certain time and compare the rates of the lungcancer in two groups at the end of the study.


Main Types of Observational Studies

Retrospective Study (case-control study): An observational studyof archived data. A retrospective study looks backwards and examinesvariables in relation to an outcome that is pre-established.

The smoking example: asking lung cancer patients who smoke howmuch they smoked and then trace back the data. Retrospective studiestakes less time that prospective study, and cancer patients are morelikely to remember (and perhaps regret) how much they smoked.


Example 4 (Hypertension-Obesity)

Question : Is obesity related to hypertension?Ethically we are unable to adress this question with an experimental study!This question can be addressed by designing either prospective orretrospective study:

Prospective Study.Create two groups of people without hypertension; one group is obese(treatment group) while the other is non-obese (control group). Followthe groups for a certain period of time (e.g., 5 years). At the end ofthe study compare the proportions of hypertensive people in the twogroups.

Retrospective Study.Find people with hypertension (“cases”) and without hypertension(“controls”). Compare the proportion of obese people among casesand controls.



The roles of the treatment and response variables are flipped in the twostudies:

In the prospective study the treatment variable is obesity (Yes/No),while the response is hypertension (Yes/No).

In the retrospective study the treatment variable is hypertension(Yes/No), while the response variable is obesity (Yes/No).



In general, retrospective studies are cheap and quick, however:

Recall of past events may be inaccurate and selective.

As events occurred in the past, no control over the course of thestudy is possible.

BEWARE CONFOUNDING VARIABLES!It is difficult to gather reliable past data on potential confounding variablessuch as frequency of exercise, work related stress, use of medication, etc.

Alternatively, in prospective studies subjects can keep diaries of theirdaily activities, thus giving more trustworthy data. Also, subjects can beasked to avoid certain practices which might bias the study.

In conclusion, compared to retrospective studies, prospective studies aremore costly, but provide a control on gathered data.


Cautions about Observational Studies

Confounding factors are very difficult to control! The main problem withany observational study is whether the control group was really similar tothe treatment group with respect to confounding factors.

In the smoking example the obvious confounding factors are genderand age:

Smokers are disproportionately males, and men in general havedifferent odds of getting lung cancer.

Smokers are likely to be older, and older people have higher chances toget any type of cancer.

We mentioned just 2 confounding factors above, but there may be manyothers which are not obvious upon first glance.


Interpreting the Results of Observational Studies

Because of potential hidden confounding factors, an observational studymay only establish an ASSOCIATION between the treatment and theresponse, but NOT the cause-effect relationship.

!Association 6= Causation!

CAUSATION can be established only from well-designed experimentalstudies which have better control over the confounding factors.

In the lung cancer study, which is an observational study, we mayNOT conclude that smoking causes lung cancer. All we may say isthat lung cancer is associated with smoking.


Are Sharks Attracted to Coke?

Figure: Shark attack Figure: Coca-Cola

If there is a correlation between shark attacks and buying soda, canwe then conclude that drinking more soda leads to shark attacks?

One potential source of the error could be that data was collected on ahot day, near the beach which is prone to shark attacks. People boughtsoda and then went swimming.


Nobel Prize Winners Love Milk?

A recent observational study has found a positive correlation between theaverage milk consumption and the number of nobel prize winners in acountry.

How many obvious possible confounding variables can you list in tenminutes?


Experimental Studies

Well-designed experimental studies use the following principles:1 Control;2 Randomization;3 Repetition;4 Blinding.

An experiment using principles 1-3 is called a randomized controlledexperiment. The structure of the randomized controlled experiment isplotted below.

Figure: Structure of a randomized controlled experiment.



Well-designed experiments establish the CAUSE-EFFECT relationshipbetween the treatment and response.

Control:There should be a treatment group and the control group.

If the control group is comparable to the treatment group, apartfrom the treatment, then a difference in the responses of the twogroups is likely to be due to the effect of treatments.

When an experiment involves people the control group should not beleft without a treatment! If the control group is not supposed to betreated, it should be given a placebo treatment to avoid the placeboeffect.



Randomization:When there are just two groups, one treatment and one control, eachunit should have 50-50 chance to be assigned into treatment group orcontrol group. Every experimental unit should have equal chances toget into each group.

In many medical studies the data collection starts at “baseline”, whenthe units (subjects) are recruited into the study but before they arerandomized to treatment or control.

Experimenters should divide experimental units into treatment andcontrol groups at random, without using a personal judgement.

Why do we need randomization?Randomization guarantees that the treatment and control groups aresimilar with respect to potential confounding factors before thetreatments are applied.



Repetition:

Repeat each treatment on many units to reduce chance variation inthe results.

This explains why we usually have not just one experimental unit ineach of the treatment and control groups, but as many experimentalunits as possible.

The more units we have in each group, the greater the precision ofcomparison.


CONTROL: Example 5 (Gastric freezing)

Gastric freezing is a treatment for ulcers in the upper intestine. A patientswallows a deflated balloon with tubes attached, then a refrigerated liquidis pumped through the balloon for one hour. The intent is that cooling thestomach will reduce its production of acid and so relieve ulcers.

An experiment reported in the Journal of American Medical Associationreported that gastric freezing did reduce acid production and relieve ulcerpain. The treatment was safe and easy and was widely used for severalyears.

Was there a flaw to the experiment?The experiment described above did not have a control treatment!

The observed patients’ response may have been due the placebo effect.So the effect of treatment may have been confounded with the placeboeffect.


CONTROL: Example 5 (Gastric freezing)

The end of the story ...A later experiment divided ulcer patients into two groups. One group wastreated by gastric freezing as before. The other group (control) received aplacebo treatment in which the liquid in the balloon was at bodytemperature rather than freezing.

The results ...

34% of the 82 patients in the treatment group improved.

38% of the 78 patients in the placebo group improved.

This and other properly designed experiments showed that gastricfreezing was no better than a placebo, and its use was abandoned.


BLINDING: Example 6 (Marijuana)

Figure: Cannabis Figure: Cigarette

A study of the effects of marijuana recruited young men who usedmarijuana. Some were randomly assigned to smoke marijuana cigarettes,while others were given placebo cigarettes.Was there a flaw to the experiment?

The blinding failed in this experiment! The control group recognizedthat their cigarettes were phony and complained loudly.

It may be quite common for blindness to fail because the subjects canrealize which treatment they are receiving.


Example 7 (Aspirin)

Question : Can Aspirin reduce the risk of heart attack in humans?

Sample : 22,071 male physicians between the ages of 40 and 84,were randomly assigned to one of two groups. One group took anordinary aspirin tablet every other day. The other group (control)took a placebo every other day.

Result : The rate of heart attacks in the group taking aspirindropped by 45% compared to the rate in the placebo group.

Conclusion: Taking aspirin CAUSES lower rate of heart attacks inhumans.

Figure: Aspirin Figure: The chemical form


SAMPLING: Example 8 (Internet Poll)

An Internet Poll asked people if they are Internet users or not. The resultof the poll was stated in the following form:

100% of people use the Internet!

Is something wrong here? Was there a flaw to the experiment?

Only Internet users could participate in the poll. The result of the samplecan not be extended to the population of all people. It is an example howa nonrepresentative sample can lead to a flawed conclusion.


Example 9 (Physical Exercises for the Elderly)

A study was designed to determine if a 50 days exercise program can increaseendurance in elderly people living in the retirement community. A sample of 100elderly people residing in the community have been taken. Selected people wererandomly distributed between two treatment groups: aerobic exercise andnormal (control) activity. The outcome of the study is the difference in distance aperson could walk in 5 minutes in the beginning of the study and at the end ofthe study.

Question: Should we worry about different age distributions of people in the twogroups? We know that functional ability usually decreases with every year after65, so different age distributions in the two groups may substantially obscure theresults of the study.

Answer: We should not really worry about the confounding factor in this

problem. This is an experimental study that used randomization.If the

randomization was done properly, the age distributions in two treatment groups

should be roughly the same.


chapter 2: observational studies - binghamton...

Documents