bias,confounding, causation and experimental designs

Bias, confounding and causation

Professor Tarek Tawfik Amin

Epidemiology and Public Health, Faculty of Medicine, Cairo University

Geneva Foundation for Medical Education and Training

Asian Pacific Organization for Cancer Prevention

[email protected] [email protected]

Basic Research Competency Program for Research Coordinators August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.

mailto:[email protected]


05/02/2023 Professor Tarek Tawfik 2

Types of variables

Causal model Study design Unit of measurement

Independent InterveningExtraneous Dependent

Active Attribute

Quantitative Qualitative

Continuous Categorical

Constants

Dichotomous

Polytomies

Can be manipulatedChanged or controlled

CharacteristicsAge, gender, genetics

Only one value or category


Smoking Assumed causeIndependent

variableDependent variable

Assumed effect Cancer

Affect the relationship

Age of the personExtent of smoking

Duration of smokingExercise

Extraneous (undesirable) variables- Modulate the cause-effect relationship (random error)

Intervening variables - Confounders A confounding variable is associated with the exposure and it affects the outcome, but it is not

an intermediate link in the chain of causation between exposure and outcome. (systematic error)

Occupation


Bias in observational designs Bias in research denotes deviation from the

truth. (when there is systematic difference between

theresults from study and the truth). All observational studies and badly done

randomized controlled trials have built-in bias.

The most often used classification of bias includes:

I. Selection bias,II. Information bias,III. Confounding.


I- Selection Bias Are the groups similar in all important respects?

Selection bias stems from absence of comparability between groups being studied.

In a cohort study, are participants in the exposed and unexposed groups similar in all important respects except for exposure?

In case-control study, are cases and controls, similar in all respects except for the disease in questions?


Selection BiasBias accompanying case-control study:

Berkson bias (admission-rate bias): knowledge of the exposure of interest might lead to an increased rate of admission to hospital. Admission preference of disease of interest.

Neyman bias (an incidence-prevalence bias): arises when a gap in time occurs between exposure and selection of study subjects. This bias crops up in studies of diseases that are quickly fatal, transient, or sub-clinical.

Myocardial infarction and its relation to snow shoveling.


Selection Bias

Unmasking bias: An exposure might lead to provoking of an

outcome. Estrogen replacement therapy and

symptomless endometrial cancer.

Non-respondent bias: In observational studies, non-respondents are

different from respondents. Smokers are less likely to return

questionnaires than are non-smokers or pipe and cigar smokers.


II- Information BiasHas the information been gathered in the same way?

Also known as observation, classification or measurement bias, results from incorrect determination of exposure or outcome or both.

Information should be gathered in the same way in any comparative study.


II- Information BiasHas the information been gathered in the same way?

Sources: Differentials in information gathering: (bedside for cases while using telephone for

control).

Diagnostic suspicion bias: (intensive search for HIV in drug addicts).

Family history bias: Medical information flows differently to

affected and non-affected family members (rheumatoid arthritis).


Information BiasRecall bias: cases are motivated to search their memories in order to identify the cause of their illness than the healthy people.Observer bias: one observer consistently under or over reports a particular variable. Meticulous observation of those who are exposed than the non-exposed.


Information Bias controlObserver and data gatherer should be blinded.Using a standardized instruments for data collection,Proper selection of the subjects are the possible maneuvers to lower the information bias.


III- Confounding.Is an external factor blurring the effect?

A confounding variable is associated with the exposure and it affects the outcome, but it is not an intermediate link in the chain of causation between exposure and outcome.

Myocardial infarctionOral contraceptive

Smoking

IUD insertion

STDs

Salpingitis


Confounding ‘Control’

Restriction (exclusion or specification): Enrollment with restricted selection criteria,

including non-smokers.Matching: A pair wise matching (for every case who

smokes, a control who smokes is found).Stratification: Used after completion of the study. Results

can be stratified by the levels of the confounding factor.

Multivariate analysis techniques: logistic regression, proportional hazard

regression, and others.


Judgment of AssociationsBogus, indirect, or real? Statistical associations do not imply causal

associations.

Types of associations: Bogus or spurious associations: Results of selection, information bias and

chance. Indirect association: Stems from confounding. Real associations.


Hill’s Criteria for Real Associations

Temporal sequence: Did exposure precede outcome? the cause

must antedate the outcome.Strength of association: How strong is the effect, measured as relative

risk (>3 ) or odds ratio (> 1)? Consistency of association: Has effect been seen by others? In different

populations with different study designs.



Biological gradient (dose-response relationship):

Does increased exposure result in more of the outcome?

Lung cancer and years of cigarette smoking.Specificity of association: Does exposure lead only to outcome? “weak criterion, few exposure will only lead

to the outcome”.Biological plausibility: biological

experimentation Does the association make sense? “weak criterion, limited by our lack of

knowledge”.



Coherence with existing knowledge:

Is the association consistent with available evidence?

The effect of cigarette smoke on the bronchial epithelium of animals is coherent with an increased risk of caner in human.

Experimental evidence: Has a randomized controlled study

been done?Analogy: Is the association similar to others?


Post test 1- Confounder: a. Associated with the exposure b. If known matching minimize it occurrence c. Directly linked the exposure and the outcomed. Modulate the cause-effect relationship 2- Extraneous (prognostic) factors.e. Cause variation in cause effect relationship f. A form of systematic errorg. Increase by increasing sample size h. None of the above. 3- Biasi. Affects all observational designs j. Can affect sound randomized experimentk. A built-in error in all research design


For each suggest a descriptor

Case-control Cohort Cross-sectional Hill’s criteria Experimental evidence

Experimental design

Validity of experimental

research Professor Tarek Tawfik

AminEpidemiology and Public Health, Faculty of

Medicine, Cairo University Geneva Foundation for Medical Education and

TrainingAsian Pacific Organization for Cancer

Prevention International Osteoporosis Foundation

Wiley Innovative Panel [email protected]

[email protected]

Basic Research Competency Program for Research Coordinators August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.




Experimental designs


Experimental designs Observational designs 1. The research

question. What if? What is?

2. Description - The researcher manipulates independent variables (e.g., type of treatment, teaching method) and measures dependent variables (Efficiency, disease control, symptoms, scales) in order to establish cause-and-effect relationships between them.

- The independent variables controlled or set by the researcher.

- The dependent variables measured by the researcher

- Just observation of the both dependent variables (outcome) and the predictors (exposures) without interfering

- Both are only measured (no control)

3. Errors and biases A “good” experiment is one that confines the variation of measurement scores to variation caused by the treatment itself.

- Many

Differences between observational and experimental designs


True experimental designs Experimental research is the only type which can establish cause-and-effect relationships between variables.

Two purposes: (especially the true experimental)1- Provide answer to research question 2- Control the difference (covariances)


• Experimental designs should be developed to ensure internal and external validity of the study.


The triad of tension (researcher quest) Maximizing variation (effect,

impact) of independent variable

(treatment) and the

outcome.

Controlling extraneous variables

(unwanted)Limit factor other than treatment

on the outcome

Minimizing random errors

[unreliable instrument

and assessment

)

Researcher

EliminationStratification

RandomizationMatchingStatistics (ANCOVA)

Controlled conditionsIncrease reliability measures


The ultimate question: Internal validity • Are the changes (improvement,

variation) of the dependent variable (outcome) caused by the independent variable (intervention) or they are caused by other factors (extraneous variables), which were not part of the study?


Validity Internal validity External validity The ultimate question

The changes in the dependent variable influenced only by intervention and not by other influences?

“How confidently can I generalize my experimental findings to the world?”

Causes Extraneous sources of variation not controlled.

Non representativeness).

Threats - History, - Maturation, - Testing, - Instrumentation, - Statistical regression,- Differential selection,- Experimental mortality,- Selection-maturation

interaction - The John Henry effect- Experimental treatment

diffusion

- The reactive effects of testing, - The interaction of treatment

and subject, - The interaction of testing and

subject,- Multiple treatment interference.


I- History • Events other than the treatment during the course of an experiment may influence treatment effect.• Its influence must occur during the experiment.• In conducting an experiment both groups are statistically similar in exposure to historical events. II- Maturation • Subjects change over the course of an experiment: physical, mental, emotional, or spiritual.• Perspective can change. • The natural process of human growth result in changes in post-test scores quite apart from the treatment.

Internal validity threats


III- Testing • In pre-test, treatment, and post-test design. • Using the same test both times, the group may show an

improvement simply because of their experience with the test. This is especially true when the treatment period is short and the tests are given within a short time.

• It is better to only give a post-test and randomly assign equal groups.

IV-Instrumentation • Using different tests for pre- and post-measurements, then

the change in pre- and post-scores may be due to differences between the tests rather than the treatment.

• The best remedy is randomization and a post-test only design.


V- Statistical regression [regression towards the mean]

• Statistical regression refers to the tendency of extreme scores, whether low or high, to move toward the average on a second testing.

• Subjects who score very high or very low on one test will probably score less high or low when they take the test again. That is, they regress toward the mean.

• Do not study groups formed from extreme scores. Study the full range of scores.


VI- Differential selection • If we select groups for “treatment”

and “control” differently, then the results may be due to the differences between groups before treatment.

• Randomization solves this problem


VII- Experimental mortality • Also called “attrition,” refers to the loss of

subjects from the experiment.• If there is a systematic bias in the subjects who

drop out, then posttest scores will be biased. • If subjects drop out because they are

aware that they’re not improving as they should, then the post-test scores of those completed the treatment will be positively biased.

• Your results will appear more favorable than they really are.


VIII- John Henry Effect • John Henry, the legendary “steel driving’ man,”

set himself to prove he could drive railroad spikes faster and better than the newly invented steam-powered machine driver. He exerted himself so much in trying to outdo the "experimental" condition that he died of a ruptured heart.

• If subjects in a control group find out they are in competition with those in an experimental treatment, they tend to work harder.

• The differences between control and treatment groups are decreased, minimizing the perceived treatment effect.


IX- Treatment diffusion • If subjects in the control group perceive the

treatment as very desirable, they may try to find out what’s being done.

• Over the course of the experiment, some of the materials of the treatment group may be borrowed by the control group members. Over time, the treatment “diffuses” to the control group, minimizing the treatment effect.

• This often happens when the groups are in close proximity.

• Both the John Henry Effect and Treatment Diffusion can be controlled if experimental and control groups are isolated.


External validity threats I- Reactive effects of testing

• Subjects may respond differently to experimental treatments merely because they are being tested.

• Since the population at large is not tested, experimental effects may be due to the testing procedures rather than the treatment itself. This reduces generalizability.

• Pretest sensitization: Subjects who take a pre-test are sensitized to the following treatment (creating a different population compared to the untested population): DO NOT USE PRETEST.

• Post-test sensitization: Posttest can be a learning experience that helps subjects to “put all the pieces together.”


II- Treatment and Subject Interaction• Subjects in a sample may react to the

experimental treatment in ways that are hard to predict.

• This limits the ability of the researcher to generalize findings outside the experiment itself.

• If there is a systematic bias in a sample, then treatment effects may be different when applied to a different sample.


III- Testing and Subject Interaction• Subjects may react to testing in ways that are

hard to predict. • Test anxiety or “test-wiseness” in a sample, the

treatment effects will be different when applied to a different sample.

IV- Multiple Treatment Effect• An experiment exposes subjects to several

treatments and test scores show that treatment X produced the best results; one cannot declare treatment X the best. It may be the combination of the treatments that led to the results.

• Treatment X, given alone, may produce different results.


True experimental designs

Experimental research designs

39

Pretest-posttest control group

Intervention

group

Control group

Randomization

Baseline assessment

Pretesting

Treatment Intervention

PlaceboStandard care

Intervention

group

Control group

Post intervention assessm

ent Post testing

t-test independent(or equivalent)

t-paired or equivalent Pretest sensitizationPossible interaction (pretest and treatment)


Posttest ONLY control group

Intervention

group

Control group

Randomization

Treatment Intervention

PlaceboStandard care

Intervention

group

Control group

Post intervention assessm

ent Post testing

t-test independent(or equivalent)

Professor Tarek Tawfik 41

Solomon Four Groups Design

Intervention 1

Control 1

Intervention 2

Control 2Rand

omiza

tion

Base

line

asse

ssm

ent

Pret

estin

g Intervention 1

Control 1

Intervention 2

Control 2

TreatmentIntervention

TreatmentIntervention

Placebostandard

Placebostandard

Post intervention assessment

Post testing

t-test: effect of the pre-test Effect of treatment

ANOVA

Several ways to analyze dataControl extraneous variables Large sample size


The quasi experimental designs

• When randomization is difficult or can’t be done


Time series

Group

Baseline Assessment

Test 1 Test 2 Test 3Test 4 Test 5 Test 7 Test 8

Group

Outcome Assessment

Test 6

InterventionTreatment

No randomization

Compare mean scores (pre-post intervention)

-No control group- Complex data analysis-Instrumental problem- Possible Reactive Effect of Repeated Testing.

44

Non equivalent Control Group Design

Group 1

Group 2

Baseline data

Pretesting

Intervention Treatment Group 1

Group 2

Outcome dataPost

testing

Non randomizedNon-equal groups

ANCOVA is applicable

No control for:-Selection-Maturation interaction-Statistical regression -Suffers from pre-test sensitization Professor Tarek Tawfik


Counter balanced design

Group 2

Group 1 Treatment 1Assessment 1

Treatment 1Assessment 1



Group 2

Group 1Time

Latin Square Analysis

Selection Maturation Interaction Multiple treatments (interventions) effect [external validity]


Pre-experimental design

• Pre-experimental designs should not be considered true experiments, and are not appropriate for formal research.

• Data collected with these designs is highly suspect.


One shot case study

Group InterventionTreatment

Group

Post test Assessment

- None of the sources of internal or external invalidity are controlled. - Suffers history, maturation, regression, and differential selection. - Suffers from the external source of “treatment and subject.” - The design is useless for most practical purposes

Only descriptive analysis (no comparison group)


One-Group Pretest/Posttest

Group Group Intervention Treatment

Baseline assessmentPretesting

Outcome assessmentPost-testing t-Paired or Wilcoxin Rank

- History, maturation, testing, instrumentation, and selection-maturation interaction. - The reactive effects of pre-and post- tests and treatment-subject are external sources of invalidity


Static-Group comparison

Group 1

Group 2 Group 2

Group 1

InterventionTreatment

OutcomeAssessment Post testing

The results of statistical analysis are meaningless since there is no assurance that groups were the same at the beginning of the treatment.

- This design suffers most from selection, attrition, and selection-maturation interaction problems. - It fails to control the external invalidity source of treatment and subject.


Post testing

1- In experimental design, the researcher tries to:a. Control independent variableb. Measure extraneous variables c. Observe confounders 2- Internal validity is violated throughd. Sound methodology and design e. Change in population geography f. Lack of representativeness3- Long duration experimentation may suffer from the

following type of internal validity threats:g. History h. Maturation i. Selection


Multiple interventions Pre-post design Quasi randomized designs Post test design Time series Counter balanced design

State the name of validity threats

Group activity: criticize validity of the use design

• Hydrocortisone Cream to Reduce Perineal Pain after Vaginal Birth: A Randomized Controlled Trial. Abstract

• PURPOSE: To determine if the use of hydrocortisone cream decreases perineal pain in the immediate postpartum period.

• STUDY DESIGN AND METHODS: This was a randomized controlled trial (RCT), crossover study design, with each participant serving as their own control. Participants received three different methods for perineal pain management at three sequential perineal pain treatments after birth: two topical creams (corticosteroid; placebo) and a control treatment (no cream application). Treatment order was randomly assigned, with participants and investigators blinded to cream type. The primary dependent variable was the change in perineal pain levels (posttest minus pretest pain levels) immediately before and 30 to 60 minutes after perineal pain treatments. Data were analyzed with analysis of variance, with p < 0.05 considered significant.

• RESULTS:A total of 27 participants completed all three perineal pain treatments over a 12-hour period. A reduction in pain was found after application of both the topical creams, with average perineal pain change scores of -4.8 ± 8.4 mm after treatment with hydrocortisone cream (N = 27) and -6.7 ± 13.0 mm after treatment with the placebo cream (N = 27). Changes in pain scores with no cream application were 1.2 ± 10.5 mm (N = 27). Analysis of variance found a significant difference between treatment groups (F2,89 = 3.6, p = 0.03), with both cream treatments having significantly better pain reduction than the control, no cream treatment (hydrocortisone vs. no cream, p = 0.04; placebo cream vs. no cream, p = 0.01). There were no differences in perineal pain reduction between the two cream treatments (p = .54).

• CLINICAL IMPLICATIONS: This RCT found that the application of either hydrocortisone cream or placebo cream provided significantly better pain relief than no cream

application.

52


• Intramyocardial injection of hydrogel with high interstitial spread does not impact action potential propagation.

Abstract• Injectable biomaterials have been evaluated as potential new therapies for myocardial infarction (MI) and

heart failure. These materials have improved left ventricular (LV) geometry and ejection fraction, yet there remain concerns that biomaterial injection may create a substrate for arrhythmia. Since studies of this risk are lacking, we utilized optical mapping to assess the effects of biomaterial injection and interstitial spread on cardiac electrophysiology. Healthy and infarcted rat hearts were injected with a model poly(ethylene glycol) hydrogel with varying degrees of interstitial spread. Activation maps demonstrated delayed propagation of action potentials across the LV epicardium in the hydrogel-injected group when compared to saline and no-injection groups. However, the degree of the electrophysiological changes depended on the spread characteristics of the hydrogel, such that hearts injected with highly spread hydrogels showed no conduction abnormalities. Conversely, the results of this study indicate that injection of a hydrogel exhibiting minimal interstitial spread may create a substrate for arrhythmia shortly after injection by causing LV activation delays and reducing gap junction density at the site of injection. Thus, this work establishes site of delivery and interstitial spread characteristics as important factors in the future design and use of biomaterial therapies for MI treatment.STATEMENT OF SIGNIFICANCE:Biomaterials for treating myocardial infarction have become an increasingly popular area of research. Within the past few years, this work has transitioned to some large animals models, and Phase I & II clinical trials. While these materials have preserved/improved cardiac function the effect of these materials on arrhythmogenesis, which is of considerable concern when injecting anything into the heart, has yet to be understood. Our manuscript is therefore a first of its kind in that it directly examines the potential of an injectable material to create a substrate for arrhythmias. This work suggests that site of delivery and distribution in the tissue are important criteria in the design and development of future biomaterial therapies for myocardial infarction.


• Protective effects of fish oil, allopurinol, and verapamil on hepatic ischemia-reperfusion injury in rats.

Abstract• BACKGROUND:The major aim of this work was to study the protective effects of fish oil (FO),

allopurinol, and verapamil on hepatic ischemia-reperfusion (IR)-induced injury in experimental rats.MATERIALS AND METHODS:Sixty male Wistar albino rats were randomly assigned to six groups of 10 rats each. Group 1 served as a negative control. Group 2 served as hepatic IR control injury. Groups 3, 4, 5, and 6 received N-acetylcysteine (standard), FO, allopurinol, and verapamil, respectively, for 3 consecutive days prior to ischemia. All animals were fasted for 12 h, anesthetized and underwent midline laparotomy. The portal triads were clamped by mini-artery clamp for 30 min followed by reperfusion for 30 min. Blood samples were withdrawn for estimation of serum alanine transaminase (ALT) and aspartate transaminase (AST) activities as well as hepatic thiobarbituric acid reactive substances, reduced glutathione, myeloperoxidase, and total nitrate/nitrite levels, in addition to histopathological examination.RESULTS:Fish oil, allopurinol, and verapamil reduced hepatic IR injury as evidenced by significant reduction in serum ALT and AST enzyme activities. FO and verapamil markedly reduced oxidative stress as compared to control IR injury. Levels of inflammatory biomarkers in liver were also reduced after treatment with FO, allopurinol, or verapamil. In accordance, a marked improvement of histopathological findings was observed with all of the three treatments.CONCLUSION:The findings of this study prove the benefits of FO, allopurinol, and verapamil on hepatic IR-induced liver injury and are promising for further clinical trials.


Thank you

bias,confounding, causation and experimental designs

Health & Medicine