Bias, confounding and causation
Professor Tarek Tawfik Amin
Epidemiology and Public Health, Faculty of Medicine, Cairo University
Geneva Foundation for Medical Education and Training
Asian Pacific Organization for Cancer Prevention
[email protected] [email protected]
Basic Research Competency Program for Research Coordinators August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.
05/02/2023 Professor Tarek Tawfik 2
Types of variables
Causal model Study design Unit of measurement
Independent InterveningExtraneous Dependent
Active Attribute
Quantitative Qualitative
Continuous Categorical
Constants
Dichotomous
Polytomies
Can be manipulatedChanged or controlled
CharacteristicsAge, gender, genetics
Only one value or category
05/02/2023 Professor Tarek Tawfik 3
Smoking Assumed causeIndependent
variableDependent variable
Assumed effect Cancer
Affect the relationship
Age of the personExtent of smoking
Duration of smokingExercise
Extraneous (undesirable) variables- Modulate the cause-effect relationship (random error)
Intervening variables - Confounders A confounding variable is associated with the exposure and it affects the outcome, but it is not
an intermediate link in the chain of causation between exposure and outcome. (systematic error)
Occupation
05/02/2023 Professor Tarek Tawfik 4
Bias in observational designs Bias in research denotes deviation from the
truth. (when there is systematic difference between
theresults from study and the truth). All observational studies and badly done
randomized controlled trials have built-in bias.
The most often used classification of bias includes:
I. Selection bias,II. Information bias,III. Confounding.
05/02/2023 Professor Tarek Tawfik 5
I- Selection Bias Are the groups similar in all important respects?
Selection bias stems from absence of comparability between groups being studied.
In a cohort study, are participants in the exposed and unexposed groups similar in all important respects except for exposure?
In case-control study, are cases and controls, similar in all respects except for the disease in questions?
05/02/2023 Professor Tarek Tawfik 6
Selection BiasBias accompanying case-control study:
Berkson bias (admission-rate bias): knowledge of the exposure of interest might lead to an increased rate of admission to hospital. Admission preference of disease of interest.
Neyman bias (an incidence-prevalence bias): arises when a gap in time occurs between exposure and selection of study subjects. This bias crops up in studies of diseases that are quickly fatal, transient, or sub-clinical.
Myocardial infarction and its relation to snow shoveling.
05/02/2023 Professor Tarek Tawfik 7
Selection Bias
Unmasking bias: An exposure might lead to provoking of an
outcome. Estrogen replacement therapy and
symptomless endometrial cancer.
Non-respondent bias: In observational studies, non-respondents are
different from respondents. Smokers are less likely to return
questionnaires than are non-smokers or pipe and cigar smokers.
05/02/2023 Professor Tarek Tawfik 8
II- Information BiasHas the information been gathered in the same way?
Also known as observation, classification or measurement bias, results from incorrect determination of exposure or outcome or both.
Information should be gathered in the same way in any comparative study.
05/02/2023 Professor Tarek Tawfik 9
II- Information BiasHas the information been gathered in the same way?
Sources: Differentials in information gathering: (bedside for cases while using telephone for
control).
Diagnostic suspicion bias: (intensive search for HIV in drug addicts).
Family history bias: Medical information flows differently to
affected and non-affected family members (rheumatoid arthritis).
05/02/2023 Professor Tarek Tawfik 10
Information BiasRecall bias: cases are motivated to search their memories in order to identify the cause of their illness than the healthy people.Observer bias: one observer consistently under or over reports a particular variable. Meticulous observation of those who are exposed than the non-exposed.
05/02/2023 Professor Tarek Tawfik 11
Information Bias controlObserver and data gatherer should be blinded.Using a standardized instruments for data collection,Proper selection of the subjects are the possible maneuvers to lower the information bias.
05/02/2023 Professor Tarek Tawfik 12
III- Confounding.Is an external factor blurring the effect?
A confounding variable is associated with the exposure and it affects the outcome, but it is not an intermediate link in the chain of causation between exposure and outcome.
Myocardial infarctionOral contraceptive
Smoking
IUD insertion
STDs
Salpingitis
05/02/2023 Professor Tarek Tawfik 13
Confounding ‘Control’
Restriction (exclusion or specification): Enrollment with restricted selection criteria,
including non-smokers.Matching: A pair wise matching (for every case who
smokes, a control who smokes is found).Stratification: Used after completion of the study. Results
can be stratified by the levels of the confounding factor.
Multivariate analysis techniques: logistic regression, proportional hazard
regression, and others.
05/02/2023 Professor Tarek Tawfik 14
Judgment of AssociationsBogus, indirect, or real? Statistical associations do not imply causal
associations.
Types of associations: Bogus or spurious associations: Results of selection, information bias and
chance. Indirect association: Stems from confounding. Real associations.
05/02/2023 Professor Tarek Tawfik 15
Hill’s Criteria for Real Associations
Temporal sequence: Did exposure precede outcome? the cause
must antedate the outcome.Strength of association: How strong is the effect, measured as relative
risk (>3 ) or odds ratio (> 1)? Consistency of association: Has effect been seen by others? In different
populations with different study designs.
05/02/2023 Professor Tarek Tawfik 16
Hill’s Criteria for Real Associations
Biological gradient (dose-response relationship):
Does increased exposure result in more of the outcome?
Lung cancer and years of cigarette smoking.Specificity of association: Does exposure lead only to outcome? “weak criterion, few exposure will only lead
to the outcome”.Biological plausibility: biological
experimentation Does the association make sense? “weak criterion, limited by our lack of
knowledge”.
05/02/2023 Professor Tarek Tawfik 17
Hill’s Criteria for Real Associations
Coherence with existing knowledge:
Is the association consistent with available evidence?
The effect of cigarette smoke on the bronchial epithelium of animals is coherent with an increased risk of caner in human.
Experimental evidence: Has a randomized controlled study
been done?Analogy: Is the association similar to others?
05/02/2023 Professor Tarek Tawfik 18
Post test 1- Confounder: a. Associated with the exposure b. If known matching minimize it occurrence c. Directly linked the exposure and the outcomed. Modulate the cause-effect relationship 2- Extraneous (prognostic) factors.e. Cause variation in cause effect relationship f. A form of systematic errorg. Increase by increasing sample size h. None of the above. 3- Biasi. Affects all observational designs j. Can affect sound randomized experimentk. A built-in error in all research design
05/02/2023 Professor Tarek Tawfik 19
For each suggest a descriptor
Case-control Cohort Cross-sectional Hill’s criteria Experimental evidence
Experimental design
Validity of experimental
research Professor Tarek Tawfik
AminEpidemiology and Public Health, Faculty of
Medicine, Cairo University Geneva Foundation for Medical Education and
TrainingAsian Pacific Organization for Cancer
Prevention International Osteoporosis Foundation
Wiley Innovative Panel [email protected]
Basic Research Competency Program for Research Coordinators August 2015, MEDC, Faculty Of Medicine, Cairo University, Cairo, Egypt.
05/02/2023 Professor Tarek Tawfik 21
Experimental designs
05/02/2023 Professor Tarek Tawfik 22
Experimental designs Observational designs 1. The research
question. What if? What is?
2. Description - The researcher manipulates independent variables (e.g., type of treatment, teaching method) and measures dependent variables (Efficiency, disease control, symptoms, scales) in order to establish cause-and-effect relationships between them.
- The independent variables controlled or set by the researcher.
- The dependent variables measured by the researcher
- Just observation of the both dependent variables (outcome) and the predictors (exposures) without interfering
- Both are only measured (no control)
3. Errors and biases A “good” experiment is one that confines the variation of measurement scores to variation caused by the treatment itself.
- Many
Differences between observational and experimental designs
05/02/2023 Professor Tarek Tawfik 23
True experimental designs Experimental research is the only type which can establish cause-and-effect relationships between variables.
Two purposes: (especially the true experimental)1- Provide answer to research question 2- Control the difference (covariances)
05/02/2023 Professor Tarek Tawfik 24
• Experimental designs should be developed to ensure internal and external validity of the study.
05/02/2023 Professor Tarek Tawfik 25
The triad of tension (researcher quest) Maximizing variation (effect,
impact) of independent variable
(treatment) and the
outcome.
Controlling extraneous variables
(unwanted)Limit factor other than treatment
on the outcome
Minimizing random errors
[unreliable instrument
and assessment
)
Researcher
EliminationStratification
RandomizationMatchingStatistics (ANCOVA)
Controlled conditionsIncrease reliability measures
05/02/2023 Professor Tarek Tawfik 26
The ultimate question: Internal validity • Are the changes (improvement,
variation) of the dependent variable (outcome) caused by the independent variable (intervention) or they are caused by other factors (extraneous variables), which were not part of the study?
05/02/2023 Professor Tarek Tawfik 27
Validity Internal validity External validity The ultimate question
The changes in the dependent variable influenced only by intervention and not by other influences?
“How confidently can I generalize my experimental findings to the world?”
Causes Extraneous sources of variation not controlled.
Non representativeness).
Threats - History, - Maturation, - Testing, - Instrumentation, - Statistical regression,- Differential selection,- Experimental mortality,- Selection-maturation
interaction - The John Henry effect- Experimental treatment
diffusion
- The reactive effects of testing, - The interaction of treatment
and subject, - The interaction of testing and
subject,- Multiple treatment interference.
05/02/2023 Professor Tarek Tawfik 28
I- History • Events other than the treatment during the course of an experiment may influence treatment effect.• Its influence must occur during the experiment.• In conducting an experiment both groups are statistically similar in exposure to historical events. II- Maturation • Subjects change over the course of an experiment: physical, mental, emotional, or spiritual.• Perspective can change. • The natural process of human growth result in changes in post-test scores quite apart from the treatment.
Internal validity threats
05/02/2023 Professor Tarek Tawfik 29
III- Testing • In pre-test, treatment, and post-test design. • Using the same test both times, the group may show an
improvement simply because of their experience with the test. This is especially true when the treatment period is short and the tests are given within a short time.
• It is better to only give a post-test and randomly assign equal groups.
IV-Instrumentation • Using different tests for pre- and post-measurements, then
the change in pre- and post-scores may be due to differences between the tests rather than the treatment.
• The best remedy is randomization and a post-test only design.
05/02/2023 Professor Tarek Tawfik 30
V- Statistical regression [regression towards the mean]
• Statistical regression refers to the tendency of extreme scores, whether low or high, to move toward the average on a second testing.
• Subjects who score very high or very low on one test will probably score less high or low when they take the test again. That is, they regress toward the mean.
• Do not study groups formed from extreme scores. Study the full range of scores.
05/02/2023 Professor Tarek Tawfik 31
VI- Differential selection • If we select groups for “treatment”
and “control” differently, then the results may be due to the differences between groups before treatment.
• Randomization solves this problem
05/02/2023 Professor Tarek Tawfik 32
VII- Experimental mortality • Also called “attrition,” refers to the loss of
subjects from the experiment.• If there is a systematic bias in the subjects who
drop out, then posttest scores will be biased. • If subjects drop out because they are
aware that they’re not improving as they should, then the post-test scores of those completed the treatment will be positively biased.
• Your results will appear more favorable than they really are.
05/02/2023 Professor Tarek Tawfik 33
VIII- John Henry Effect • John Henry, the legendary “steel driving’ man,”
set himself to prove he could drive railroad spikes faster and better than the newly invented steam-powered machine driver. He exerted himself so much in trying to outdo the "experimental" condition that he died of a ruptured heart.
• If subjects in a control group find out they are in competition with those in an experimental treatment, they tend to work harder.
• The differences between control and treatment groups are decreased, minimizing the perceived treatment effect.
05/02/2023 Professor Tarek Tawfik 34
IX- Treatment diffusion • If subjects in the control group perceive the
treatment as very desirable, they may try to find out what’s being done.
• Over the course of the experiment, some of the materials of the treatment group may be borrowed by the control group members. Over time, the treatment “diffuses” to the control group, minimizing the treatment effect.
• This often happens when the groups are in close proximity.
• Both the John Henry Effect and Treatment Diffusion can be controlled if experimental and control groups are isolated.
05/02/2023 Professor Tarek Tawfik 35
External validity threats I- Reactive effects of testing
• Subjects may respond differently to experimental treatments merely because they are being tested.
• Since the population at large is not tested, experimental effects may be due to the testing procedures rather than the treatment itself. This reduces generalizability.
• Pretest sensitization: Subjects who take a pre-test are sensitized to the following treatment (creating a different population compared to the untested population): DO NOT USE PRETEST.
• Post-test sensitization: Posttest can be a learning experience that helps subjects to “put all the pieces together.”
05/02/2023 Professor Tarek Tawfik 36
II- Treatment and Subject Interaction• Subjects in a sample may react to the
experimental treatment in ways that are hard to predict.
• This limits the ability of the researcher to generalize findings outside the experiment itself.
• If there is a systematic bias in a sample, then treatment effects may be different when applied to a different sample.
05/02/2023 Professor Tarek Tawfik 37
III- Testing and Subject Interaction• Subjects may react to testing in ways that are
hard to predict. • Test anxiety or “test-wiseness” in a sample, the
treatment effects will be different when applied to a different sample.
IV- Multiple Treatment Effect• An experiment exposes subjects to several
treatments and test scores show that treatment X produced the best results; one cannot declare treatment X the best. It may be the combination of the treatments that led to the results.
• Treatment X, given alone, may produce different results.
05/02/2023 Professor Tarek Tawfik 38
True experimental designs
Experimental research designs
39
Pretest-posttest control group
Intervention
group
Control group
Randomization
Baseline assessment
Pretesting
Treatment Intervention
PlaceboStandard care
Intervention
group
Control group
Post intervention assessm
ent Post testing
t-test independent(or equivalent)
t-paired or equivalent Pretest sensitizationPossible interaction (pretest and treatment)
05/02/2023 Professor Tarek Tawfik 40
Posttest ONLY control group
Intervention
group
Control group
Randomization
Treatment Intervention
PlaceboStandard care
Intervention
group
Control group
Post intervention assessm
ent Post testing
t-test independent(or equivalent)
Professor Tarek Tawfik 41
Solomon Four Groups Design
Intervention 1
Control 1
Intervention 2
Control 2Rand
omiza
tion
Base
line
asse
ssm
ent
Pret
estin
g Intervention 1
Control 1
Intervention 2
Control 2
TreatmentIntervention
TreatmentIntervention
Placebostandard
Placebostandard
Post intervention assessment
Post testing
t-test: effect of the pre-test Effect of treatment
ANOVA
Several ways to analyze dataControl extraneous variables Large sample size
05/02/2023 Professor Tarek Tawfik 42
The quasi experimental designs
• When randomization is difficult or can’t be done
05/02/2023 Professor Tarek Tawfik 43
Time series
Group
Baseline Assessment
Test 1 Test 2 Test 3Test 4 Test 5 Test 7 Test 8
Group
Outcome Assessment
Test 6
InterventionTreatment
No randomization
Compare mean scores (pre-post intervention)
-No control group- Complex data analysis-Instrumental problem- Possible Reactive Effect of Repeated Testing.
44
Non equivalent Control Group Design
Group 1
Group 2
Baseline data
Pretesting
Intervention Treatment Group 1
Group 2
Outcome dataPost
testing
Non randomizedNon-equal groups
ANCOVA is applicable
No control for:-Selection-Maturation interaction-Statistical regression -Suffers from pre-test sensitization Professor Tarek Tawfik
05/02/2023 Professor Tarek Tawfik 45
Counter balanced design
Group 2
Group 1 Treatment 1Assessment 1
Treatment 1Assessment 1
Treatment 2Assessment 2
Treatment 2Assessment 2
Group 2
Group 1Time
Latin Square Analysis
Selection Maturation Interaction Multiple treatments (interventions) effect [external validity]
05/02/2023 Professor Tarek Tawfik 46
Pre-experimental design
• Pre-experimental designs should not be considered true experiments, and are not appropriate for formal research.
• Data collected with these designs is highly suspect.
05/02/2023 Professor Tarek Tawfik 47
One shot case study
Group InterventionTreatment
Group
Post test Assessment
- None of the sources of internal or external invalidity are controlled. - Suffers history, maturation, regression, and differential selection. - Suffers from the external source of “treatment and subject.” - The design is useless for most practical purposes
Only descriptive analysis (no comparison group)
05/02/2023 Professor Tarek Tawfik 48
One-Group Pretest/Posttest
Group Group Intervention Treatment
Baseline assessmentPretesting
Outcome assessmentPost-testing t-Paired or Wilcoxin Rank
- History, maturation, testing, instrumentation, and selection-maturation interaction. - The reactive effects of pre-and post- tests and treatment-subject are external sources of invalidity
05/02/2023 Professor Tarek Tawfik 49
Static-Group comparison
Group 1
Group 2 Group 2
Group 1
InterventionTreatment
OutcomeAssessment Post testing
The results of statistical analysis are meaningless since there is no assurance that groups were the same at the beginning of the treatment.
- This design suffers most from selection, attrition, and selection-maturation interaction problems. - It fails to control the external invalidity source of treatment and subject.
05/02/2023 Professor Tarek Tawfik 50
Post testing
1- In experimental design, the researcher tries to:a. Control independent variableb. Measure extraneous variables c. Observe confounders 2- Internal validity is violated throughd. Sound methodology and design e. Change in population geography f. Lack of representativeness3- Long duration experimentation may suffer from the
following type of internal validity threats:g. History h. Maturation i. Selection
05/02/2023 Professor Tarek Tawfik 51
Multiple interventions Pre-post design Quasi randomized designs Post test design Time series Counter balanced design
State the name of validity threats
Group activity: criticize validity of the use design
• Hydrocortisone Cream to Reduce Perineal Pain after Vaginal Birth: A Randomized Controlled Trial. Abstract
• PURPOSE: To determine if the use of hydrocortisone cream decreases perineal pain in the immediate postpartum period.
• STUDY DESIGN AND METHODS: This was a randomized controlled trial (RCT), crossover study design, with each participant serving as their own control. Participants received three different methods for perineal pain management at three sequential perineal pain treatments after birth: two topical creams (corticosteroid; placebo) and a control treatment (no cream application). Treatment order was randomly assigned, with participants and investigators blinded to cream type. The primary dependent variable was the change in perineal pain levels (posttest minus pretest pain levels) immediately before and 30 to 60 minutes after perineal pain treatments. Data were analyzed with analysis of variance, with p < 0.05 considered significant.
• RESULTS:A total of 27 participants completed all three perineal pain treatments over a 12-hour period. A reduction in pain was found after application of both the topical creams, with average perineal pain change scores of -4.8 ± 8.4 mm after treatment with hydrocortisone cream (N = 27) and -6.7 ± 13.0 mm after treatment with the placebo cream (N = 27). Changes in pain scores with no cream application were 1.2 ± 10.5 mm (N = 27). Analysis of variance found a significant difference between treatment groups (F2,89 = 3.6, p = 0.03), with both cream treatments having significantly better pain reduction than the control, no cream treatment (hydrocortisone vs. no cream, p = 0.04; placebo cream vs. no cream, p = 0.01). There were no differences in perineal pain reduction between the two cream treatments (p = .54).
• CLINICAL IMPLICATIONS: This RCT found that the application of either hydrocortisone cream or placebo cream provided significantly better pain relief than no cream
application.
52
05/02/2023 Professor Tarek Tawfik 53
• Intramyocardial injection of hydrogel with high interstitial spread does not impact action potential propagation.
Abstract• Injectable biomaterials have been evaluated as potential new therapies for myocardial infarction (MI) and
heart failure. These materials have improved left ventricular (LV) geometry and ejection fraction, yet there remain concerns that biomaterial injection may create a substrate for arrhythmia. Since studies of this risk are lacking, we utilized optical mapping to assess the effects of biomaterial injection and interstitial spread on cardiac electrophysiology. Healthy and infarcted rat hearts were injected with a model poly(ethylene glycol) hydrogel with varying degrees of interstitial spread. Activation maps demonstrated delayed propagation of action potentials across the LV epicardium in the hydrogel-injected group when compared to saline and no-injection groups. However, the degree of the electrophysiological changes depended on the spread characteristics of the hydrogel, such that hearts injected with highly spread hydrogels showed no conduction abnormalities. Conversely, the results of this study indicate that injection of a hydrogel exhibiting minimal interstitial spread may create a substrate for arrhythmia shortly after injection by causing LV activation delays and reducing gap junction density at the site of injection. Thus, this work establishes site of delivery and interstitial spread characteristics as important factors in the future design and use of biomaterial therapies for MI treatment.STATEMENT OF SIGNIFICANCE:Biomaterials for treating myocardial infarction have become an increasingly popular area of research. Within the past few years, this work has transitioned to some large animals models, and Phase I & II clinical trials. While these materials have preserved/improved cardiac function the effect of these materials on arrhythmogenesis, which is of considerable concern when injecting anything into the heart, has yet to be understood. Our manuscript is therefore a first of its kind in that it directly examines the potential of an injectable material to create a substrate for arrhythmias. This work suggests that site of delivery and distribution in the tissue are important criteria in the design and development of future biomaterial therapies for myocardial infarction.
05/02/2023 Professor Tarek Tawfik 54
• Protective effects of fish oil, allopurinol, and verapamil on hepatic ischemia-reperfusion injury in rats.
Abstract• BACKGROUND:The major aim of this work was to study the protective effects of fish oil (FO),
allopurinol, and verapamil on hepatic ischemia-reperfusion (IR)-induced injury in experimental rats.MATERIALS AND METHODS:Sixty male Wistar albino rats were randomly assigned to six groups of 10 rats each. Group 1 served as a negative control. Group 2 served as hepatic IR control injury. Groups 3, 4, 5, and 6 received N-acetylcysteine (standard), FO, allopurinol, and verapamil, respectively, for 3 consecutive days prior to ischemia. All animals were fasted for 12 h, anesthetized and underwent midline laparotomy. The portal triads were clamped by mini-artery clamp for 30 min followed by reperfusion for 30 min. Blood samples were withdrawn for estimation of serum alanine transaminase (ALT) and aspartate transaminase (AST) activities as well as hepatic thiobarbituric acid reactive substances, reduced glutathione, myeloperoxidase, and total nitrate/nitrite levels, in addition to histopathological examination.RESULTS:Fish oil, allopurinol, and verapamil reduced hepatic IR injury as evidenced by significant reduction in serum ALT and AST enzyme activities. FO and verapamil markedly reduced oxidative stress as compared to control IR injury. Levels of inflammatory biomarkers in liver were also reduced after treatment with FO, allopurinol, or verapamil. In accordance, a marked improvement of histopathological findings was observed with all of the three treatments.CONCLUSION:The findings of this study prove the benefits of FO, allopurinol, and verapamil on hepatic IR-induced liver injury and are promising for further clinical trials.
05/02/2023 Professor Tarek Tawfik 55
Thank you