fundamentals of biostatistics lecture 2 1.clinical trials 2.validity/reliability 3.assessing...

Fundamentals of Biostatistics

Lecture 2

1. Clinical Trials2. Validity/Reliability3. Assessing Evidence

Randomized Clinical Trials (RCT)

Randomization first used by RA Fisher in agriculture expts in 1920s

First clinical trial using randomization in 1931 by Anderson on use of sanocrysin on TB patients. Also, first trial using blinding.

Placebo first used in RCT in 1938 in cold vaccine trial.


Fundamental Point:

A properly planned clinical trial is a powerful expt’l technique for assessing effectiveness of a drug or intervention.

Defn: A clinical trial is a prospective study comparing the effect of intervention(s) against a control in humans.


Phases of clinical trials:

Phase I: Determine tolerance of a compound in humans (i.e., how large a dose can be given until unacceptable toxicity?).

Phase II: Evaluation of biologic activity or effect, and estimate rate of adverse events.

Phase III: Definitive comparative trial. Designed to determine effectiveness and its role in clinical practice.


Terminology

Efficacy: How well an intervention works in an ideal setting.

Effectiveness: How well an intervention works in actual practice.

Phase IV: Evaluation of long-term saftey of an intervention believed to be effective in phase III trials.


Why are clinical trials needed?

Because evaluating the effectiveness of a treatment using uncontrolled observations is very difficult, since other factors affecting treatment outcome may not be balanced in treatment groups.


Advantages of RCT:

Groups more comparable b/c confounding variables are

balanced

Ability to detect small effects

Most stat tests based on assumption of random allocation of pts to trt groups (validity)


Disadvantages of RCT:

Expensive and time consuming

Subject pool may not be representative

Effective treatment may be withheld

Expose pts to dangerous drugs


What is the question?

Each clinical trial must have a primary question.

The primary, as well as secondary questions, must be carefully selected, clearly defined, clinically relevant, and stated in advance.

This includes the choice of response variable (ie., true clinical endpoint or surrogate endpoint).


Study population

The study population is a subset of the population with the condition or characteristics of interest defined by the eligibility criteria.

This population should be defined in advance, stating unambiguous inclusion (eligibility) criteria.

The impact that the inclusion criteria will have on study design, ability to generalize, and participant recruitment must be considered.


Randomized control studies

comparative studies with an intervention group and a control group

the assignment of the participant to a group is determined by the formal process of randomization.


Randomized control studies . . .

Sound scientific investigation almost always demands that a control group be used against which the new intervention can be compared.

Randomization is the preferred way of assigning participants to control and intervention groups.


Randomization

Randomization tends to produce study groups that are:

1. comparable with respect to known and unknown risk factors

2. Removes investigator bias in the allocation and treatment of patients

3. Guarantees statistical tests will have valid significance levels.


Randomization . . .

Simple randomization is easiest to understand and use, but randomization can also be blocked, stratified, adaptive, etc.

Randomization is best accomplished by an independent central statistical unit.


Blinding (Masking)

Ideally, a clinical trial should use a double-blind design to avoid potential problems with bias during data collection and assessment.

If using a double-blind design is not feasible, a single-blind design and other measures to reduce bias should be used.


Baseline Assessment Relevant baseline data should be

measured in all study participants before the start of intervention.

These baseline measurements can be used to determine eligibility (if obtained prior to assignment to treatment group).

Can be also used to determine if the randomization produced identical groups (if not, can be used as covariates for adjustment of imbalance).


Data Analysis Excluding randomized participants or

observed outcomes from analysis and sub-grouping on the basis of outcome or response variables (including non-adherence) can lead to biased results of unknown magnitude or direction.

Including all randomized participants in the analysis, in the group they were assigned, is the intent-to-treat principle.

The intent-to-treat analysis is viewed as the most valid approach (least susceptible to bias).


Intent-to-Treat Principle

Once randomized . . .

. . . Always analyzed!


Covariate Adjustment

While randomization eliminates bias, it does not guarantee comparable baseline characteristics of patients in different treatment groups in a particular trial.

Baseline balance is not a requirement for obtaining valid variables.


Covariate Adjustment . . .

Imbalance will matter only if characteristic is related to patient outcome, ie., it is prognostic.

When randomization leads to chance baseline imbalance, estimates of treatment effects will be biased when using unadjusted analysis.

Validity and Reliability

Validity

Represents the degree to which a measurement represents a true value.

Reliability

A measure of the reproducibility of a result or observation. ie., How closely do repeated measurements on the same object agree?

Validity and Reliability

Errors can be caused by a lack of either validity or reliability

Validity and reliability are related. If a measure is unreliable it is not capable of producing valid results.

Better reliability is necessary, but not sufficient for validity.

Threats to Study Validity

Bias• Selection• Information• Confounding

Regression to the Mean


Selection Bias

Distortion of effect estimate because of

(i) manner subjects selected: biased sampling,

(ii) selective losses: loss to follow-up and non- responses.

(Case-control studies are particularly susceptible)


Information Bias

Distortion of effect estimate when measurement of exposure or disease is systematically inaccurate.

(i) misclassification bias - incorrect classification of exposure or disease,

(ii) recall bias: accuracy of self-reported data varies across comparison groups.


Other common types of bias:

(i) Investigator/Patient bias

(ii) Measurement error

Many forms of bias can be eliminated or reduced by using randomization and blinding (preferably double-blinding).


Confounding

Type of bias occurring when effect of exposure mixed-up with one or more extraneous variables.

• can create appearance of exposure-disease relationship, when none exists.

• can hide true nature of exposure-disease relationship.


Confounding . . .

• due to presence of key relationships between extraneous variable(s) and both exposure and disease.

• confounding variables are risk factors for the disease, or a correlate of a causal factor.

• confounding variables are associated with exposure of interest.


Confounding . . .Addressing the Problem:

• At the design stage we can use:– Randomization– Matching

• At the analysis stage we can use:– Stratified analysis (Mantel-Haenszel Methods)– logistic regression


Regression to the Mean (RTM)

tendency of extreme observations by chance to move closer to the mean when repeated

What is the danger of ignoring RTM?Causality may erroneously be

inferred


Regression to the Mean (RTM) . . .

Examples:

• Patients having higher than average cholesterol levels at initial screening, have lower levels on repeat.

• Acute pain patients seek help when symptoms severe, and any change is likely to be improvement => useless treatment may appear effective.



RTM is a function of:

• Correlation between pre- and post-treatment values.

• How extreme the pre-treatment values are.



RTM implies that if we select subjects because they appear abnormal on some test, AND

do nothing to them,

they will seem to improve when retested, thus

treatment effects become confounded with RTM.



Four ways to minimize RTM:

1. Increasing reliability of screening test

2. Testing each subject twice, and requiring all tests be extreme before entry into study.

3. Adjust for the correlation of pre/post measures

4. Use a randomized placebo-control trial

Types of Reliability

1. Interobserver – agreement among observers(Kappa, Intraclass correlation)

2. Test-Retest – stability, does the same measure give the same results repeatedly?

3. Parallel form – Two parallel test forms with different items are correlated.

4. Split-half- Split individual measure into two random parts.

Ten Steps forEvaluating Evidence1. Be skeptical

2. Don’t rely on biological plausibility

3. Reliable info requires comparison

4. Ensure cause precedes the effect

5. Post-trial questions maybe unreliable

6. Pre-trial question should be specific and clinically relevant

Ten Steps forEvaluating Evidence . . .7. Discovering small effects requires

randomization

8. Be wary of conflicts of interest

9. Non-specific exposure effects can be important

10. Unblinded examiners may introduce bias.

fundamentals of biostatistics lecture 2 1.clinical trials 2.validity/reliability 3.assessing...

Documents

clinical practice

true clinical endpoint

trt groups validity

intervention group

control group

treatment groups

definitive comparative

cold vaccine trial