clinical trials jean bourbeau, md respiratory epidemiology and clinical research unit mcgill...

Clinical Trials

Jean Bourbeau, MD

Respiratory Epidemiology and Clinical Research Unit McGill University

Clinical Epidemiology (679)

June 17, 2005

Clinical Trial Objectives

a) Define an experimental study and distinguish the major types

b) Define and distinguish the population involved in planning and conducting experimental studies, and the impact that it might have on generalization and recruitment

c) Discuss the role of randomization in experimental studies, and distinguish individual vs group randomization

d) Define unblinded and blinded studies, distinguish the types of blinded studies, their advantages and disadvantages

e) Define and discuss the consequence of withdrawal

f) Discuss various issues in data analysis

g) Discuss ethical considerations

h) Define and distinguish : efficacy and effectiveness

i) Describe the factors that might influence the response to a treatment or an intervention

Clinical Trial Objectives

Reading

Fletcher, Chapter 7

Study Design

Fundamental point

• Sound scientific clinical investigation almost always demands that a control group be used against which the new intervention can be compared. Randomization is the preferred way of assigning participants to control and intervention groups

Experimental/Clinical trial

Definition:

Prospective study comparing the effect and value of intervention technique(s) against a control in human subjects

Experimental/Clinical trial

•Employ one or more intervention techniques (prophylactic, diagnostic or therapeutic agents, devices, regimens, procedures, etc.);

•Contain a control group against which the intervention group is compared.

Randomized clinical trial

RCT remains the research methodology

of choice whenever randomization is

feasible, but a study’s use of this

methodology does not necessarily confer

certainty on its conclusion

Randomized clinical trial

RCT have become the sine qua non for

the proof of efficacy the Food and Drug

Administration requires for marketing

new drugs

Clinical trial phases (drugs)

Phase I Studies: Pharmaco/Toxicity

–Participants have already tried and failed to improve on the existing standard intervention. –Maximally tolerated dose (MTD).

Phase II Studies: Treatment effect

–Once the MTD is established, the next goal is to evaluate whether the drug has any biologic activity or effect and to estimate the rate of adverse events.

Clinical trial phases (drugs)

Phase III/IV Studies: Full-scale evaluation/Postmarketing surveillance

–Clinical trial (Phase III): generally designed to assess the effectiveness of the new intervention, and thereby, its role in clinical practice.

–Long term studies, which do not involve control groups, are referred to as Phase IV Studies

Types of Clinical TrialRandomized

– Control

Non randomized– Concurrent control– Historical

Others– Cross-over– Withdrawal– Factorial –Group allocation design–Studies of equivalency

Randomized Control Trial

Def : Comparative study with an intervention and a control groups; the assignment of the subject to a group is determined by formal procedure of randomization.

Advantages Disadvantages

• Removes the potential of bias in the allocation of subjects to the study groups

• Tends to balance study groups in covariates

• Guarantees the validity of statistical tests of significance

• Emotional and ethical aspects

• Rare prevalence of a disease

• Complexe, expensive and time-consuming

Non-Randomized Control Trial

Def :

Advantages Disadvantages•Easier to select patient (increased investigator and subject acceptance);

•Prefer to select the control group by means of matching key characteristics;

• Potential bias that the study groups are not strictly comparable.

Comparative study with an intervention and a control group where subjects of either groups are treated at approximately the same time; the assignment is not done by a random process.

Historical Control Trial

Def :

Advantages Disadvantages• All new subjects can receive

the new intervention;

• Easier to select patient (increased investigator and subject acceptance);

• Ethical aspects;

• Rapid and relatively inexpensive.

• Potential bias introduced by time changes in the nature of the patient population, in exposure to pathological agents, or in supportive care and diagnostic criteria;

• Missing data.

Comparative study with an intervention and a control group where a new intervention is used in a series of subjects and the results are compared to the outcome in a previous series of comparable subjects; this type of study is non-randomized and non-concurrent.

Cross-Over Design

Def :

Advantages Disadvantages

• Reduction in variability enables to use smaller sample size to detect a specific difference in treatment response.

• A fairly strict assumption must be made, that the effect of the intervention during the first period must not carry over into the second period;

• Both subjects and disease state change over time; it cannot apply to treatment administered immediately after an acute disease.

Special case of a RCT; it allow each subject to serve as his own control. In the two period cross-over design, each subject will receive either intervention or control in the 1st period and the alternative in the succeeding period; the order in which the intervention and the control treatments are given is randomized

Withdrawal Studies

Def :

Advantages Disadvantages• Evaluate the duration of

benefit of an intervention already known to be useful;

• Assess the efficacy of an intervention that has never conclusively been shown to be beneficial.

• Highly selected sample is evaluated, e.g. only those subjects who physicians thought were benefiting from the intervention were to have been on it and anyone who had a major side effect would have been taken off. Thus it can overestimate benefit and underestimate toxicity.

Withdrawal studies have been conducted in which the subjects on a particular treatment for chronic disease are taken off or have the dosage reduced;

Factorial Design

Def :

Advantages Disadvantages• Can be very informative,

efficient and less expensive. • Possibility of the existence of

interaction and its impact on the sample size;

• Added complexity, impact on compliance or recruitment and potential adverse effects of polypharmacy.

Factorial design attempts to evaluate two interventions compared to control in a single experiment.

Group Allocation Design

Def :

Advantages Disadvantages• May be more acceptable or

decrease the difficulty of approaching people about the idea of randomization

• May not be as efficient as the traditional one (sampling units and the units of analysis are groups)

• If the response rate vary across clinics or groups, efficiency is further decreased.

In group allocation or cluster randomization design, a group of individuals, a clinic, or a community is randomized to a particular intervention or control.

Study of Equivalency

Def :

Advantages Disadvantages• Similar use of treatments with

ease to apply, less adverse effect or cost.

• It cannot be statistically shown that two therapies are identical, as an infinite sample size would be required

• Control or standard treatment have been shown to be effective; that is, truly better than placebo.

In study of equivalency or trial with positive control, the objective is to test whether a new intervention is as good as an established one; the investigator must specify what is meant by equivalence.

Study population

Fundamental point

• The study population should be defined in advance, stating unambiguous inclusion (eligibility) criteria. The impact that these criteria will have on study design, ability to generalize, and participant recruitment must be considered.

Study population

Defining the study population is an integral part of posing the primary question–It is not enough to claim that a treatment is or not effective–The description requires specifications of criteria for subject eligibility

Study population

Population with condition

Studypopulation

Study sample

Population at large

Populationwithout

condition

Definitionof

condition

Entry

criteriaWith

conditionbut ineligible

Eligible butnot enrolled

Enrollment

Eligibility criteria

In general, eligibility criteria relate to participant safety and anticipated effect of

the intervention.

Eligibility criteria

These criteria will have an impact on:

• Study design

• Ability to generalize

• Participant recruitment

Generalization

Generalize to a broader population:

• Study subjects are usually non randomly chosen from the study population, which in turn is defined by eligibility criteria.

• Selective participation in a trial entails the risk that the findings may not be generalizable.

Generalization

It is often forgotten that participants

must agree to enrol in a study:

What sort of person volunteers for a study ?

Why do some agree to participate while

others do not ?

Randomization in Experimental Studies

Fundamental point

Randomization tends to :

• produce study groups comparable with respect to known/unknown risk factors

• remove investigator bias in the allocation of subject

• guarantee that statistical tests will have valid significance levels

Experimental bias

Two forms:

• Selection bias, occurs if the allocation process is predictable

• Accidental bias, can arise if the randomization procedure does not achieve balance on risk factors or prognostic covariates

Types of randomization

Individual randomization

• Simple

• Blocked

• Stratified

Group randomization

Simple randomization

The most elementary form of randomization:

toss an unbiased coin each time a subject is eligible;

use a random number producing algorithm (a more convenient method for large studies).

Blocked randomization

Def :

Advantages Disadvantages

• Avoid imbalance in the number of subjects assigned to each group; this is true particularly if the sample size is small

• If the study is not blinded, the study staff know the assignment for the last person before randomization of that person

Blocked randomization, sometimes called permuted block randomization

Blocked randomization

For example, in the case of blocksize 4, there are 6 possible combinations of group assignments :  

AABB, ABAB, BAAB, BABA, BBAA, and ABBA.

Blocked randomization

For example, another method :

Assignment Random number Rank

  A 0.069 1

A 0.734 3

B 0.867 4

B 0.312 2

Stratified randomization Def :

Advantages Disadvantages• Reduce variability

in group comparison if the stratification is used in the analysis

• Sometime the variables initially though to be most prognostic and therefore used in the stratified randomization turn out to be unimportant

• Other factors may be identified later on to be more important

For any single study, especially a small study, there is no guarantee that all baseline characteristics will be similar in the 2 groups.

Stratified randomization

Age Sex Smoking Hx

1.     40-49 yr 1. Male 1. Current sm.

2.     50-59 yr 2. Female 2. Ex-sm.

3.     60-69 yr

In this example, there will be 18 strata…

Stratified randomization with block size of four

Group randomization

• For some interventions (psychosocial, education, etc) random assignment by individuals can be detrimental, because of the potential risk of interaction among subjects.

• Group of individuals, a clinic or a community are randomized to a particular intervention or control; in this design, the basic sampling units are groups, not subjects.

• Because the basic sampling units are groups, the design is not as efficient as the traditional one

Blindness

Fundamental point• A clinical trial should, ideally, have a double-blind

design to avoid potential problems of bias during data collection and assessment. In a study where such design is not possible, a single-blinded approach and other measures to reduce potential bias are favored

Unblinded and Blinded Studies

Definition

• Bias can occur at a number of places in a clinical study, and it can be caused by conscious factors, subconscious factors, or both

• The general solution to the problem of bias is to keep the subject and the investigator blinded, or masked, to the identity of the assigned intervention

Types of Blinded Studies

Single Blind • Only the investigator is aware of which intervention each

subject is receiving.

Double Blind • Neither the subjects nor the investigators responsible for

following the subjects know the identity of the intervention assignment.

Triple-Blind

• An extension of the double-blind design; the committee monitoring response variables is not told the identity of the groups.

Importance of Blindness in a study

Example :Benefits of the ascorbic acid (vit C) in the common cold

Lewis et al. Ann NY Acad Sci 1975; 258 : 505-12

Participants: medical staff, discovered whether they were on Vit C or placebo

Evaluation: severity and duration self-reported by the participants

Importance of Blindness in a study

Results: • Participants who claimed not to know

the identity of the Rx Vit C not better than placebo 

• Participants who claimed to know

the identity of the Rx Vit C better than placebo

Sample size

Fundamental point

• Clinical trials should have sufficient statistical power to detect differences between groups considered to be of clinical interest. Therefore, calculation of sample size with provision for adequate levels of significance and power is an essential part of planning a trial.

Baseline assessment

Fundamental point

• Relevant baseline data should be measured in all study participants before the start of the intervention

Baseline assessment

Use of baseline data:

• analysis of baseline comparability• stratification and subgrouping• evaluation of change• natural history analysis

Data collection and quality control

Major types of problems:• missing data (one indicator of the quality of the trial)• erroneous data (error will not necessarily be recognized)• variability in the observed characteristics (reduce the opportunity to detect the real changes): random, systematic or combination of both

Adverse Effects in RCT

Difficulties in using clinical trials:

•Most clinical trials are •Too small•Too short duration

to detect uncommon adverse effects

•Patients are very selected (those more likely to develop AE are excluded)

Issues in Data analysis

Fundamental point

• Excluding randomized participants or observed outcomes from analysis on the basis of outcome or response variables can lead to biased results of unknown magnitude or direction

Issues in Data analysis

Exclusion are peoples who are screened as potential participants but who do not meet all of the entry criteria; this will not impact on the internal validity of the study but on the external validity (capacity to generalize)

Withdrawals are participants who have been randomized but are deliberately not included in the analysis; this can bias the results of the study (consequently, the participants remaining may not be comparable)

Reasons for withdrawing

• Ineligibility • Nonadherence or non-compliance• Poor quality or missing data

Ineligible patients • Any trial requires a precise definition of which patients

are eligible for inclusion

• If the proportion of ineligible patients becomes unduly large (these patients have to be detected and reported),

• this may reflect a generally poor standard of study organization;

• It could also indicate that the study’s eligibility criteria were too

restrictive.

• Another issue is whether ineligible patients should be included in the analysis of study results : Intention-to-treat vs Per protocol analysis

Non-compliance

Definition:

Behaviours that are not consistent with health care recommendations

Compliance

Factors that minimize compliance problems :

• Study design; the shorter the study, the more likely subjects are to comply with the intervention;

• Simplicity of the intervention; for example, single dose drug regimens are preferable to multiple dose regimens;

• Subject selection; people likely to follow the study protocol (run-in period can be used);

• A truly informed subject is likely to be better complier (provider-patient interactions are the most consistent determinants).

Compliance

The ultimate in non-compliance is for patients to withdraw totally from the study. Withdrawal from treatment, whatever the reason, should not preclude a patient from subsequent evaluation.

Compliance

One need to draw a distinction between

• non-compliance attributed to lack of patient cooperation or misunderstanding,

• as opposed to cessation or modification of therapy because of adverse reactions or disease progression.

Poor quality or missing data

Participants may be withdrawn from trial because their data are found to be of poor quality, the extreme being missing data.

Comparison of multiple variables

If many significance tests are done, some of

them may be significant by chance alone:

• Repeat look at the same response variable

• Comparison of multiple variables

Comparison of multiple variables

For example:

If an investigator has 100 independent comparisons, 5 of them, on the average, will be significantly different by chance alone ( if the 0.05 level of significance is used)

K comparisons, each comparison should be made at the significance level of α/k

Comparison of multiple variables

Therefore it is more reasonable to calculate sample size based on one primary response variable comparison and be cautious in claiming significanct results for other comparisons.

Efficacy and Effectiveness

Efficacy :

The extent to which a specific intervention, procedure, regimen, or service produce a beneficial result under ideal conditions; ideally, the determination of efficacy is based on the results of a randomized control study .

Efficacy and Effectiveness

Effectiveness

In the usage made standard among epidemiologists by A. L. Cochrane (1909-88), effectiveness is a measure of the extent to which a specific intervention, procedure, regimen, or service, when deployed in the field in routine circumstances, does what it is intended to do for a specified population (Cochrane AL. Effectiveness and efficiency; Random reflections on health service. London : nuffield Provincial Hospitals Trust, 1972).

Efficacy and Effectiveness

Intention-to-treat

A procedure in the conduct and analysis of randomized controlled trial. All patients allocated too each arm of the treatment regimen are analysed together as representing that treatment arm, whether or not they received or completed the prescribed regimen. Failure to follow this step defeats the main purpose of random allocation and can invalidate the results (Newell DJ. Editorial, Int J Epidemiol 1992; 21 : 837-41).

Ethical Considerations

The most serious objections to randomized control studies are ethical issues.

A randomized control study can usually be undertaken when :

• There is uncertainty about the value of a new therapy or dispute about the relative merits of existing therapies.

Although studies might not actually prove the superiority of a new treatment, it can show that either new or existing treatment is valueless or even hazardous. These facts are also important to discover.

Real Treatment effect?

Factors that might influence the response to a treatment or an intervention 

• Natural history of the disease

• Placebo effect

• Effect of participation : Hawthorne effect

• Real treatment or intervention effect

Assessing quality of RCT• Overwhelming evidence indicates that the

quality of reporting RCT is less than optimal

• Bias results from poorly designed and reported trials; mislead medical decision

• Reporting often incomplete, compounding problems arising from poor methodology

Altman et al. Ann Intern Med 2001

Deficiencies in reports RCT

Blind assessment of outcome (1979-80)

30% (67 trials)

Primary endpoint defined (1985)

27% (45 trials)

Sample size calculation

43% (37 trials)

Report method of allocation

Most

Reporting inaccurate

Intention-to-treat but did not analyse all patients

13%

(119 trials)

Inadequate reporting in specialty journals

Common

Supposed RCTs (obstetric and gyneco journals)

5%

(206 trials)

Improving reporting RCTs

The CONSORT statement (consolidated standards or reporting trials)

•The CONSORT statement - JAMA 1996

•The Revised CONSORT Statement.

Ann Intern Med 2001

Trial profile

Ann Intern Med 2001; 134:666

Revised template of the CONSORT (Consolidated Standards of Reporting Trials) diagram showing the flow of participants through each stage of a randomized trial (56-58)

Trial profile

Trial profile based on completion of telephone interviews for evaluation of acute exacerbation's and other health problems and related health

service utilization

Randomization process

Generation

Preparation of the random sequence

Implementation

Enrolling participants

Assessing eligibility

Discussing the trial

Obtaining informed consent

Enrolling patient in trial

Preparation of an allocation

system (such as coded bottles or envelopes), preferably designed to be concealed from the person assigning participants to groups

Ascertaining treatment assignment (such as by opening the next envelope)

Administering intervention

Ann Intern Med 2001; 134:674

Generation and implementation of a

Random Sequence of Treatment:

Baseline characteristics

Example of reporting of Baseline Demographic and Clinical Characteristics of Trial Groups

Ann Intern Med 2001; 134:680

ResultsEnd Point Interventio

Group

(n=30)

Placebo Group

(n=30)

Difference

(95% CI)

P Value

n (%) %

Primary

Achieved psoriatic arthritis response criteria at 12 weeks

Secondary

Proportion of patients meeting ACR criteria

ACR 20

ACR50

ACR70

26 (87)

22 (73)

15 (50)

4 (13)

7(23)

4 (13)

1 (3)

0 (0)

63 (44-83)

60 (40-80)

47 (28-66)

13 ( 1-26)

<0.001

<0.001

<0.001

0.04

Ann Intern Med 2001; 134:682

Example of reporting of Summary Results for Each Study Group

… Discussion• Brief synopsis

•Consideration of possible explanation

•Comparison with relevant findings from other published studies

•Limitations of the study (method used to minimize or compensate)

•Summary of clinical and research implications