critical appraisal - rct

JOURNAL APPRAISALRandomized Controlled Trial (RCT)

Ann Meredith U. Garcia, MD, DPCP

RANDOMIZED CONTROLLED TRIAL (RCT) Type of scientific (often medical) experiment, where the people being

studied are randomly allocated to the different treatments under study

Gold standard for a clinical trial Often used to test the efficacy or effectiveness of various types of

medical interventions and may provide information about adverse effects, such as drug reactions

RANDOMIZED CONTROLLED TRIAL (RCT) Contains control groups, in which groups receiving the experimental

treatment are compared with control groups receiving: No treatment (a placebo-controlled study) Previously tested treatment (a positive-control study)

RANDOMIZED CONTROLLED TRIAL (RCT) Random assignment of intervention is done after subjects have

been assessed for eligibility and recruited, but before the intervention to be studied begins.

After randomization, the two (or more) groups of subjects are followed in exactly the same way. The only differences between the care they receive, for example, in terms

of procedures, tests, outpatient visits, and follow-up calls, should be those intrinsic to the treatments being compared.

The most important advantage of proper randomization is that it minimizes allocation bias, balancing both known and unknown prognostic factors, in the assignment of treatments.

CLASSIFICATION OF RCTs By study design

Parallel-group – each participant is randomly assigned to a group, and all the participants in the group receive (or do not receive) an intervention

Crossover – over time, each participant receives (or does not receive) an intervention in a random sequence

Cluster – pre-existing groups of participants are randomly selected to receive (or not receive) an intervention

Factorial – each participant is randomly assigned to a group that receives a particular combination of interventions or non-interventions

CLASSIFICATION OF RCTs By outcome of interest (efficacy vs. effectiveness)

Explanatory RCTs – test efficacy in a research setting with highly selected participants and under highly controlled conditions

Pragmatic RCTs – test effectiveness in everyday practice with relatively unselected participants and under flexible conditions

By hypothesis (superiority vs. noninferiority vs. equivalence) Superiority trials – one intervention is hypothesized to be superior

to another in a statistically significant way Noninferiority trials – to determine whether a new treatment is no

worse than a reference treatment Equivalence trials – hypothesize that two interventions are

indistinguishable from each other

RANDOMIZATION Advantages of proper randomization:

Eliminates bias in treatment assignment, specifically selection bias and confounding

Facilitates blinding of the identity of treatments from investigators, participants, and assessors

Permits the use of probability theory to express the likelihood that any difference in outcome between treatment groups merely indicates chance

RANDOMIZATION Two processes:

Randomization procedure – to generate an unpredictable sequence of allocations

Allocation concealment – stringent precautions taken to ensure that the group assignment of patients are not revealed prior to definitively allocating them to their respective groups Sequentially numbered, opaque, sealed envelopes (SNOSE) Sequentially numbered containers Pharmacy controlled randomization Central randomization

SAMPLE SIZE Affects reliability

As the sample size increases, a trial may be able to demonstrate a significant effect of the treatment, even if this effect is small.

BLINDING Procedures that prevent study participants, caregivers, or

outcome assessors from knowing which intervention was received

Sometimes inappropriate or impossible to perform in an RCT (e.g., if an RCT involves a treatment in which active participation of the patient is necessary)

IMPORTANT CONSIDERATIONS IN THE ANALYSIS OF RCT DATA “Intention-to-treat analysis” – the extent to which the groups

can be analyzed exactly as they existed upon randomization “Pure" intention-to-treat analysis – possible only when complete

outcome data are available for all randomized subjects When some outcome data are missing, options include analyzing

only cases with known outcomes and using imputed data. The more that analyses can include all participants in the groups to

which they were randomized, the less bias that an RCT will be subject to.

Subgroup analysis – often discouraged because multiple comparisons may produce false positive findings that cannot be confirmed by other studies

LIMITATIONS OF EXTERNAL VALIDITY Factors that can affect RCTs' external validity include:

Where the RCT was performed Characteristics of the patients Study procedures Outcome measures Incomplete reporting of adverse effects of interventions

CLINICAL SCENARIO A 56/F ECOG 1 patient comes to your clinic for second

opinion after being diagnosed with unresectable Stage III pancreatic cancer. Her first doctor advised her to undergo chemotherapy with gemcitabine alone, but she is reluctant to proceed because she has an acquaintance with the same condition who died shortly after receiving a few cycles of the same regimen. She found out on Google that gemcitabine monotherapy is indeed the current standard of care for advanced pancreatic cancer, but she is still interested to know if there are other new treatment options on top of this that may improve survival in patients like her.

CLINICAL QUESTIONAmong adult patients with unresectable pancreatic cancer, are there other treatment options which have been shown to improve survival compared to gemcitabine alone? P: Adult patients with unresectable pancreatic cancer I: Other treatment options vs. gemcitabine alone O: Survival M: RCT

APPRAISING DIRECTNESSClinical Question Research Question

P Adult patients with unresectable pancreatic cancer

Adult patients with locally advanced or metastatic pancreatic adenocarcinoma

I Other treatment options vs. gemcitabine alone

Gemcitabine plus erlotinib (100 or 150 mg/d orally) vs. gemcitabine plus placebo

O Survival Overall survival

M RCT RCT

APPRAISING VALIDITY1. Were patients randomly

assigned to treatment groups?

YES

*Tip: To check if randomization was performed, look for the words ‘randomization’, ‘randomized’ or ‘randomly allocated/assigned’ in the title, abstract or methodology.

APPRAISING VALIDITY2. Was allocation

concealed?

NOT STATED

*Tip: To ascertain allocation concealment, look for a phrase or paragraph stating one of the following: that a clinician recruiting a patient

must contact a remote center to obtain a patient treatment assignment;

that envelopes containing the allocation sequence were sealed and numbered; or

that vehicles or packages containing the medications were indistinguishable.

APPRAISING VALIDITY3. Were baseline

characteristics similar at the start of the trial?

YES

APPRAISING VALIDITY4. Were patients blinded to

treatment assignment?

YES

5. Were caregivers blinded to treatment assignment?

YES

*Tip: To decide if caregivers were blinded in the conduct of the trial, look for use of identical preparations.

APPRAISING VALIDITY6. Were outcome

assessors blinded to treatment assignment?

YES

*Outcome assessors can be the patients themselves, or their caregivers.

*Sometimes, the outcome assessors are study personnel), responsible for deciding how a patient responded to therapy.

APPRAISING VALIDITY7. Were all patients analyzed in the

groups to which they were originally randomized?

YES

*Analyzed in their original treatment group (intention-to-treat analysis), or excluded from the study (censored analysis)

*Tip: To make sure that investigators did not censor non-compliant patients from the analysis, look for the term ‘intention- to-treat’ under the analysis section of the article. If not stated explicitly, look for some indication that patients retained their original assignment even if they were non-compliant. If this is not stated as well, a last resort would be to check that the number of patients randomized is equal to the number of patients analyzed at the end of the study.

APPRAISING VALIDITY8. Was follow-up rate

adequate?

YES

N = 285 N = 284

239 / 282 247 / 280

3 4

(239 + 3) / (282 + 3) = 242 / 285(84.9%)

(247 + 0) / (280 + 4) = 247 / 284(87%)

(239 + 0) / (282 + 3) = 239 / 285(83.9%)

(247 + 4) / (280 + 4) = 251 / 284(88.4%)

APPRAISING VALIDITY

Were patients randomly assigned to treatment groups?

Was allocation concealed? ☐

Were baseline characteristics similar at the start of the trial?

Were patients blinded to treatment assignment?

Were caregivers blinded to treatment assignment?

Were outcome assessors blinded to treatment assignment?

Were all patients analyzed in the groups to which they were originally randomized?

Was follow-up rate adequate?

APPRAISING THE RESULTS

APPRAISING THE RESULTS1. How large was the effect of treatment?

“The final analysis was conducted after 486 deaths (239 on erlotinib and gemcitabine and 247 on placebo and gemcitabine). Overall survival was significantly longer in the erlotinib and gemcitabine arm with an estimated HR of 0.82 (95% CI, 0.69 to 0.99; P = .038; log-rank test stratified for performance status, extent of disease, and pain score at baseline; Fig 1A). Median survival times were 6.24 months versus 5.91 months for the erlotinib and gemcitabine versus placebo and gemcitabine groups with 1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023). A multivariate Cox regression analysis showed that erlotinib treatment (HR, 0.82; 95% CI, 0.69 to 0.99; P = .04) and female sex (P = .03) were significantly associated with longer overall survival. While there was an imbalance in male:female ratio between the arms, the treatment effect remains significant when adjusted for sex. Progression-free survival was significantly longer in the erlotinib and gemcitabine arm than the placebo and gemcitabine arm with an estimated HR of 0.77 (95% CI, 0.64 to 0.92; P = .004; log-rank test stratified for performance status, extent of disease, and pain score at baseline; median, 3.75 months v 3.55 months; Fig 1B).“

`

APPRAISING THE RESULTS2. How precise was the estimate of the treatment effect?

`

`

`

ASSESSING APPLICABILITYSCRAPS (Sex, Co-morbidities, Race, Age,

Pathology, Socioeconomic issues)

INDIVIDUALIZING THE RESULTS

83%(1-year survival rate = 17%)

0.82

INDIVIDUALIZING THE RESULTS

83% X 0.82 = 68.06%

83% – 68.06% = 14.94%

100 / 14.94 = 7