Concerns About Adverse Event Reporting in

Randomized Clinical Trials

Yusuf Yazıcı, MD

NYU Hospital for Joint Diseasesyusuf.yazici@nyumc.org

Overview

• Completeness of safety reporting is inadequate • Methodological concerns• Reporting selective results• Errors in reporting

1- Time to event2- SIR3- Adequate sample size4- False impression of safety

Ioannidis J, Lau J. JAMA 2001

Time to AE

• Problems in reporting AE in RCT

• Time to event Clue to causality When to expect AE in clinical care

• Instantaneous risk might be different for drugs but the cumulative risk will be similar

• “Average risk” does not reflect the true risk faced by patients, especially when decisions are being made at the beginning of treatment

Time to AE reporting in RCT

• All RCT of COX-2 inhibitors and TNF inhibitors

91% industry sponsored

1/3 gave time to AE in the report either as a table, text or Kaplan-Meier curve

No better reporting for SAE

• 8/17 RA TNF trials reporting malignancy used SEER database

Time to AE and SAE

SEER database

• Based on annual incidence in population, which is expected to be evenly distributed

• If there is some skewing as to time of malignancies in the treatment arm, this would make comparisons based on an annual incidence inaccurate

• If timing of a malignancy is not evenly distributed then we shouldn’t be using a yearly rate to compare incidence rates

SEER=12 expected lymphomas in RA patients over 12 months, 1 per month

TNF=12 seen in RCT, o Conclusion is that there is no difference

SEER database (2)

• However TNF lymphoma

o 8 in the first 4 months = 2/montho 4 in the last 8 months = 0.5/month

• Does not carry the same message for the practicing physician

• Variable period comparison needs to be used 2,3,4,6 monthly rates

• Excluding events in the first 60-180 days Not done with other AE No strong evidence base for doing so

TNF and lymphomas

• Brown et al, A&R 2002• TNF use and lymphoma development, as

reported to FDA• 26 lymphomas

14 occurred within 2 months of use

• NHL 2 monthly incidence in normal population 3/100,000 10/16 occurred in 2 months 2 monthly NHL occurrence in etanercept

users=10/100,000o Nearly 4 times

Yazici H, A&R 2003

TNF inhibitors and lymphoma

• An increased frequency of lymphomas, tuberculosis and demyelinating CNS disease have been associated with TNF –α antagonist use.

• In the case of lymphoma development, using annual incidence risks in the general population as

comparator is inappropriate development of such lymphomas was within the few months

after drug initiation in more than 50% of these patients (1)

• The expected random occurrence of lymphomas in the general population would make the real risk look smaller (2).

(1) Brown SL et al. Arthritis Rheum 2002; 3151-3158(2) Yazici Y, Yazici H. Arthritis Rheum 2004 Suppl.

Demyelinating CNS disease

• Mohan et al. TNF-α antagonist use and demyelinating CNS disease. 9 cases among 77,152 users of etanercept during a 19

month period. They noted that this frequency was not different than that

found in the general population of 4-6/100 000 per annum.

• Half of the cases within 4 months of etanercept use, 4 monthly incidence risk of etanercept use of 5.8/100,000.

• 4 monthly risk in the general population on the other hand would be 2.0/100,000, if the upper limit, 6.0/100,000, quoted for the general population is used.

Coccidioidomycosis and TNF

• Bergstrom et al. from Tucson, USA, reported (A&R 2004) about the increased risk of coccidioidomycosis in patients treated with TNF-α antagonists.

• They documented 13 cases of coccidioidomycosis associated with TNF therapy.

• Again 7/13 cases developed within 2 months of starting TNF therapy.

TNF and CNS, TB

• The annual incidence risks

• Because more than half of these cases occur during first months of therapy, quarterly or monthly incidence risks in the normal population should be used for comparison.

• Increased incidence risks for TNF–α antagonist related adverse events would necessitate a re-look at the screening protocols and patient selection criteria used for these therapies.

• Finally in the case of lymphomas this approach may also point to a decrease of incidence risk later in disease, consistent with less lymphoma incidence in RA as inflammation is better controlled *

* Baecklund E et al. Arthritis Rheum 2003; 48: 1543-1550.

Infection and Neoplasia with Anti-TNF Agents

Neoplasia

Median: 19 weeks

Time to Neoplasia

Courtesy of H Yazici, “Based on” T Bongartz et al JAMA 2006

Infections

Infections

• Claims database• 20 months follow up period• Hospital records reviewed• Compared TNF based treatment vs. MTX (or other

DMARD-based) treatment• 187/217 identified was reviewed

TNF=65 in 2393 exposed persons (2.7%) DMARD=58 in 2933 exposed persons (2.0%) Number needed to harm=143 Median time from exposure to medication to infection

was <30 days for both groups 93% occurred in 90 days

• Multivariable adjustment ~2 times in TNF vs MTX

Increased early risk

• In the first 6 months 32 in TNF 19 in DMARD

• Adjusted hazard ratio of infection in the first 6 months after start of treatment TNF compared to MTX controls was 4.2 (95% CI 2.0-

88.8)

• Similar to JAMA paper

“Statistics are like swimwear – what they reveal is suggestive but what they conceal is vital.”

Ashish Mahajan, Lancet 2007

Patient years

• Commonly used in RCT: “patient-years” to define the time frame of AE incidence.

• 6 manuscripts were found in the survey.

• Relatively rare idiosyncratic drug reactions usually occur early in the treatment course and in only a few individuals.

• Apart from the few with AEs, remaining patients who are prescribed the drug will never get these reactions however long they use the drug.

• Unduly inflate the denominator of the related incidence ratio; potential of under-representing AE. Late onset AEs are also apt to be missed when we use patient-years.

• In short, only “…when an event is (or is believed to be) likely to occur at any stage during continuous treatment with a drug then an event

rate with a time component (rate per person year, etc) has a true meaning.”

Paterson KR. BMJ. 1995; 310: 1470

Patient years (2)

• Not clear whether more than one event per patient goes into the numerator.

• If more than one AE/patient is included in a numerator, statistics done with that incidence ratio will be erroneous.

• An AE can repeat itself, like skin rashes in TNF-α antagonist use in any one patient.

• This leads to over-representation of the said individual in tests of significance.

“Efficacy and safety” RCT

• Trend of increasing number of RCT called “efficacy and safety” trials.

• All RCT of TNF inhibitors in RA, PsA and AS.

• Only original reports “Efficacy”, “safety” or “efficacy and safety” Power for safety and efficacy Type II error

• Power Alpha Beta Expected treatment effects in both arms

Yazici Y, Adler NM, Yazici H. ACR 2007

“Efficacy”, “safety” and “efficacy and safety”

Powering for safety analysis

• 24/34 (71%) called “efficacy and safety” trials• 5 called such in the title

• 2/24 so called “efficacy and safety” trials were powered to look at safety

• Of the remaining 22, only 3 (14%) included any consideration of type II error as a reason for lack of noted harm

• Of the total 34, only 5 discussed type II error• 9/34 (28%) did not give any power/sample size

calculation for efficacy

Before and after 2004

RCTs

“Efficacy and safety” (continued)

• Designation RCT as “safety” as well as “efficacy” has the potential to give false impressions that lack of AE increase is evidence of overall safety of the medication tested

• Except 1, none of these trials were powered to look at safety

• In addition, RCT are highly selective for lack of major comorbidities So even if they were powered to look at safety, results

need to be interpreted with caution Phase 4 trials

• RCT of course report on AE but these efficacy trials cannot provide any conclusive evidence about safety and these shortcomings should be reported, and required by journals and reviewers

Conclusions

• Time to adverse effecto Causationo What to expect for the clinician

• Patient yearso Falsely give sense of experienceo Rare adverse effects hard to analyze

• SEER• RCT provide information about efficacy and not safety.

They report on AE that happen during a defined time in a small, selected population

“A conclusion is the place where you got tired of thinking”

—Arthur Block