fda's critical path initiative: a perspective on contributions of biostatistics

FDA’s Critical Path Initiative:A Perspective on Contributions of Biostatistics*

R. T. O’Neill**

Office of Biostatistics, OTS/CDER/FDA, 10903 New Hampshire Avenue, Bldg 22, Room 6012,Silver Spring, MD 20993-0002, USA

Received 22 December 2005, accepted 6 April 2006

Summary

This article describes the motivation for, description of, and the objectives and plans for the FDA’sinitiative that was introduced in March of 2004 by way of a report titled ‘Innovation or Stagnation? –Challenge and Opportunity on the Critical Path to New Medical Products’. The FDA initiative is verymuch an outreach effort and a wake-up call to many constituencies to contribute and partner to improvethe product development process and thereby to contribute to the success rate of new products that willbenefit the public. We discuss in general terms where some of the opportunities and challenges existfor the discipline of biostatistics to make contributions to this effort over the next few years. In particu-lar, guidance development in five areas is considered as is the need to devote new energy and efforts toquantitative risk assessment and safety evaluation, an area that has lagged the attention received in theefficacy evaluation area.

Key words: Adaptive study designs; Clinical trial simulation; FDA Critical Path; Guidancedevelopment; Multiple endpoints; Missing data; Non-inferiority trials; Quantita-tive risk/safety assessment; Regulatory biostatistics.

1 Introduction

In March of 2004, the United States Food and Drug Administration issued a report titled ‘Innovationor Stagnation? – Challenge and Opportunity on the Critical Path to New Medical Products’ (FDA,2004). The report contains many messages but the main focus is that there has been a slowdown, notthe expected acceleration in innovative medical therapies reaching patients and that the medical pro-duct development path is becoming increasingly challenging, inefficient and costly. FDA’s view is thatthis situation is critical and we need to act now to improve the development process in many areasand impact public health in a positive way. The message is broad in scope and is intended to engagethe attention of the public, academic researchers, funding agencies and industry. And as a result, thereis challenge to these constituencies to develop and use new tools that would address a variety of needsarticulated under the critical path umbrella.

To be specific, the critical path initiative refers to the medical product development path fromcandidate selection to product launch and production. The ‘critical path initiative’ is a serious attemptto bring attention and focus to the need for more scientific effort and for more publicly availableinformation about the evaluative tools used in product development, namely the techniques and meth-odologies needed to evaluate the safety, efficacy and quality of pharmaceuticals, and other medical

* The views expressed in this article are those of the author and do not necessarily represent the view of the U.S. Food andDrug Administration.

** Corresponding author: e-mail: [email protected], phone: +1 3017960779, fax: +1 3017969880

Biometrical Journal 48 (2006) 4, 559–564 DOI: 10.1002/bimj.200510237 559

# 2006 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim

products as they move down the path from discovery to patient. Thus, the crisis we have today is toimprove upon the science infrastructure behind product development and evaluation.

The FDA Critical Path report implies a sense of urgency to actively move forward, noting that thereis considerable current optimism about drug development that is based upon new biomedical discov-eries such as the sequencing of the human genome, genomic and proteonomic and other ‘nomic’technologies, systems biology, advances in medical imaging, nanotechnology advances, tissue engi-neering, combinatorial chemistry and automated microscale screening in drug discovery. However,despite major increasing investments in research and development over the last ten years, and efficien-cies in the drug review process, new product submissions to FDA have been flat and the success rateof product development has not improved in recent years. In fact, it is estimated that new compoundsentering Phase 1 development today have an 8% chance of reaching the market versus a 14% chance15 years ago; and the Phase III failure rate in now reported to be 50%, versus 20% about 10 yearsago.

After the FDA report was made public, FDA actively sought public input from all constituenciesand the responses we received were universally in support of a change. The responders to the reportcalled for FDA to undertake research, to develop guidances, to initiate collaborations and to conveneconsensus development activities on a wide range of scientific issues. Many responders offered towork with FDA to make the progress needed. Of course the main challenge will be setting prioritiesamong all the possible suggested areas. For this purpose, FDA is aligning its own priorities and sug-gested areas for improvement with those of external constituencies. It is expected that a ‘critical pathlist’ of high priority projects will be published in late 2005.

In this paper I will focus on pharmaceutical products and on the contributions that the field ofbiostatistics can make to improving efficiency of the development and evaluation process, to improv-ing the success rates of new products moving through development, and to the implementation of newtools to reach these objectives. Particular emphasis is placed on methodology and innovative ap-proaches to clinical trials, an area high on the list of where new tools are needed.

1.1 Contributions to the critical path that biostatistics can make

The critical path initiative deals with three major dimensions of the medical product developmentprocess, namely, safety, medical utility (efficacy/benefits) and industrialization (production). In each ofthese areas, the opportunities exist for significant contributions from the biostatistics discipline.

In the area of medical utility, we are of the opinion that there is a real need and opportunity for theFDA statistical program, in conjunction with the industry and academia to conduct research, gainconsensus, and develop guidance to remove obstacles to efficient drug development and to enhancethe success rates of clinical trials. Guidance development and promulgation has long been a mechan-ism to bring predictability to the drug development and review process. But guidance alone is likelynot sufficient. We may also need to revisit the effectiveness of the current process of how planning ofindividual clinical trials occurs, and evaluate how effective the current planning process for use of theaccumulation of information during product development derived from the collection and/or the se-quence of clinical trials. In addition to evaluating how this information is prospectively planned for inearly development, we may also revisit how analyses are planned for, and how the prospective scenar-ios that need to be simulated in advance of actually conducting the development program are consid-ered.

In the area of drug safety, we feel that there is a need to improve the processes and approaches toquantitative analysis of clinical safety data from clinical trials to enhance risk assessment and riskmanagement initiatives. Biostatisticians have not devoted the same attention to the issues of quantita-tive safety assessment as they have to the evaluation of efficacy. So opportunities exist to introducemore efficient and impactful study designs, data collection and data analysis in the study of safety,harms, and benefit to risk evaluations.

560 R. T. O’Neill: FDA’s Critical Path Initiative

# 2006 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.biometrical-journal.com

Finally, in the area of product quality, FDA has placed considerable emphasis on updating theassessment and assurance of product quality and manufacturing. There is a long statistical tradition inquality control but the advance in modern product manufacturing may not be benefiting from any newstatistical thinking in this area. Accordingly, we think that there is a need to improve the statisticalunderstanding and application of modern statistical approaches to product testing and process control.

Let me now focus on the medical utility area and, in particular, on new tools for clinical trials tosupport efficacy and safety claims because that is an area where FDA received the most feedbackfrom the public, the industry and patient groups supporting the notion that the infrastructure for thisarea needs more research, science based standards and collaboration. In addition, the development ofbetter biomarkers and endpoints for clinical trials is another high profile priority that was supportedby the public. There are many aspects of biomarker identification, validation, and linkage to end-points that involve a statistical structure for the study design, data analysis and result interpretation.This area intersects with much of the statistical literature and thinking regarding identification andvalidation of surrogate markers, a more stringent criteria and hurdle than a biomarker identificationand validation.

In addition to the development of new tools to be applied to the development process, the criticalpath initiative encourages greater guidance development by FDA in order to make more predictable aswell as clear the criteria and roadmap for how a product might meet standards for market access.

2 Guidance Development

Guidance development and the promulgation of guidances is one way to develop consensus and bringmore predictability to the process of product development. This has been the motivation for all of theguidances issued by the FDA, by the International Conference on Harmonization of Technical Re-quirements for produce development (ICH), and by the EMEA with its Points to Consider documentsand by other regional guidances that are available. The Office of Biostatistics has identified threeareas for guidance development where it would be beneficial to clarify and to reach consensus on howthe clinical trial community deals with such issues, all of which have strong statistical content.

2.1 Active control non-inferiority clinical trials

One of the most pressing areas is that of the active control non-inferiority clinical trial study design.Non-inferiority active control studies are usually used when placebos can’t be used in a trial as thecontrol group. There is extreme confusion in when to use this design appropriately, in the use ofacceptable statistical methods for design and analysis of this type trial, and in the heterogeneousunderstanding by different medical areas that use these designs to develop evidence for requestingapproval of new products. Because of this situation, we feel the need for more consensus on when touse the design, for how to set the margins for what might be a clinically and statistically acceptablemargin for a treatment effect, and for appropriate statistical data analysis approaches that account foruncertainty in the inferences associated with various study data rich and data poor scenarios availablefor objectively setting margins.

2.2 Multiple endpoints, primary and secondary

Another area critical to the design, interpretation and ultimate success of a clinical trial is the statisti-cal handling of multiple endpoints in clinical trials. There is a need for consensus on how such end-points are used in clinical trials and the statistical criteria that is used in evaluating the importanceand evidence in support of many endpoints, especially when these endpoints are the basis for a claim,or for an additional modification to an already achieved endpoint, thus suggesting an additional bene-fit for a product.

Biometrical Journal 48 (2006) 4 561


There is sufficient experience with the non-inferiority and the multiple endpoint areas that a gui-dance could be developed that would afford more consensus on the issues to be considered at thedesign, analysis and interpretation stage of clinical trials. The many statistical methods and ap-proaches that have been suggested in the literature in recent years now deserve to be digested andplaced in the context of how they can best be used in clinical trials.

2.3 Missing data due to patient dropouts and withdrawal from trials

A third area that remains problematic in most all clinical trials, regardless of the disease area studiedis that of the appropriate statistical handling of data or lack of data due to patient withdrawals anddropouts in clinical trials. Many of the problems relate to the need to enforce follow-up on patients toplanned completion date of the trial after they withdraw from assigned therapy for whatever reason.While there are a variety of statistical methods that have been developed over the last few years,including imputation strategies, the many approaches to modeling missing data, likelihood based meth-ods, mixed model repeated measures (MMRM), pattern-mixture models, selection models and ad hocmethods (LOCF, worst case imputation), these methods generally have not been used in clinical trialanalysis. Moreover, the reporting of results of clinical trials that experience patient withdrawals ordropouts needs to be improved so that an independent assessment can be made of the extent to whichthe conclusions are robust to various assumptions regarding how the missing data or how partialexposure has been handled in the analysis. We feel this area needs more consensus and that it is timeto make that happen under the critical path initiative.

2.4 Adaptive study designs

A topic of current interest in the literature and one that is generating considerable active research isthat of flexible and/or adaptive clinical trial designs that are intended to prospectively plan for datadependent changes during the conduct of the clinical trial in order to improve the success rate oftrials. There are both Bayesian and frequentist strategies for such designs covering a variety of differ-ent objectives such as sample size re-estimation during an interim analysis in order to maintainplanned statistical power or power against a modified alternative hypothesis, selection of a treatmentarm or dropping a treatment arm, and study designs that might enroll a patient subset that is expectedto be enriched in the second stage conditional on having observed a certain sample path in the firststage of a trial. So far, these designs have received more research attention than actual implementationbecause of concern about how the integrity of the trial might be maintained after unblinded access totreatment differences at interim stages which are needed to make the adaptations. As further publicdiscussion is held on these designs that results in increased clarity about what are acceptable prospec-tively planned adaptations and standard operating procedures are put in place, it is likely that thesedesigns will be implemented more then currently. That is one of the goals of the critical path, toexplore new tools that will enhance the success rate of clinical trials.

2.5 Simulation of clinical trials

A tool that has been available to statisticians for many years and which is included in some modernsoftware is clinical trial modeling and simulation. Use of such tools in conjunction with data bases ordata sets that contain information on disease endpoints, disease progression, patient variability andrisk factor or biomarkers should allow for more thorough planning of clinical trials in advance ofactually conducting the trials. These tools can be valuable to challenge and explore deviations fromthe assumptions made for a planned clinical trial, or perhaps for several trials that will be conductedas part of a development program, and these tools can explore before the trial begins the variety ofscenarios that may be possible during the actual conduct of the trial. Such modeling and simulations



as a tool for better design and interpretation of clinical trials can be considered a modern version ofprotocol planning that involves much more from statisticians than sample size calculations and othertraditional study planning inputs.

To accomplish this goal of better protocol planning, there will need to be a major change in howbiostatisticians and other disciplines interact and communicate with each other. Indeed, the improvedplanning derived from modeling and simulation of clinical trials for understanding the implications ofclinical trial design assumptions and for predicting better success of such trials will require a multi-disciplinary team effort and a different dialogue among disciplines. Using data from within or acrossnew drug applications that is available to FDA can provide insight into disease progression, dropoutpatterns, endpoint performance, and decision criteria for a win or for endpoint interpretation. Suchmodeling and simulation exercises at the planning stage of trials are able to explore the impact of thechoice of dose(s), formulation, and patient selection scenarios on the trial’s eventual predicted success,just by asking the key questions in advance of trial conduct.

Most all clinical trial planning questions rely on some model assumptions. For example, for re-peated measured designs it is natural to ask questions such as what is the time course of the assumeddisease progression and what is the anticipated dropout pattern and if the such patterns are informa-tive or not; for time to event designs, it is natural to question how many different endpoints will beused to characterize the clinical effect of the treatment, or what is the relationship of the endpoints tothe disease progression model, or how is the power of the study and therefore the size of the study,determined for various multiple endpoint trial ‘win’ decision criteria.

As interest moves to identifying and using planned biomarker positive subgroups to enrich re-sponse, and to exploring which k out of K clinical endpoints are sensitive or most sensitive to drugeffects, the opportunity exists for more adaptiveness, especially in phase 2 trials. The dose response inboth the safety and efficacy dimensions may need to be considered especially as function of time onstudy and duration of patient exposure so that the patient withdrawals for adverse events or lack ofefficacy may be better planned for and therefore better analyzed.

These are not the only areas that have been identified for which biostatistics can play a role. Otherpotential new areas of involvement for which biostatistics can contribute include development of bio-markers and surrogate endpoints, genomics, proteonomics, and other ‘omic’ areas, datamining andformal incorporation of patient centered evaluations or endpoints (PRO’s).

3 Quantitative Risk Assessment

Another emerging area that crosses both clinical trials and observational studies is that of quantitativerisk assessment. In the pre-marketing clinical trials area, statistical methods and concepts have notbeen a routine aspect of study planning. Generally, there has not been pre-specifed statistical analysisplans in place that might influence how safety data is collected, combined, coded or analyzed at thecompletion of the trial. The consequence of this lack of prospective planning for the collection andanalysis of safety data is a less rigorous and sometimes less sophisticated treatment of the safety datathan is desired. Finally, the efficiency of the exploratory nature of quantitative safety analysis is im-peded because of the lack of standards for clinical trial data collection, and safety data collection inparticular. Standards for clinical trial data collection, such as the Clinical Data Interchange StandardsConsortium (CDISC) [2] effort, are proceeding at a very encouraging pace and the availability ofthese standards will improve how safety data is evaluated and analyzed.

The critical path initiative encourages new tools and methods for all aspects of drug development,and thus we feel there is an opportunity as well as a need to revisit the process of clinical trial safetyplanning and analysis. For this purpose, the Office of Biostatistics formed a statistical safety teamwhose charter is to have more concentrated and focused advice in the safety area. Our goal would beto encourage earlier planning and perhaps modeling and simulation of scenarios for adverse eventoccurrence and more graphical and informative displays for summarizing clinical trial safety data. The

Biometrical Journal 48 (2006) 4 563


current routine approaches of univariate, time independent summaries of event rates would be re-placed by more longitudinal patient profile analysis to evaluate important relationships among events,better quantification of rates and risk factors, especially for serious events. The combining of datafrom patients with different durations of exposure and the role of hazard rates of adverse events wouldreceive more attention. We would begin to understand the impact of bias in detecting increased risksresulting from event misclassification, coding errors, lumping vs. splitting terms, the uncertainties inevent rate characterization derived from grouping of terms in the highly granular Meddra terminologynow used to code safety events in clinical trials.

The Office of Biostatistics is supporting FDA’s ECG Research Database and Toolkit (AssessingSafety) which is being developed. This FDA effort fills the need to improve FDA’s ability to evaluatedrugs for cardiac safety, and therefore to enhance the efficiency of ECG data collection and submis-sion. In this regard, the FDA goal is to develop standards for electronic exchange of ECGs, to developECG research databases and web-based tools, to support development of clear standards and guidancein this area as tools to test and guide development of product safety.

4 Discussion

This paper has attempted to describe certain aspects of FDA’s new ‘critical path initiative’ within thecontext of the contributions that the discipline of biostatistics can make to providing new tools andapproaches to advance the efficiency and success of drug development. FDA’s biostatistics program iscommitted to making progress in the subject areas this paper described earlier. Progress will dependupon the success of partnering with the academic and industry colleagues to address the challengesoutlined in this paper, and on the resources needed to complete the chosen initiatives. FDA is stillrefining its priorities for the many topics that our stakeholders and FDA staff proposed. For the biosta-tistics topics identified, solutions will involve reaching consensus in areas of methodology that arewell researched. Some of the solutions will involve breaking new ground (e.g., modeling and simula-tion and adaptive strategies) and for these we will need to build understanding and expertise. Thereare real challenges in the field of drug product development, and these challenges include statisticalissues that have existed for decades but have not been fully addressed. Along with these challengescomes the opportunity to make new biostatistical contributions and impacts. With so many opportu-nities, the challenge will be in setting priorities.

References

CDISC (2001). http://www.cdisc.org and http://www.fda.gov/cder/regulatory/ersr/ectd.htm.FDA (2004). Innovation or Stagnation? – Challenge and Opportunity on the Critical Path to New Medical Pro-

ducts. http://www.fda.gov/oc/initiatives/criticalpath/.



fda's critical path initiative: a perspective on contributions of biostatistics

Documents